Manga Translator

Why ?

• Boy read manga.
• Boy reach latest chapter.
• Chapter not translated.
• Boy get mad.
• Boy do code instead of waiting like a normal person.

How ?

• Yolo for bubble recognition and text segmentation
• Open CV for bubble and masking
• PIL for text replacement
• Api's or neural networks for translation
• Deepfillv2 for inpainting and bubble cleanup

Progress

• Bubble inpainting using deepfillv2
• Bubble recognition (should improve with more training data)
• Free text recognition (should improve with more training data)
• Bubble text extraction
• Bubble masking
• Bubble text insertion
• Bubble Text ocr
• Bubble Text translation
• Bubble Text hypenation
• Format and structure dataset
• Create converters i.e. yolo => coco, my dataset => yolo etc
• Detection of text color
• Better free text ocr
• Better free text translation
• Improve text resize algorithm, some texts are too small/big
• Implement lama for cleaning

Models

• Available in the latest release

Install

• setup Anaconda
• clone this repository
• Open a terimal in the cloned repository
• Run the following

conda create -n manga-translator python=3.10
conda activate manga-translator
conda install -c conda-forge poetry

poetry install
# For cuda support run "poe force-cuda"

• Download models to models/modelname (i.e. models/detection.pt)

Usage

UI

poe run-server

CLI

python main.py -f image1.png image2.png

Datasets

Detection

Segmentation

assets/examples

Original	Translated
Japanese => English Jujutsu Kaisen
Japanese => "Meow" Solo Leveling
Japanese => Clean The Rising of the Shield Hero
Japanese => English A Certain Scientific Accelerator
Japanese => English One Punch Man
Japanese => English Oshi No Ko

Glossary

• Bubble: a speech bubble
• Free text: text found on pages but not in speech bubbles
• Bubble Text: text within speech bubbles