Manga Translator

Python tool for translating text found within speech bubbles in manga images. It utilizes YOLOv8 for bubble detection, MangaOCR for text extraction, OpenCV, NumPy and Pillow for image processing and translation services like google, bing, baidu and Helsinki-NLP's opus-mt-ja-en model for text translation.

How it works

• Detect speech bubbles in manga images.
• Extract text from detected speech bubbles.
• Clear speech bubbles for better visualization.
• Translate extracted text to the desired language.
• Add translated text back into the speech bubbles.

Dataset

Getting Started

1. Clone this repository:

   git clone https://github.com/georgescutelnicu/Manga-Translator.git

2. Install the required dependencies:

   pip install -r requirements.txt

Command Line Interface

1. Navigate to the directory where you cloned the repository:

   cd Manga-Translator

2. Run the main.py script with the required arguments:

   python main.py --model-path /path/to/model --image-path /path/to/image --font-path /path/to/font --translator [google/hf] --save-path /path/to/save

   Replace the arguments as follows:

   • /path/to/model: Path to the YOLO model.
   • /path/to/image: Path to the input manga image.
   • /path/to/font: Path to the font file for adding text. (default: "fonts/animeace_i.ttf")
   • [google/hf]: Choose the translation service, either "google" for Google Translator or "hf" for Helsinki-NLP. (default: "google")
   • /path/to/save: Path to save the output image.

Flask Web App

1. Navigate to the directory where you cloned the repository:

   cd Manga-Translator

2. Run the Flask app:

   python app.py

3. Open your web browser and go to http://localhost:5000 to access the app.

Demo

Examples

1.

Original Image	Bubbles Detection

Clear Bubbles	Add translation

2.

Original Image	Bubbles Detection

Clear Bubbles	Add translation

Caveats

• Translation Quality: Please note that the translation quality may vary, as we utilize free translation services.
• Potential Solution: Training a translation model or using a paid translation service for improved translation quality.
  
  
• Limited Dataset: The model's performance may be affected by the size and quality of the dataset used for training.
• Potential Solution: Creating a custom dataset.

License

This project is licensed under the MIT License.