Localization road-map

[momokrono]

Adding translations and localization would definitely improve the user experience, allowing people from all over the world to use the app even when they struggle with the English language.
Also, it could be a way to allow even less tech-savvy users to contribute to the project, since creating translations is definitely easier than making PRs and add features.

Initially I wanted to work on it, create a new branch and then make a PR, but I've realized adding translations could be done in several different ways and I don't want to force the project to forcibly adopt my implementation. So I'm here asking for your opinions.

Since the project is built on top of Qt, we could either use Qt-native I18n implementation or a dedicated library/module to do it.
For now I'll just write down what I managed to understand while reading online and I hope people more competent than me in this field could join the discussion. I have no idea how this will impact the compilation (for the binary release), but I don't think it will be a problem.

Qt:

• seamless integration: we could simply call QCoreApplication.translate() or self.tr() and Qt will handle the translations for us.
• .ts, .qm files: these files are used by Qt for the translations, but I've yet to understand how they work. From what I've understood, a .ts file is generated for the app and Qt Linguist (a GUI program) is to be used to translate such file into the different languages. Once this is done, the .ts file is compiled into a .qm file that is loaded at run-time and later used by the app for the translation.

python's gettext:

• native python module, apparently simpler to use compared to the method above
• requires a particular directory structure to handle localization
• .po, .pm files: from what I understood, one file contains the translations while the other is the compiled from the prior and later loaded at run-time.

There are also other libraries, like babel or polib, but I've yet to read and understand if they provide benefits over the two solutions above.

Again, I'm completely new when it comes to localization for python apps, so take everything I say with a grain of salt. But I think for a project like this adding localization would be a great feature and once it's decided how to implement the whole thing I would gladly do it myself.

@theJayTea what do you think? Would you wait for this feature? Or do you think it could be a good one?

[theJayTea]

Hi @momokrono ! This is a noble idea and one that I hoped to implement, thanks so much for this detailed write-up.

I’ll research into this further in a few days and get back to you - I’m currently extremely weak and hospitalised :(

At the end, the implementation should allow for a language dropdown in Settings in the compiled binary, and hopefully automatic locale detection & selection on first launch.

[theJayTea]

Hi @momokrono ! Thank you once again for your willingness to help build this :D

I've researched a bit about this, and I think Python's gettext would be the best option for us.

It's very easy to work with—the .po files are simple to understand and create without external editor programs—which means we can later use an LLM to add translations for most major languages with ease.
We'll just have to feed it the codebase (so it hopefully has context of/understands what the strings refer to/intend) and ask it to output the .po files for major languages. Of course, users could always suggest improvements/fixes to the translations.

Also, it should theoretically work fine with PyInstaller.

I worked with an LLM to create a possible implementation plan. I hope this may help.

---

Implementation Guide: Adding Translations to Writing Tools

1. Create files/folders for this

Add the following to the existing folder structure (TL;DR: add a new locale folder and localization.py file):

writing_tools/
├── locale/
│   ├── en/
│   │   └── LC_MESSAGES/
│   ├── es/
│   │   └── LC_MESSAGES/
│   └── ... (other languages)
├── localization.py
└── ... (existing files & folders)

2. Code Changes

1. Add Translation Import:
   Add to the top of each UI file:

from gettext import gettext as _

2. Wrap Strings:
   Wrap all user-visible strings with _():

# Before
title_label = QtWidgets.QLabel("Settings")
self.shortcut_input.setPlaceholderText("Ask your AI...")

# After
title_label = QtWidgets.QLabel(_("Settings"))
self.shortcut_input.setPlaceholderText(_("Ask your AI..."))

Note: An LLM can help add the _() wrapper to all strings in your Python files - just show it your files and ask it to add the wrapper to all user-visible strings.

3. Add Localization Manager: (example below)

# localization.py

import gettext
import os

class LocalizationManager:
    def __init__(self, app):
        self.app = app
        self.current_locale = "en"
        
    def set_locale(self, locale_code):
        """Set the application locale."""
        locale_path = os.path.join(os.path.dirname(__file__), "locale")
        
        try:
            lang = gettext.translation(
                'base',           # Domain name
                locale_path,      # Locale directory
                languages=[locale_code],
                fallback=True     # Use default strings if translation missing
            )
            lang.install()
            self.current_locale = locale_code
            
            # Save to config
            self.app.config['locale'] = locale_code
            self.app.save_config(self.app.config)
            
            return True
        except Exception as e:
            print(f"Error setting locale: {e}")
            return False

    def get_available_languages(self):
        """Get list of available language codes."""
        locale_path = os.path.join(os.path.dirname(__file__), "locale")
        return [d for d in os.listdir(locale_path) 
                if os.path.isdir(os.path.join(locale_path, d))]

4. Initialize in WritingToolApp: (example)

class WritingToolApp(QtWidgets.QApplication):
    def __init__(self, argv):
        super().__init__(argv)
        self.locale_manager = LocalizationManager(self)
        
        # Set initial locale from config or default
        initial_locale = self.config.get('locale', 'en')
        self.locale_manager.set_locale(initial_locale)

3. Create Translation Files

1. Create the .po files:
   xgettext is a powerful command-line tool that would significantly simplify the process. It can automatically scan your Python files and extract all strings marked with _() into a .po template file.

Just for context, each final .po file for each language would look like this:

msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

msgid "Settings"
msgstr "Configuración"

msgid "Ask your AI..."
msgstr "Pregunta a tu IA..."

# rest of strings

Once we get the template file (with a placeholder for the msgstr translations) with xgettext, we can feed it into an LLM and ask it to add all the translations and create the filled .po file for each of the major languages! We can give the LLM context of the entire codebase so it can hopefully understand what each string intends/means. Of course, users can contribute enhancements/fixes.

2. Compile .po to .mo (do this after any translation changes):

# Run from project root for each language:
msgfmt locale/es/LC_MESSAGES/base.po -o locale/es/LC_MESSAGES/base.mo

4. PyInstaller Update

Add translations to PyInstaller build script:

pyinstaller_command.extend([
    '--add-data', 'locale/*:locale'
])

Key Points

• Translations MUST be compiled (.po → .mo) before distribution
• Use wrapper function _() around all user-visible strings
• PyInstaller needs both .po and .mo files
• Keep strings identical to make updates easier
• An LLM can help:
  • Add _() wrapper to strings
  • Generate initial translations
  • Format .po files

After we get this working, it should be pretty simple to add a dropdown to settings to change the language manually too instead of only automatically reading the system locale.

[momokrono]

@theJayTea I just pushed into the "localization" branch the initial implementation!

I've followed the roadmap you suggested with some changes here and there, for example I didn't need a LocalizationManager at all, I just added a simple function to the WritingToolApp class which then propagates the translation function through the custom components.

Now, I created a helper script that will automatically create the translations files and compile them so that gettext can do its magic.

It's a first implementation, I need to split the script into two, one for the creation of the .po files and one for the compilation into .mo. This way you can create the files, write the translations and then compile. For now you simply run the script once, make the required changes and run it again.

But at least I've automated the whole thing: if you want a new locale you just create the directory, create the LC_MESSAGES sub-dir and the files are created for you. The script checks if a string is passed to the _() function and populates the .po files with it.

One last thing: for now you can setup the locale inside the config.json file, the first time you run the app, it tries with the global locale from your OS and defaults to 'en' otherwise.

If you want to handle the GUI-part, you should just add a button/drop-down menu/I-have-no-idea to set the language and then save the config. Hot-reloading the translations should be trivial, just a call to function that does the language setup.

I've already localized the other windows too (settings, about, custom popup and so on), but the options are still to be localized (the ones you see in the custom popup when you have some text selected). I'll think about how to translate them during these days.

If you don't like the way I've structured the code or implemented it, just let me know.

[theJayTea]

Hi @momokrono !

I'm beyond grateful for your contributions. Thank you so, so much!

Your implementation is absolutely amazing. That script is so smart!

Don't worry about the Settings drop-down GUI, I'll add that.

---

Here's my two cents for the button translations:
We could make it so that only if the buttons are one of our defaults ones ("Proofread", "Rewrite"...), we translate it for the user.

Because if it's not one of the default buttons, the user would have made it themselves — in their language of choice — anyway.

On a technical level, we can simply change our default options.json entries to _("Proofread") instead of simply "Proofread", and so on for all those 8 default options.
And of course, we'll need to add the strings of our 8 default option names to our code/localisation system.

Would this work as easy as I'm imagining haha?
And/or is there something else you have in mind... I'd love to hear what you think!

Also, of course, we don't need to change the default prompts as they anyway instruct the LLM to "respond in the language of the original text" (we could make this all caps for emphasis lol).

---

Before we finally release this version, I'll get the translation files created for most major world languages (with an LLM initially and with the vetting of some volunteers for their languages), and add these languages to the Settings drop-down.

Thank you so much once again for everything :D

[momokrono]

It's been a pleasure, really!

About the options... yes I was thinking the same, translate the default ones and let the user take care of the rest.
Unfortunately we can't just write something like _("Proofread") inside options.json since it's not valid json syntax, and we should most likely work on this piece of code here (inside CustomPopupWindow):

options = []
for key in data:
    if key != "Custom":
        options.append(
            (key,                     # <--- this is what we need to localize
             data[key]["icon"] + ('_dark' if colorMode == 'dark' else '_light') + '.png',
             partial(self.on_generic_instruction, key))
        )

Now, I think the simplest way is to write a wrapper function that checks if the key is one of the default ones, and in this case return the localized version of it, or otherwise return the user-defined one. Probably not the prettiest way to solve it, but it should work just fine.

[theJayTea]

Ah, right! That's a fine implementation haha.

[momokrono]

There, done. Just pushed 😅

[theJayTea]

Wow, thank you so much! The coming week (after a really important exam I have on Monday, so by ~Wednesday), I'll finish making the GUIs, add the translations, some other small features/bug fixes, and we'll have our next release :D
I'll also update our README by then, showcasing your contributions much more prominently.
Thank you again @momokrono!