Update internationalization docs for 4.0

tutorials/assets_pipeline/importing_translations.rst

@@ -47,7 +47,7 @@ without a `byte order mark <https://en.wikipedia.org/wiki/Byte_order_mark>`__.
 .. warning::
 
     By default, Microsoft Excel will always save CSV files with ANSI encoding
-    rather than UTF-8. There is no built-in way to do this, but there are
+    rather than UTF-8. There is no built-in way to save as UTF-8, but there are
     workarounds as described
     `here <https://stackoverflow.com/questions/4221176/excel-to-csv-with-utf8-encoding>`__.
 

tutorials/i18n/internationalizing_games.rst

@@ -14,13 +14,13 @@ often require localization. Godot offers many tools to make this process
 more straightforward, so this tutorial is more like a collection of
 tips and tricks.
 
-Localization is usually done by specific studios hired for the job and,
-despite the huge amount of software and file formats available for this,
-the most common way to do localization to this day is still with
-spreadsheets. The process of creating the spreadsheets and importing
-them is already covered in the :ref:`doc_importing_translations` tutorial,
-so this one could be seen more like a follow-up to that one.
-
+Localization is usually done by specific studios hired for the job. Despite the
+huge amount of software and file formats available for this, the most common way
+to do localization to this day is still with spreadsheets. The process of
+creating the spreadsheets and importing them is already covered in the
+:ref:`doc_importing_translations` tutorial. If you haven't read the Importing
+translations page before, we recommend you give it a read before reading this
+page.
 
 .. note:: We will be using the official demo as an example; you can
           `download it from the Asset Library <https://godotengine.org/asset-library/asset/134>`_.
@@ -71,35 +71,16 @@ in the current translation, then the text will automatically be translated.
 This automatic translation behavior may be undesirable in certain cases. For
 instance, when using a Label to display a player's name, you most likely don't
 want the player's name to be translated if it matches a translation key. To
-disable automatic translation on a specific node, use
-:ref:`Object.set_message_translation<class_Object_method_set_message_translation>`
-and send a :ref:`Object.notification<class_Object_method_notification>` to update the
-translation::
-
-    func _ready():
-        # This assumes you have a node called "Label" as a child of the node
-        # that has the script attached.
-        var label = get_node("Label")
-        label.set_message_translation(false)
-        label.notification(NOTIFICATION_TRANSLATION_CHANGED)
-
-For more complex UI nodes such as OptionButtons, you may have to use this instead::
-
-    func _ready():
-        var option_button = get_node("OptionButton")
-        option_button.set_message_translation(false)
-        option_button.notification(NOTIFICATION_TRANSLATION_CHANGED)
-        option_button.get_popup().set_message_translation(false)
-        option_button.get_popup().notification(NOTIFICATION_TRANSLATION_CHANGED)
-
-In code, the :ref:`Object.tr() <class_Object_method_tr>`
-function can be used. This will just look up the text in the
-translations and convert it if found:
+disable automatic translation on a specific node, disable **Localization > Auto
+Translate** in the inspector.
+
+In code, the :ref:`Object.tr() <class_Object_method_tr>` function can be used.
+This will just look up the text in the translations and convert it if found:
 
 ::
 
-    level.set_text(tr("LEVEL_5_NAME"))
-    status.set_text(tr("GAME_STATUS_" + str(status_index)))
+    level.text = tr("LEVEL_5_NAME")
+    status.text = tr("GAME_STATUS_%d" % status_index)
 
 .. note::
 
@@ -116,15 +97,109 @@ translations and convert it if found:
     reusability, associate a new a Theme resource to your root Control node and
     define the DynamicFont as the Default Font in the theme.
 
+Placeholders
+^^^^^^^^^^^^
+
+To feature placeholders in your translated strings, use
+:ref:`doc_gdscript_printf` or the equivalent feature in C#. This lets
+translators move the location of the placeholder in the string freely, which
+allows translations to sound more natural. Named placeholders with the
+``String.format()`` function should be used whenever possible, as they also
+allow translators to choose the *order* in which placeholders appear:
+
+::
+
+    # The placeholder's locations can be changed, but not their order.
+    # This will probably not suffice for some target languages.
+    message.text = tr("%s picked up the %s") % ["Ogre", "Sword"]
+
+    # The placeholder's locations and order can be changed.
+    # Additionally, this form gives more context for translators to work with.
+    message.text = tr("{character} picked up the {weapon}").format({character = "Ogre", weapon = "Sword"})
+
+Translation contexts
+^^^^^^^^^^^^^^^^^^^^
+
+If you're using plain English as source strings (rather than message codes
+``LIKE_THIS``), you may run into ambiguities when you have to translate the same
+English string to different strings in certain target languages. You can
+optionally specify a *translation context* to resolve this ambiguity and allow
+target languages to use different strings, even though the source string is
+identical:
+
+::
+
+    # "Close", as in an action (to close something).
+    button.set_text(tr("Close", "Actions"))
+
+    # "Close", as in a distance (opposite of "far").
+    distance_label.set_text(tr("Close", "Distance"))
+
+Pluralization
+^^^^^^^^^^^^^
+
+Most languages require different strings depending on whether an object is in
+singular or plural form. However, hardcoding the "is plural" condition depending
+on whether there is more than 1 object is not valid in all languages.
+
+Some languages have more than two plural forms, and the rules on the number of
+objects required for each plural form vary. Godot offers support for
+*pluralization* so that the target locales can handle this automatically.
+
+Pluralization is meant to be used with positive (or zero) integer numbers only.
+Negative and floating-point values usually represent physical entities for which
+singular and plural don't clearly apply.
+
+::
+
+    var num_apples = 5
+	label.text = tr_n("There is %d apple", "There are %d apples", num_apples) % num_apples
+
+This can be combined with a context if needed:
+
+    var num_jobs = 1
+	label.text = tr_n("%d job", "%d jobs", num_jobs, "Task Manager") % num_jobs
+
+.. note::
+
+    Providing pluralized translations is only supported with
+    :ref:`doc_localization_using_gettext`, not CSV.
+
 Making controls resizable
---------------------------
+-------------------------
 
 The same text in different languages can vary greatly in length. For
 this, make sure to read the tutorial on :ref:`doc_size_and_anchors`, as
 dynamically adjusting control sizes may help.
 :ref:`Container <class_Container>` can be useful, as well as the text wrapping
 options available in :ref:`Label <class_Label>`.
 
+To check whether your UI can accommodate translations with longer strings than
+the original, you can enable *pseudolocalization* in the advanced Project
+Settings. This will replace all your localizable strings with longer versions of
+themselves, while also replacing some characters in the original strings with
+accented versions (while still being readable). Placeholders are kept as-is,
+so that they keep working when pseudolocalization is enabled.
+
+For example, the string ``Hello world, this is %s!`` becomes
+``[Ĥéłłô ŵôŕłd́, ŧh̀íš íš %s!]`` when pseudolocalization is enabled.
+
+While looking strange at first, pseudolocalization has several benefits:
+
+- It lets you spot non-localizable strings quickly, so you can go over them and
+  make them localizable (if it makes sense to do so).
+- It lets you check UI elements that can't fit long strings. Many languages will
+  feature much longer translations than the source text, so it's important to
+  ensure your UI can accommodate longer-than-usual strings.
+- It lets you check whether your font contains all the characters required to
+  support various languages. However, since the goal of pseudolocalization is to
+  keep the original strings readable, it's not an effective test for checking
+  whether a font can support :abbr:`CJK (Chinese, Japanese, Korean)` or
+  right-to-left languages.
+
+The project settings allow you to tune pseudolocalization behavior, so that you
+can disable parts of it if desired.
+
 TranslationServer
 -----------------
 
@@ -200,7 +275,7 @@ Localizing icons and images
 
 Icons with left and right pointing arrows which may need to be reversed for Arabic
 and Hebrew locales, in case they indicate movement or direction (e.g. back/forward
-buttons), otherwise they can remain the same.
+buttons). Otherwise, they can remain the same.
 
 Testing translations
 --------------------
@@ -219,7 +294,7 @@ Keep in mind that since this is a project setting, it will show up in version co
 it is set to a non-empty value. Therefore, it should be set back to an empty value before
 committing changes to version control.
 
-Translations can also be tested when running Godot from the command line.
+Translations can also be tested when :ref:`running Godot from the command line <doc_command_line_tutorial>`.
 For example, to test a game in French, the following argument can be
 supplied:
 

tutorials/i18n/localization_using_gettext.rst

@@ -34,13 +34,6 @@ Disadvantages
   on their system. However, as Godot supports using text-based message files
   (``.po``), translators can test their work without having to install gettext tools.
 
-Caveats
--------
-
-- As Godot uses its own PO file parser behind the scenes
-  (which is more limited than the reference GNU gettext implementation),
-  some features such as pluralization aren't supported.
-
 Installing gettext tools
 ------------------------
 
@@ -59,15 +52,51 @@ install them.
 - **Linux:** On most distributions, install the ``gettext`` package from
   your distribution's package manager.
 
-Creating the PO template (POT) manually
----------------------------------------
+Creating the PO template
+------------------------
+
+Automatic generation using the editor
+-------------------------------------
+
+Since Godot 4.0, the editor can generate a PO template automatically from
+specified scene and script files. This POT generation also supports translation
+contexts and pluralization if used in a script, with the optional second
+argument of ``tr()`` and the ``tr_n()`` method.
+
+Open the Project Settings' **Localization > POT Generation** tab, then use the
+**Add…** button to specify the path to your project's scenes and scripts that
+contain localizable strings:
+
+.. figure:: img/localization_using_gettext_pot_generation.webp
+   :align: center
+   :alt: Creating a PO template in the Localization > POT Generation tab of the Project Settings
+
+   Creating a PO template in the **Localization > POT Generation** tab of the Project Settings
 
-Godot currently doesn't support extracting source strings using ``xgettext``,
-so the ``.pot`` file must be created manually. This file can be placed anywhere
+After adding at least one scene or script, click **Generate POT** in the
+top-right corner, then specify the path to the output file. This file can be
+placed anywhere in the project directory, but it's recommended to keep it in a
+subdirectory such as ``locale``, as each locale will be defined in its own file.
+
+You can then move over to
+:ref:`creating a messages file from a PO template <doc_localization_using_gettext_messages_file>`.
+
+.. note::
+
+    Remember to regenerate the PO template after making any changes to
+    localizable strings, or after adding new scenes or scripts. Otherwise, newly
+    added strings will not be localizable and translators won't be able to
+    update translations for outdated strings.
+
+Manual creation
+---------------
+
+If the automatic generation approach doesn't work out for your needs, you can
+create a PO template by hand in a text editor. This file can be placed anywhere
 in the project directory, but it's recommended to keep it in a subdirectory, as
 each locale will be defined in its own file.
 
-Create a directory named `locale` in the project directory. In this directory,
+Create a directory named ``locale`` in the project directory. In this directory,
 save a file named ``messages.pot`` with the following contents:
 
 ::
@@ -76,50 +105,31 @@ save a file named ``messages.pot`` with the following contents:
     msgid ""
     msgstr ""
 
+    # Example of a regular string.
     msgid "Hello world!"
     msgstr ""
 
+    # Example of a string with pluralization.
+    msgid "There is %d apple."
+    msgid_plural "There are %d apples."
+    msgstr[0] ""
+    msgstr[1] ""
+
+    # Example of a string with a translation context.
+    msgctxt "Actions"
+    msgid "Close"
+    msgstr ""
+
 Messages in gettext are made of ``msgid`` and ``msgstr`` pairs.
 ``msgid`` is the source string (usually in English), ``msgstr`` will be
 the translated string.
 
-The ``msgstr`` value in PO template files (``.pot``) should **always** be empty.
-Localization will be done in the generated ``.po`` files instead.
-
-Creating the PO template (POT) using pybabel
---------------------------------------------
-
-The Python tool pybabel has support for Godot and can be used to automatically
-create and update the POT file from your scene files and scripts.
-
-After installing ``babel`` and ``babel-godot``, for example using pip:
-
-.. code-block:: shell
-
-    pip3 install babel babel-godot
-
-Write a mapping file (for example ``babelrc``) which will indicate which files
-pybabel needs to process (note that we process GDScript as Python, which is
-generally sufficient):
-
-.. code-block:: none
-
-    [python: **.gd]
-    encoding = utf-8
-
-    [godot_scene: **.tscn]
-    encoding = utf-8
-
-You can then run pybabel like so:
-
-.. code-block:: shell
+.. warning::
 
-    pybabel extract -F babelrc -k text -k LineEdit/placeholder_text -k tr -o godot-l10n.pot .
+    The ``msgstr`` value in PO template files (``.pot``) should **always** be
+    empty. Localization will be done in the generated ``.po`` files instead.
 
-Use the ``-k`` option to specify what needs to be extracted. In this case,
-arguments to :ref:`tr() <class_Object_method_tr>` will be translated, as well
-as properties named "text" (commonly used by Control nodes) and LineEdit's
-"placeholder_text" property.
+.. _doc_localization_using_gettext_messages_file:
 
 Creating a messages file from a PO template
 -------------------------------------------