Support translating plural forms (i18n vol 3)

[ma2ciek]

📝 Provide a description of the new feature

Reasoning

The current implementation of the internalization process doesn't support translating plural forms. Supporting many plural forms is necessary in cases where there's an unknown number of elements like Add %0 bananas and there is an unknown language and not known number and rules of its plural forms. For sure we could write the Add bananas (%0) but this looks much less friendly and IMO it's a little bit distractive.

Required changes an checks

• Check if the plural forms work on the Transifex. If they can be easily downloaded and uploaded. It could be done on a Transifex new demo project.
• Ensure that the upload/download scripts work well for plural forms. Fix them if necessary.
• A function or functions that could handle plural forms or more advanced translation cases for single forms are needed. (ckeditor5-utils)
  • All functions should work if the translation files are missing and should default to the English ones.
  • A function that can return either translation fragments or an interpolated string would be useful in some scenarios, however, this would conflict with the current API.
  • All necessary t() function changes should be backward-compatible.
• A tool that collects translation strings from the t() function needs to support new function(s) too. (ckeditor5-dev)
• Translation files should provide all information needed to the translate() function in ckeditor5-utils. (ckeditor5-dev)
  • a single and all plural forms (an array?) for the given string
  • we could use window.Intl.PluralRules but it is not implemented in Safari, so each translation file should include its own plural rules

Possible further improvements

• upgrade tooling to e.g. gettext-extract , xgettext etc.
• move contexts directly to t() function invocations, remove contexts.json file (?) This would be only valuable if we'd need to have contexts directly in new functions
• make easy adding 3rd-party translations for plugin developers, create a guide for it → issue

Limitations

• Currently, all translation strings need to have a unique id (as all translation strings become the translation ids as well). We could change that if it would be a problem (conflicting translation string) and introduce a way to create unique translation ids (e.g. by adding a translationId parameter to the translation function)
• In-place translations and multi-language translations with short ids won't work anymore, but we want to kill these features. This change would automatically solve these issues:
  • Translations in third party plugins #1471
  • Allow passing your own translations when building CKEditor 5 #763
  • Rebuilding the bundle gives non-deterministic results due to language files #988

API proposal

Translation functions

Proposal 1: Full sentences

Without the function returning intermediate results we could just add an overload to the known t() function:

t( singleForm, pluralForm?, [ ...values ]? ) 

• and do it in a backward-compatible way. This way the plural form could be filled based on the amount of the first value and the rest of placeholders could be filled as well.

The above API could be used in the following way:

t( 'Add a banana', 'Add %0 bananas', [ bananas ] );
t( 'Remove a %1 banana', 'Remove %0 %1 bananas', [ bananas, 'sweet' ] );

Note that concatenating strings could be quite dangerous due to the fact that languages can have various declension rules so we can't do the following:

t( 'Add' ) + ' ' + t( 'a banana', '%0 bananas', [ bananas ] );

The styling should be added directly into the code like as the t() function produces strings and there's no way to determine later which part should be be bolded. The bolded text could be done only this way:

t( '*Add* a *banana*', '*Add* %0 *bananas*', [ bananas ] );

So after the translation phases these styling could be e.g. wrapped in bold.

Proposal 2 - Add additional support for defining translation ids

Add contexts/translation ids to translation strings so some translations can be reused without worrying about different declension rules.

E.g.:

t( 'Add' ) + ' ' + ct( 'Add %0 bananas', 'a banana', '%0 bananas', [ bananas ] );

Pluses

• less number of translations (especially for languages with many declensions.

Minuses

• one or two translation functions more to handle in each translation step
• inconsistency between ct and t functions - in the ct function we'd need to use contexts as translation id or add a unique id as the first argument
• more work

Proposal 3 - return an object instead of a string for more customizable usages

Create a function that returns an object with translation fragments

e.g.:

const translation = ct( 'Add %0 bananas', 'a banana', '%0 bananas', [ bananas ] );
    
translation.toString() // -> 'Add 4 bananas'
translation.toParts(); // ->
// [ { type: 'literal', value: 'Add ' },
// { type: 'value', value: 4 },
// { type: 'literal', value: ' bananas' } ]

Pluses

• more customizable translations, no need for asterisks in the code to generate bold afterward

Minuses

• an API much different than the t() function API while the function name seems to be familiar
• 2 more dedicated functions (one for single forms and one for plural forms).
• complicated code

With the toString() function defined, we could also use it indirectly in the string concatenations as we use the t() function currently. However such a not-explicit way seems a little bit scary.

Translation files

The translation files need to have defined translation rules like:

"Plural-Forms: nplurals=3; plural=(n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<12 || n%100>14) ? 1 : 2);\\n"

They also need to include these plural forms:

{
    '*Add* a *banana*': [ 'Dodaj banana', 'Dodaj %0 banana', 'Dodaj %0 banany', 'Dodaj %0 bananów' ]
}

Refs

• https://docs.transifex.com/localization-tips-workflows/plurals-and-genders
• https://docs.transifex.com/formats/gettext#plural-forms-in-a-po-file
• https://docs.transifex.com/formats/introduction#plurals

cc @Reinmar @scofalik @mlewand

---

If you'd like to see this feature implemented, add a 👍 reaction to this post.

[ma2ciek]

Extending the 2. proposal we could add the support for the t( translationOptions, values ) instead of adding more params to the t() function, with the following interface for translationOptions:

interface TranslationOptions {
   // A string id that would be used for translation uniqueness.
   // The value defaults to the `string`'s value if it isn't set.
   // We could also use it as a translation context to not define two similar things.
   // This way that string could be also used by translators.
   // E.g. `'Show/Hide %0 editors'`.
   id?: string;

   // A translation string. Required. E.g. `'editor'`.
   string: string;

   // A plural form of the translation. E.g. `'%0 editors'`.
   // It can be easily ommited if the translation will not have plural forms.
   plural?: string
}

This way we could still use the t() function but provide 2 overloads.

t( translationString, [ ...values ] ) would be an equivalent to t( { string: translationString, id: translationString }, [ ...values ] ) but the second API would provide options for more advanced usages.

One more advantage is that this API is open-ended for future enhancements. In contrast, the previously proposed APIs would be impossible to change in the future without introducing breaking changes.

I've checked and both, parsing (when looking for translation strings and other things) and obfuscation should work for this API.

[ma2ciek]

As an update, I'll write that I chose the above API as it seems the most flexible, extendible and descriptive and uses the old t() function that everyone already knows.