Best practices for multi-lingual indexes

[tacman]

We use machine translation for the 'label' and 'description' properties. We have lots of other fields that are numeric or codes for lookups.

What is the best structure to use for the index? One index per language (duplicating the numeric fields)? A language index for just the label and description properties, then use some sort of federated search for the rest of the properties? Or put the language in as array keys, e.g.

{ 
   'id': 123,
   'label' : {
      'en': 'book',
      'es': 'libro'
   },
   pages: 200,
   author: '@whitman_walt'
}

And this might be pretty basic, but how do I set a language within an index (or document or field)? Gracias.

[ManyTheFish]

Hello @tacman,

Meilisearch automatically detects the language used in each field, so you can have several languages in the same document.
Moreover, for all Latin-based languages, there is no specialized tokenization so this should behave as the same.
The only concern I have is if you mix the Japanese Language and Chinese Language in the same index. These two languages conflict, it's a known issue and the only fix we have so far is to deactivate one of the two Languages. 😔

Unless you mix Chinese and Japanese, if it's easier to put everything in the same index you can do it without issues. ☺️

[tonjohn]

Is there guidance on how to best support localized content?

Currently I'm leaning towards an index per locale (eg product_enUS, product_esMX, etc.) but was wondering if there is an official recommendation.

[tacman]

I went through a similar process, and in the end decided on a _translations key in the record, keyed by language and under that the fields that were translatable. The next version of MeiliSearch will support searching by individual fields, which will prevent the false hits that can happen now. There's a discussion in the MeiliSearch chat on this as well.