Is there any great idea to handle oov(out of vocab.) problem in implementation?

[Zaoyee]

Hi,

When I develop a language based OCR system, I faced a problem that the recogition fed an image with a unknown token .

etc. Lets say, I develop a french OCR model with french charset which has À Á Â characters, but when recoginition image is based on other lanugage like German character Ä.

In prediction, it could be predicted as any character between À Á Â or even general capital A in good case, however it could be predicted as anything ridiculous, such as character E.

I was wondering if there is great way to handle this instead of developing a Latin system model which needs amount of data in different language.
I understand a couple bad options:

1. discard the data with unknown character
   or
2. put the unknown character into some general unknown token, like <unk>. (ps: It souds great to common sence but it does not make too much sence for classifiction problem in deep learning.)
   or
3. make or A like charaters which do not belong to french charset to the general character A.
   or
   any great ideas?

[Evezerest]

Hi, we are preparing a colab tutorial about how to modify the language dictionary to solve the unknown character problem.
For now, you can follow the recognition model training doc, use your own dictionary including the characters you want to recognize, collect (or synthesize) some data to train a new model