Speller: words containing the dictionary separator are not handled properly

[jaumeortola]

We found this bug in LanguageTool using the British English dictionary. See: languagetool-org/languagetool#619

The dictionary has this structure:
<word form><separator><byte containing frequency information A..Z>

When a word like "eta_I" is looked up in the speller, the speller stops working for all the next words. I have written a test here.

The problem is clearly in the method isInDictionary(). Once containsSeparators = false;, it is never initialized to true again and isInDictionary is false for all next words.

An obvious solution is to check if the original word contains the separator and then return false in isInDictionary() even before searching for the word, because it is just impossible to find such a word in the dictionary.

Anyway, I don't understand the logic for the variable containsSeparators, which should be reinitialized to true somewhere. @milekpl

[jaumeortola]

The bug doesn't happen with other separator characters, like "+". So perhaps the issue is related to #85.

[dweiss]

Thanks Jaume! Is there anything else coming or do you want me to publish a point release?

[dweiss]

Went ahead and released 2.1.3, all tests passed.