-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question regarding benchmark Lingua comparison #9
Comments
Soon I'm going to redo all benchmarks, for an ELD v3, so it is a good opportunity to fix anything that might be incorrect. For lingua I use the same detector for each line, so that is not the problem. I did the benchmarks on a 16GB machine, now I have 32GB. I don't see any problem with memory, it uses ~400mb, not too much really. On windows 10. Have you tried it? Lingua <2.0 against any of the other detectors I tested to see if the performance difference matches? I have not tried Lingua v2, I guess I will for the new benchmarks. |
Yes you are right, that should be more than enough.
Sorry, I hadn't actually tried Lingua < 2.0 yet. But I have compared Lingua 1.3.5 and 2.0.2 now:
So it seems you are right, the performance of Lingua < 2.0 is really not that great. Would really be worth it giving Lingua 2 a try. Footnotes |
I’m redoing the benchmarks for v3, and I’m trying Lingua 2.0.2, what a difference really, with my installation of 1.3.2 I’m seeing a great difference. I’m also using with_preloaded_language_models() and it is reasonably fast now. |
I uploaded ELD v3-beta with the new benchmarks, now Lingua is reasonably fast. I still find discrepancies in their benchmarks, according to them Lingua-low is x2 slower than fasttext, which is fine; I tested x2-x5 depending on the benchmark, but then their test with CLD2 is very similar in speed to fasttext, and I think CLD2 should be >= x2 faster than fasttext. Discussion for v3-beta at: #10 |
Hello,
in your benchmark in the README you got pretty bad performance for Lingua. How exactly do you execute Lingua?
Lingua uses quite large models which have to be loaded once (or lazily during usage), but afterwards detection speed should be quite fast if you keep reusing the same
detector
I think (which is the intended usage). However, if you keep creating newdetector
instances for every detection, then performance will be rather bad. Also, Lingua requires a lot of memory during runtime, so if you are running it in a memory-constrained environment, maybe its performance will not be that good either.Have you tried Lingua version 2 as well1? It is based on the Rust implementation and its performance will likely be better. For measuring performance it might also be useful to:
Thanks for doing this benchmark in the first place though!
Footnotes
That version might also cover more than the 54 languages you mention in the README. ↩
The text was updated successfully, but these errors were encountered: