Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perfomance export #61

Closed
Loruki opened this issue Mar 25, 2021 · 0 comments · Fixed by #69
Closed

Perfomance export #61

Loruki opened this issue Mar 25, 2021 · 0 comments · Fixed by #69

Comments

@Loruki
Copy link
Contributor

Loruki commented Mar 25, 2021

Lors d'un export, le script est très long. Plus de 2 minutes pour 800 lignes et 10 langues.

Il n'y a aucun log qui s'affiche durant cette période ce qui est d'autant plus perturbant.

Ce ticket a pour vocation d'améliorer les perfs de la synchronisation et potentiellement ajouter des logs pour dire que cela fonctionne

felginep added a commit that referenced this issue Aug 25, 2021
For each key, we looped through all the keys already read. This was very unefficient.
Now we use a hash table to store the previous keys.

Results:
- 5 minutes export before optimization
- less than 5 seconds after optimization

What's more, we added some warnings along the way when we see duplicates, or singular / plural mismatch.
@felginep felginep mentioned this issue Aug 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant