Skip to content
This repository has been archived by the owner on Aug 23, 2023. It is now read-only.

smarter bulkindex throttling #215

Closed
Dieterbe opened this issue May 19, 2016 · 1 comment
Closed

smarter bulkindex throttling #215

Dieterbe opened this issue May 19, 2016 · 1 comment

Comments

@Dieterbe
Copy link
Contributor

currently all the addToEs() calls are not throttled, so we can send an arbitrary volume to ES.
there is no backpressure, ES just sometimes errors when its queues are full.
currently we just reschedule those to be retried within 30~60 minutes.
it would be better to throttle ourselves with a backoff mechanism as to not overload ES but also do things as soon as possible
@Dieterbe
Copy link
Contributor Author

Dieterbe commented Feb 9, 2017

fixed by deprecating ES. #504

@Dieterbe Dieterbe closed this as completed Feb 9, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Dieterbe