Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Batch the update and delete operations in Task Manager #65551

Closed
mikecote opened this issue May 6, 2020 · 2 comments
Closed

Batch the update and delete operations in Task Manager #65551

mikecote opened this issue May 6, 2020 · 2 comments
Assignees
@gmmorris
Labels
Feature:Task Manager Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@mikecote
Copy link
Contributor

mikecote commented May 6, 2020

From @kobelb

This would increase the performance by drastically reducing the likelihood of tasks themselves filling up the Elasticsearch write queue, and lets us execute more tasks concurrently by increasing the concurrent tasks per instance.
@mikecote mikecote added Feature:Task Manager Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels May 6, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

@gmmorris gmmorris self-assigned this Jul 13, 2020
@gmmorris
Copy link
Contributor

It seems the SavedObjectsRepository doesn't yet have support for balk delete, so I'm going to split this into three incremental PRs:

  1. First PR will bulk up the update operation, as we already have support for bulkUpdate and that will also cover most of the work as adding support to these bulk operations in TM is the same which ever operation we choose to focus on first.
  2. The second PR will add bulkDelete to the SavedObjectsRepository, which will require coordination with @elastic/kibana-platform
  3. Finally, we'll use the new bulkDelete api to batch up the remove operation.

It is worth reprioritising parts 2 & 3 in this work, as we do not remove tasks anywhere near as often as we update them (Alerting, for example, reschedules all it's alerts so those tasks are only removed when an alert is disabled or deleted).
I'll bring this up in the sync

@gmmorris gmmorris mentioned this issue Jul 13, 2020
Merged
2 tasks
gmmorris added a commit that referenced this issue Jul 21, 2020
@gmmorris
[Task Manager] Batches the update operations in Task Manager (#71470)
8fdebc9 
This PR attempts to batch update tasks in Task Manager in order to avoid overloading the Elasticsearch queue.
This is the 1st PR addressing #65551

Under the hood we now use a Reactive buffer accumulates all calls to the `update` api in the TaskStore and flushes after 50ms or when as many operations as there are workers have been buffered (whichever comes first).
gmmorris added a commit to gmmorris/kibana that referenced this issue Jul 21, 2020
@gmmorris
[Task Manager] Batches the update operations in Task Manager (elastic…
a342260 
…#71470)

This PR attempts to batch update tasks in Task Manager in order to avoid overloading the Elasticsearch queue.
This is the 1st PR addressing elastic#65551

Under the hood we now use a Reactive buffer accumulates all calls to the `update` api in the TaskStore and flushes after 50ms or when as many operations as there are workers have been buffered (whichever comes first).
gmmorris added a commit that referenced this issue Jul 21, 2020
@gmmorris
[Task Manager] Batches the update operations in Task Manager (#71470) (…
6378647 
…#72626)

This PR attempts to batch update tasks in Task Manager in order to avoid overloading the Elasticsearch queue.
This is the 1st PR addressing #65551

Under the hood we now use a Reactive buffer accumulates all calls to the `update` api in the TaskStore and flushes after 50ms or when as many operations as there are workers have been buffered (whichever comes first).
gmmorris added a commit to gmmorris/kibana that referenced this issue Jul 21, 2020
@gmmorris
[Task Manager] Batches the update operations in Task Manager (elastic…
7bd057e 
…#71470)

This PR attempts to batch update tasks in Task Manager in order to avoid overloading the Elasticsearch queue.
This is the 1st PR addressing elastic#65551

Under the hood we now use a Reactive buffer accumulates all calls to the `update` api in the TaskStore and flushes after 50ms or when as many operations as there are workers have been buffered (whichever comes first).
gmmorris added a commit that referenced this issue Jul 22, 2020
@gmmorris
[7.x] [Task Manager] Batches the update operations in Task Manager (#…
e389968 
…71470) (#72724)

* [Task Manager] Batches the update operations in Task Manager  (#71470)

This PR attempts to batch update tasks in Task Manager in order to avoid overloading the Elasticsearch queue.
This is the 1st PR addressing #65551

Under the hood we now use a Reactive buffer accumulates all calls to the `update` api in the TaskStore and flushes after 50ms or when as many operations as there are workers have been buffered (whichever comes first).

* removed next tick scheduling as we dont use it and its test was too flaky
@mikecote mikecote mentioned this issue Aug 11, 2020
Closed
36 tasks
@mikecote mikecote mentioned this issue Sep 29, 2020
Closed
@kobelb kobelb added the needs-team Issues missing a team label label Jan 31, 2022
@botelastic botelastic bot removed the needs-team Issues missing a team label label Jan 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gmmorris gmmorris

Labels
Feature:Task Manager Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@gmmorris @kobelb @mikecote @elasticmachine