-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[docdb] Add a limit on number of outstanding tablet splits #6667
Comments
One thing we've discussed offline, is that we can do this probably easier on the TS side, to make sure we don't send extra info to the master, about new tablets to split, if we have not finished compacting all the currently being split tablets. Notes from @ttyusupov
|
was discussing with @robertsami , that if we move the control of when to do splits, to the master, then this task will be even easier so @mikhpolitov let's hold off on this a bit, until Rob wraps up the master side work then! |
…ate all split decisions to yb-master Summary: In order to adopt the more complex multi-phase split rate strategy outlined in #8039, we must defer triggering of tablet splits to the master entirely. This revision moves that control to the master and adds a basic test for automatic tablet splitting based on the existing single split size threshold mechanism. On the master side, we inject into the tablet metrics drive info processing to synchronously qualify/disqualify tablets as split candidates. Qualified candidates are added to a queue and split in a background thread in the order they are seen. This approach should lend itself well to future work which will control tablet split rates more intelligently (see #6667). Test Plan: ybd --cxx-test integration-tests_tablet-split-itest --gtest_filter TabletSplitITest.AutomaticTabletSplitting Reviewers: timur Reviewed By: timur Subscribers: zyu, amitanand, azhao, mbautin, mpolitov, ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D11562
…nd delegate all split decisions to yb-master Summary: In order to adopt the more complex multi-phase split rate strategy outlined in yugabyte#8039, we must defer triggering of tablet splits to the master entirely. This revision moves that control to the master and adds a basic test for automatic tablet splitting based on the existing single split size threshold mechanism. On the master side, we inject into the tablet metrics drive info processing to synchronously qualify/disqualify tablets as split candidates. Qualified candidates are added to a queue and split in a background thread in the order they are seen. This approach should lend itself well to future work which will control tablet split rates more intelligently (see yugabyte#6667). Test Plan: ybd --cxx-test integration-tests_tablet-split-itest --gtest_filter TabletSplitITest.AutomaticTabletSplitting Reviewers: timur Reviewed By: timur Subscribers: zyu, amitanand, azhao, mbautin, mpolitov, ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D11562
Summary: Adding in `outstanding_tablet_split_limit` flag to limit the number of outstanding tablet splits that can be processed at the same time. In order to enforce this limit, also adding in a `processing_tablets_` set to `TabletSplitManager`. This set gets added to whenever we start a splittablet operation, and gets removed by either: - when one of its children tablets reports and no longer has any orphaned split data (ie the split was succesful) - or if one of the async tasks to start the split fail, in which case we call `CatalogManager::RemoveFailedProcessingTabletSplit` Test Plan: ``` ybd --cxx-test integration-tests_tablet-split-itest --gtest_filter AutomaticTabletSplitITest.LimitNumberOfOutstandingTabletSplits ``` Reviewers: rsami Reviewed By: rsami Subscribers: ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D12308
…lits Summary: Adding in `outstanding_tablet_split_limit` flag to limit the number of outstanding tablet splits that can be processed at the same time. In order to enforce this limit, also adding in a `processing_tablets_` set to `TabletSplitManager`. This set gets added to whenever we start a splittablet operation, and gets removed by either: - when one of its children tablets reports and no longer has any orphaned split data (ie the split was succesful) - or if one of the async tasks to start the split fail, in which case we call `CatalogManager::RemoveFailedProcessingTabletSplit` Test Plan: ``` ybd --cxx-test integration-tests_tablet-split-itest --gtest_filter AutomaticTabletSplitITest.LimitNumberOfOutstandingTabletSplits ``` Reviewers: rsami Reviewed By: rsami Subscribers: ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D12308
Porting point c) from #6111 (comment)
Once we have #5647, we will have the info in the master, of outstanding tablet splits, so we should also be able to limit how many tablets are still being split, so we could reliably add a limit for this. This would help to minimize the amount of work in parallel in a cluster with many tablets that will end up being split.
The text was updated successfully, but these errors were encountered: