Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid using the same scheduler for compaction and flush #4868

Closed
evenyag opened this issue Oct 23, 2024 · 0 comments · Fixed by #4871
Closed

Avoid using the same scheduler for compaction and flush #4868

evenyag opened this issue Oct 23, 2024 · 0 comments · Fixed by #4871
Assignees
Labels
C-enhancement Category Enhancements

Comments

@evenyag
Copy link
Contributor

evenyag commented Oct 23, 2024

What type of enhancement is this?

Refactor, Tech debt reduction

What does the enhancement do?

Although compaction executes in a different runtime, it share the same scheduler with flush jobs. So it may blocks flush jobs as they share the same max_background_jobs limit. #4804 can't fully solved this.

A better approach may be separating schedulers for compaction and flush.

Implementation challenges

We need to add a new scheduler for flush and use different limit to control the concurrency of compaction and flush. We might also refactor how compaction spawns tasks.

@evenyag evenyag added the C-enhancement Category Enhancements label Oct 23, 2024
@evenyag evenyag self-assigned this Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Category Enhancements
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant