Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to disable low priority writes for full pruning #5663

Merged
merged 3 commits into from
May 5, 2023

Conversation

asdacap
Copy link
Contributor

@asdacap asdacap commented May 5, 2023

  • Add option to disable low priority writes for full pruning.
  • With 16GB mem budget, reduce full pruning time from 2:10 to 0:48, but will definitely stall block processing.
  • Originally I wanted to have configurable db tune, but it does not do much compared to just disabling low priority writes or letting the ssd cool down and probably just complicate things. Perhaps the semi-ordered insert kinda already reduce write amplification somewhat.
Scenario Low priority write Time Total Write (TB) Block Processing AVG Block Processing P50 Block Processing P90 Block Processing P99
Baseline block processing   -   359 336 539 711
Default yes 2:10 2.30 613 580 860 1260
StableWrite yes 1:30 1.92 898 822 1280 2280
WriteBias yes 0:54 1.67 1680 1130 3470 9470
HeavyWrite yes 0:43 1.54 5000 19200 12500 12500
AggesssiveHeavyWrite yes 0:39 1.08 7410 4190 12500 12500
Default no 0:48 2.00 12500 12500 12500 12500
StableWrite no 0:46 1.80 12500 12500 12500 12500
WriteBias no 0:36 1.68 4370 3440 9550 12500
HeavyWrite no 0:38 1.40 8700 5800 12500 12500
AggressiveHeavyWrite no 0:41 1.05 6710 4370 12500 12500

Changes

  • Add option to disable low priority writes.

Types of changes

What types of changes does your code introduce?

  • New feature (a non-breaking change that adds functionality)

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

@asdacap asdacap requested a review from a team as a code owner May 5, 2023 12:20
Copy link
Contributor

@smartprogrammer93 smartprogrammer93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@asdacap asdacap merged commit d6d657a into master May 5, 2023
@asdacap asdacap deleted the perf/full-pruning-tune-db branch May 5, 2023 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants