storage: add greedy log storage target size control loop monitor #11702
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dotnwat
force-pushed
the
space-management-v2
branch
from
6466ce0 to
ffa743c
Compare
dotnwat
changed the title
jcsp
reviewed
Jun 27, 2023
This is the target size for raft storage. It is approximate in that calculation is both delayed and not expected to be always be precise. We do not use this value to block writes (as is done with actual low free space of the underlying device). Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
This is used by space management control loop. The general usage is to: 1. configure partitions, for example by doing nothing or changing retention configuration. 2. trigger housekeeping with GC prioritized After HK runs the GC prioritization flag is cleared, but callers can repriotize at anytime in the future. Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
When accounting for amount of data reclaimable by gc we will count bytes that are subject to reclaim by the current local retention policy. for cloud enabled topics we want to also estimate how much data in total is reclaimable, and this is effectively how much data is present up to max collectible offset. prior to this commit we were counting this second bucket for non-cloud enabled topics where max collectible offset is roughly the end of the log in most cases. this led to misleading / incorrect stats about how much data was available for reclaiming in gc process. Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Space management control requires a few things: 1. Access segment information for each partition that allows it to reason about how much of a partition should (and can) be truncated to meet a target reclaim size. 2. A mechanism for overriding local retention rules for cloud enabled topics. Data that is uploaded into the cloud, independent of a topics retention settings, can be removed safely on disk. But choose a policy wisely. This commit adds (1) by exposing raw segments from the log, so the caller should be careful. The flip side is that its just expected to look at size information and offsets. For (2) the normal GC path is hijacked when a specific offset is requested. In this case the normal size/time based retention is ignored and the requested offset is used. The requested offset is cleared after GC runs. This makes sense because after the data has been reclaimed reads may rehydrate storage at which point space management may reconsider the data. Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
This placeholder code made subsequent commits pretty messy. Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
dotnwat
force-pushed
the
space-management-v2
branch
from
ffa743c to
44d9d86
Compare
dotnwat
changed the title
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
dotnwat
force-pushed
the
space-management-v2
branch
from
44d9d86 to
0be1c0d
Compare
|
Force-push:
|
jcsp
approved these changes
Jun 28, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a log storage monitor / control loop whose goal is to enforce the target usage size of log storage.
This PR adds a greedy approach to reclaiming cloud-uploaded data by violating local retention policies in an effort to meet the target storage limit.
Backports Required
Release Notes