[v24.1.x] cst/cache: tracker based trim #21502
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* The access time tracker now also tracks file sizes. * The key to the tracker map is now the full file path instead of a hash * The serialized form of tracker now contains a version to distinguish between the old and new data structures. * The recursive directory walker uses tracker entries to avoid file stat operations. (cherry picked from commit d76659a)
When trimming a first attempt is made to get LRU entries from the access tracker. If this does not free up enough space, then we proceed with the existing trim based on directory walk. (cherry picked from commit 63afff2)
(cherry picked from commit e256efb)
The tracker is synced periodically with on-disk data using a directory walk. This is necessary because once we switch to trimming using the tracker, the data in tracker and on-disk may drift because of tmp files, aborted downloads or manual deletion of files. A periodic sync brings the tracker in sync with the metadata on disk. (cherry picked from commit ee90a7b)
(cherry picked from commit c17d79d)
abhijat
approved these changes
Jul 18, 2024
jcipar
approved these changes
Jul 18, 2024
Lazin
approved these changes
Jul 18, 2024
|
Known failure: #20574 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of PR #18786
Fixes: #21501,