Adding experimental flag for Metrics Labels String Interning #6057
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: alanprot <alanprot@gmail.com>
alanprot
requested review from
yeya24 and
friedrichg
and removed request for
yeya24
yeya24
reviewed
Jul 3, 2024
Signed-off-by: alanprot <alanprot@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does:
We implemented an experimental feature to dedup label strings in memory.
When adding a new series, we check if the string is already used by other series and if so, we return the previous reference for the same canonical string. Additionally, a reference count is maintained to determine when a string can be safely removed from memory.
The dedup is done on the
PostCreationandPostDeletionhooks from TSDB.As far as i can tell it seems safe to do so there as those hooks are called sync when creating the new series.
https://github.com/prometheus/prometheus/blob/main/tsdb/head.go#L2026
During initial testing, we observed a notable reduction in HEAP size on the ingesters, ranging from 25% to 30%. No changes on CPU was noticed.
Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]