New cache management strategy #547
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kentquirk
added
type: enhancement
TylerHelmuth
approved these changes
Nov 1, 2022
There was a problem hiding this comment.
Looks good to me as long as @MikeGoldsmith question is addressed
MikeGoldsmith
approved these changes
Nov 2, 2022
seh
reviewed
Nov 3, 2022
ghost
pushed a commit
to opsramp/tracing-proxy
that referenced
this pull request
Jul 5, 2024
Implements a new cache management strategy that ejects "large" items from the cache rather than resizing the cache. ## Which problem is this PR solving? Currently, when Refinery is under memory pressure and exceeds the configured memory maximum, it attempts to resize the trace cache to 90% of its previous size, and ejects the oldest traces. But as the trace cache is sized by trace count, when a very large trace arrives, it can cause the cache to shrink repeatedly, discarding the smaller traces, which doesn't help much. The result is that the cache can be resized to a tiny fraction of its original size to very little benefit. Furthermore, it never recovers until configuration is manually reloaded. This PR implements a different strategy: * The memory size of individual spans is calculated when they are placed into the cache * Spans also track their arrival time * The "cacheImpact" of a span is a measure of how long the span has been in the cache, multiplied by the size of the span * Traces keep track of the total impact of all the spans in the trace When a memory overrun occurs, the system sorts the traces by cache impact and ejects (makes a sampling decision and drops or sends) those traces with the largest impact until memory usage falls below the maximum. The cache is not resized. This strategy leads to more stable memory usage and fewer overruns. ## Short description of the changes * The memory size of individual spans is calculated when they are placed into the cache * Spans also track their arrival time * The "cacheImpact" of a span is a measure of how long the span has been in the cache, multiplied by the size of the span * Traces keep track of the total impact of all the spans in the trace * There's some new telemetry * There's a config value to control switching between the two modes dynamically and it shows up in the sample config * There are tests for some of the algorithmic calculations as well as the strategy modes Note to reviewers -- this is on the large side, but I didn't see a great way to break it up.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implements a new cache management strategy that ejects "large" items from the cache rather than resizing the cache.
Which problem is this PR solving?
Currently, when Refinery is under memory pressure and exceeds the configured memory maximum, it attempts to resize the trace cache to 90% of its previous size, and ejects the oldest traces. But as the trace cache is sized by trace count, when a very large trace arrives, it can cause the cache to shrink repeatedly, discarding the smaller traces, which doesn't help much. The result is that the cache can be resized to a tiny fraction of its original size to very little benefit.
Furthermore, it never recovers until configuration is manually reloaded.
This PR implements a different strategy:
When a memory overrun occurs, the system sorts the traces by cache impact and ejects (makes a sampling decision and drops or sends) those traces with the largest impact until memory usage falls below the maximum. The cache is not resized.
This strategy leads to more stable memory usage and fewer overruns.
Short description of the changes
Note to reviewers -- this is on the large side, but I didn't see a great way to break it up.