Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure native histograms counter reset hints are corrected when merging results from different sources #9909

Open
wants to merge 36 commits into
base: main
Choose a base branch
from

Conversation

fionaliao
Copy link
Contributor

@fionaliao fionaliao commented Nov 14, 2024

What this PR does

When merging samples from different iterators, histogram counter reset hints need to be recalculated. The mergeIterator currently does not do so. This PR fixes that by remembering the iterator for a sample in a batch, so when the next sample comes along, the iterators for the two samples can be compared and an unknown counter reset hint is set for the next sample if the iterators don't match. See prometheus/prometheus#15346 for related discussion.

Which issue(s) this PR fixes or relates to

Fixes #

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

pkg/querier/batch/stream.go Outdated Show resolved Hide resolved
pkg/querier/batch/stream.go Outdated Show resolved Hide resolved
@fionaliao fionaliao marked this pull request as ready for review November 26, 2024 21:49
@fionaliao
Copy link
Contributor Author

Implementation and testing is done (pending CI), but I want to extract the mimir-prometheus update into a separate PR before removing the "WIP"

@fionaliao fionaliao changed the title WIP: fix counter resets when merging batches Ensure counter reset hints are corrected when merging batches Nov 27, 2024
@fionaliao
Copy link
Contributor Author

mimir-prometheus update was done as part of #10033

Copy link
Contributor

@krajorama krajorama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good a few nits added first round

pkg/querier/batch/stream.go Outdated Show resolved Hide resolved
pkg/querier/batch/merge_test.go Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
pkg/storage/chunk/prometheus_chunk.go Outdated Show resolved Hide resolved
fionaliao and others added 2 commits November 28, 2024 12:12
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
@fionaliao fionaliao changed the title Ensure counter reset hints are corrected when merging batches Ensure native histograms counter reset hints are corrected when merging results from different sources Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants