Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add concurrent evaluation for ruler #5766

Merged

Conversation

yeya24
Copy link
Contributor

@yeya24 yeya24 commented Feb 6, 2024

What this PR does:

Upgrade Prometheus version to latest main prometheus/prometheus@6005ac6.

Also added concurrent evaluation support for Ruler. Feature can be enabled via a FF ruler.concurrent-evals-enabled.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Ben Ye <benye@amazon.com>
@friedrichg
Copy link
Member

--- FAIL: TestCompactor_ShouldSkipOutOrOrderBlocks (0.11s)
    compactor_test.go:1069: 
        	Error Trace:	/__w/cortex/cortex/pkg/compactor/compactor_test.go:1069
        	Error:      	Received unexpected error:
        	            	unsorted chunk reference: 9666546658602718315, previous: 17232595470007484901
        	Test:       	TestCompactor_ShouldSkipOutOrOrderBlocks
FAIL

Something is wrong

@yeya24
Copy link
Contributor Author

yeya24 commented Feb 6, 2024

@friedrichg Looks like it is caused by changes in prometheus/prometheus#8085.
We need to fix our tests.

@yeya24
Copy link
Contributor Author

yeya24 commented Feb 8, 2024

With prometheus/prometheus#8085, I think there is no way to create unsorted chunks references when writing index so the tests failed because we fail to create the out of order synthetic data.

I think what I can do is to generate index with those out of order chunk refs using the old TSDB code and put the file under testdata.

Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Ben Ye <benye@amazon.com>
@yeya24 yeya24 requested a review from friedrichg February 9, 2024 18:24
@yeya24 yeya24 merged commit 0a19a7d into cortexproject:master Feb 9, 2024
16 checks passed
@yeya24 yeya24 deleted the update-prometheus-concurrent-rules-eval branch February 9, 2024 22:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants