Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry more errors in block storage querier #5558

Merged
merged 5 commits into from
Sep 13, 2023

Conversation

yeya24
Copy link
Contributor

@yeya24 yeya24 commented Sep 12, 2023

What this PR does:

We don't want to retry all 5xxs as in Thanos most of the errors will become Unknown gRPC status code, which becomes 5xx in Cortex. But not all of them are retriable.

This pr adds retry client conn closing error.

Previously only codes.Unavailable got retried. This will cause issues for instant query as we only retry range queries. And the querier retry feature is not fully utilized.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@alanprot
Copy link
Member

LGTM!

…ock storage querier

Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Ben Ye <benye@amazon.com>
@yeya24 yeya24 force-pushed the retry-more-error-querier branch from 2162e96 to e27d3b3 Compare September 13, 2023 16:22
@yeya24 yeya24 merged commit 97effe9 into cortexproject:master Sep 13, 2023
14 checks passed
@yeya24 yeya24 deleted the retry-more-error-querier branch September 13, 2023 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants