Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log query requests in QFE before query execution #6390

Merged
merged 2 commits into from
Dec 2, 2024

Conversation

afhassan
Copy link
Contributor

@afhassan afhassan commented Dec 2, 2024

What this PR does:
Add logging for query requests in QFE before query execution. Query requests are logged only if query_stats_enabled is enabled. This is useful for debugging queries that failed to return to QFE.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Ahmed Hassan <afayekhassan@gmail.com>
Signed-off-by: Ahmed Hassan <afayekhassan@gmail.com>
@dosubot dosubot bot added component/query-frontend type/observability To help know what is going on inside Cortex labels Dec 2, 2024
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Dec 2, 2024
@yeya24 yeya24 merged commit 53b0ff3 into cortexproject:master Dec 2, 2024
16 checks passed
@yeya24
Copy link
Contributor

yeya24 commented Dec 2, 2024

I'd like to add a bit more details on why it is useful to log query request before query execution.

Query Frontend only logs query string along with query stats after processing the query request. If a query is expensive to process (for example, with very large response body size), query frontend might get OOM killed and unable to log the full query string, making it hard to debug the issue. Logging it before query execution should help gather the insights by correlating request ID and query string.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/query-frontend lgtm This PR has been approved by a maintainer size/S type/observability To help know what is going on inside Cortex
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants