Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track cpu usage metrics for query execution #2140

Open
wants to merge 1 commit into
base: joshua/materialized-query-execution
Choose a base branch
from

Conversation

joshua-spacetime
Copy link
Collaborator

Description of Changes

Tracks the number of rows scanned and the number of index seeks performed during query execution.

It does not track any byte-level metrics since that depends on #2098.

API and ABI breaking changes

None

Expected complexity level and risk

2

Note, this patch depends on #2130.

Computing and recording the metrics themselves was pretty straightforward, however the previous tuple at a time execution model was not suitable for tracking operator-level metrics, hence it was necessary to employ a new execution model.

Testing

A current test was updated to assert on index seek metrics

@joshua-spacetime joshua-spacetime requested a review from jsdt January 17, 2025 21:37
@joshua-spacetime joshua-spacetime self-assigned this Jan 17, 2025
@joshua-spacetime joshua-spacetime force-pushed the joshua/materialized-query-execution branch from bc1fcdc to 816f481 Compare January 17, 2025 22:23
@joshua-spacetime joshua-spacetime force-pushed the joshua/query-billing-metrics branch from f493440 to 5bcf8c8 Compare January 17, 2025 22:23
@joshua-spacetime joshua-spacetime force-pushed the joshua/materialized-query-execution branch from 816f481 to 8a36603 Compare January 17, 2025 22:47
@joshua-spacetime joshua-spacetime force-pushed the joshua/query-billing-metrics branch from 5bcf8c8 to 9ede156 Compare January 17, 2025 22:48
@Centril
Copy link
Contributor

Centril commented Jan 17, 2025

How does this affect the subscription benchmarks?

@joshua-spacetime joshua-spacetime force-pushed the joshua/query-billing-metrics branch from 9ede156 to a776af2 Compare January 17, 2025 23:50
@joshua-spacetime
Copy link
Collaborator Author

@Centril the impact for these is negligible.

full-join        time:   [91.830 µs 92.193 µs 92.510 µs]
                 change: [-1.6552% -1.2958% -0.9618%] (p = 0.00 < 0.05)
                 Change within noise threshold.

However once we introduce "bytes scanned" for example, I fully expect the difference will no longer be negligible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants