Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: show executed native plan with metrics when in debug mode #746

Merged
merged 2 commits into from
Aug 2, 2024

Conversation

andygrove
Copy link
Member

Which issue does this PR close?

N/A

Rationale for this change

To help debug performance issues, I would like to see the executed DataFusion plan with native metrics.

Example output (goes to executor logs):

Comet native query plan with metrics:
 ProjectionExec: expr=[col_0@0 as col_0, count@1 as col_1], metrics=[start_timestamp{partition=0}=2024-07-31 19:33:43.785691143 UTC, end_timestamp{partition=0}=2024-07-31 19:33:43.883594710 UTC, elapsed_compute{partition=0}=551ns, output_rows{partition=0}=7]
  AggregateExec: mode=Final, gby=[col_0@0 as col_0], aggr=[count], metrics=[start_timestamp{partition=0}=2024-07-31 19:33:43.785670424 UTC, end_timestamp{partition=0}=NONE, elapsed_compute{partition=0}=5.799264ms, output_rows{partition=0}=7]
    ScanExec: schema=[col_0: Utf8, col_1: Int64], metrics=[]

What changes are included in this PR?

How are these changes tested?

Copy link
Contributor

@parthchandra parthchandra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@andygrove andygrove requested a review from huaxingao August 2, 2024 12:48
Copy link
Contributor

@huaxingao huaxingao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the PR @andygrove

@huaxingao huaxingao merged commit 65f22a2 into apache:main Aug 2, 2024
74 checks passed
himadripal pushed a commit to himadripal/datafusion-comet that referenced this pull request Sep 7, 2024
…he#746)

* show executed native plan with metrics when in debug mode

* rename variable

(cherry picked from commit 65f22a2)
@andygrove andygrove deleted the show-native-metrics branch December 3, 2024 04:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants