Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Profile] Visualize the query plan and query profile #5475

Merged
merged 4 commits into from
Mar 21, 2021
Merged

[Profile] Visualize the query plan and query profile #5475

merged 4 commits into from
Mar 21, 2021

Conversation

morningman
Copy link
Contributor

@morningman morningman commented Mar 6, 2021 •
edited
Loading

Proposed changes

Add command:

  1. EXPLAIN GRAPGH SELECT ...
  2. SHOW QUERY PROFILE "..."

Document will be added in next PR.
Details can be seem in #5474

Types of changes

  • New feature (non-breaking change which adds functionality)

Other

I introduced a tree printer class from https://github.com/davidsusu/tree-printer
This is a greet tree printer, but it release version has a NPE bug.
So I can't just add maven dependency of it, but has to add all its source codes in
our repo and fix the bug.

I will also send a PR to davidsusu/tree-printer, and once it merged and release a new version.
I will remove the source code of the tree printer in Doris and use its maven dependency.

Update:
Using tree-printer 1.2 instead of source code

For Reviewer:

The file in package: fe/fe-common/src/main/java/org/apache/doris/common/treeprinter/
are from davidsusu/tree-printer, no need to review.

Add a new statement: SHOW QUERY PROFILE "/";

Files under fe/fe-core/src/main/java/org/apache/doris/common/profile/ are for tree construction
and tree printer.

@morningman morningman self-assigned this Mar 6, 2021
@morningman morningman force-pushed the view_profile branch 2 times, most recently from f198ad7 to 0e7cbd2 Compare March 8, 2021 01:28
@morningman morningman mentioned this pull request Mar 8, 2021
Merged
@EmmyMiao87 EmmyMiao87 self-assigned this Mar 10, 2021
@EmmyMiao87
Copy link
Contributor

Can't his tool directly reference jar? Do we have to buckle the code into our own code base?

@morningman
Copy link
Contributor Author

Can't his tool directly reference jar? Do we have to buckle the code into our own code base?

OK, I will wait it to make to new release.

morningman-cmy and others added 3 commits March 11, 2021 23:00
@morningman-cmy @morningman
[Profile] Support to view query plan and profile in graph
c4f626a 
Add command:
1. EXPLAIN GRAPGH SELECT ...
2. SHOW QUERY PROFILE "..."

Document will be added in next PR

Change-Id: Ifd9365e10b1f9ff4fdf8ae0556343783d97545f0
@morningman
fix bug
f0de52d
@morningman
add tree-printer-1.2
9b54958
@morningman
Copy link
Contributor Author

Can't his tool directly reference jar? Do we have to buckle the code into our own code base?

OK, I will wait it to make to new release.

@EmmyMiao87 Done

@EmmyMiao87 EmmyMiao87 linked an issue Mar 15, 2021 that may be closed by this pull request
[Proposal] Visualize the query plan and query profile #5474
Closed
EmmyMiao87
EmmyMiao87 reviewed Mar 15, 2021
View reviewed changes
fe/fe-core/src/main/cup/sql_parser.cup
RESULT = query;
:}
| describe_command insert_stmt:stmt
{:
stmt.getQueryStmt().setIsExplain(true);
stmt.getQueryStmt().setIsExplain(new ExplainOptions(true, false));
RESULT = stmt;
:}
;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change KW_DESCRIBE to KW_EXPLAIN

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KW_DESCRIBE means both "desc" and "explain",
see fe/fe-core/src/main/jflex/sql_scanner.flex

fe/fe-core/src/main/java/org/apache/doris/analysis/StatementBase.java Outdated
@@ -36,9 +36,7 @@
private String clusterName;

// True if this QueryStmt is the top level query from an EXPLAIN <query>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

fe/fe-core/src/main/java/org/apache/doris/common/profile/PlanTreeBuilder.java Outdated
}

private void buildForPlanNode(PlanNode planNode, PlanTreeNode parent) {
StringBuilder sb = new StringBuilder();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not print the plan node directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

fe/fe-core/src/main/java/org/apache/doris/common/profile/PlanTreeBuilder.java Outdated
PlanTreeNode sinkNode = null;
if (sink != null) {
StringBuilder sb = new StringBuilder();
sb.append("[").append(sink.getExchNodeId().asInt()).append(": DATA SINK]");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DATA SINK or Result Sink

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to sink's class name

fe/fe-core/src/main/java/org/apache/doris/common/profile/PlanTreeBuilder.java Outdated

public class PlanTreeBuilder {

private static final String EXCHANGE_NODE = "EXCHANGE";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use final params in Exchange Node.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this to ExchangeNode

fe/fe-core/src/main/java/org/apache/doris/common/profile/PlanTreeBuilder.java Outdated
public class PlanTreeBuilder {

private static final String EXCHANGE_NODE = "EXCHANGE";
private static final String MERGING_EXCHANGE_NODE = "MERGING-EXCHANGE";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

EmmyMiao87
EmmyMiao87 approved these changes Mar 19, 2021
View reviewed changes
Copy link
Contributor

@EmmyMiao87 EmmyMiao87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@EmmyMiao87 EmmyMiao87 added the area/profile Issues or PRs related to the profile label Mar 19, 2021
@morningman morningman merged commit b9d92e0 into apache:master Mar 21, 2021
EmmyMiao87 pushed a commit to EmmyMiao87/incubator-doris that referenced this pull request Apr 21, 2021
@morningman @morningman-cmy
[Profile] Visualize the query plan and query profile (apache#5475)
7ee758f 
Add command:
1. EXPLAIN GRAPGH SELECT ...
2. SHOW QUERY PROFILE "..."

Document will be added in next PR

Change-Id: Ifd9365e10b1f9ff4fdf8ae0556343783d97545f0
@morningman morningman mentioned this pull request Oct 10, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@EmmyMiao87 EmmyMiao87 EmmyMiao87 approved these changes

Assignees

@morningman morningman

@EmmyMiao87 EmmyMiao87

Labels
area/profile Issues or PRs related to the profile kind/improvement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Proposal] Visualize the query plan and query profile
4 participants
@morningman @EmmyMiao87 @morningman-cmy @xxiao2018