planner: support join elimination rule #8021

lzmhhh123 · 2018-10-23T09:32:49Z

What problem does this PR solve?

support join elimination rule #7704

What is changed and how it works?

Add a logical optimize rule:

outer join elimination: For example left outer join, if the parent only use the columns from left table and the join key of right table(the inner table) is a unique key of the right table. the left outer join can be eliminated.
outer join elimination with distinct: For example left outer join. If the parent only use the columns from left table with 'distinct' label. The left outer join can be eliminated.

Check List

Tests

Unit test
Integration test

Code changes

Has exported function/method change
Has exported variable/fields change

Side effects

Increased code complexity

lzmhhh123 · 2018-10-24T10:29:59Z

/run-all-tests

planner/core/optimizer.go

planner/core/rule_join_elimination.go

into dev/join_elimination

lzmhhh123 · 2018-10-30T05:53:53Z

@zz-jason @winoros PTAL.

planner/core/rule_join_elimination.go

planner/core/optimizer.go

zz-jason · 2018-11-02T10:49:18Z

LGTM

zz-jason · 2018-11-02T10:51:25Z

@winoros @lamxTyler @XuHuaiyu @eurekaka PTAL

planner/core/rule_join_elimination.go

into HEAD

planner/core/rule_join_elimination.go

XuHuaiyu · 2018-11-05T11:57:41Z

Will it be better to not store o.cols and o.schemas,
it seems we can put them as parameters?

planner/core/optimizer.go

planner/core/rule_join_elimination.go

eurekaka · 2018-11-06T04:18:48Z

planner/core/rule_join_elimination.go

+				}
+			}
+			if joinKeysContainIndex {
+				return p.children[1^innerChildIdx]


Should we remove other possible paths in this case?

planner/core/logical_plan_test.go

XuHuaiyu · 2018-11-06T10:15:52Z

Please check whether this would be more readable?

rule_join_elimination.txt

planner/core/rule_join_elimination.go

XuHuaiyu

LGTM

XuHuaiyu · 2018-11-09T03:15:04Z

PTAL @zz-jason @eurekaka

zz-jason · 2018-11-09T08:37:52Z

planner/core/logical_plan_builder.go

 		joinPlan.JoinType = LeftOuterJoin
 		resetNotNullFlag(joinPlan.schema, leftPlan.Schema().Len(), joinPlan.schema.Len())
 	case ast.RightJoin:
+		// right outer join need to be checked elimination
+		b.optFlag = b.optFlag | flagEliminateOuterJoin
 		joinPlan.JoinType = RightOuterJoin
 		resetNotNullFlag(joinPlan.schema, 0, leftPlan.Schema().Len())
 	default:


In the optimization phase, the semi joins are also considered, I think we should add the optimization flag for these semi joins in here.

and please add some explain tests to cover these optimizations.

After my seriously considering. I think semi-joins can't be eliminated in any case. So I will remove semi-join in the eliminating rule.

zz-jason

LGTM

zz-jason · 2018-11-09T11:41:44Z

/run-all-tests

initialization

6820811

lzmhhh123 added type/enhancement The issue or PR belongs to an enhancement. status/WIP sig/planner SIG: Planner labels Oct 23, 2018

lzmhhh123 added 6 commits October 23, 2018 17:49

debug

fa10389

add tests

3394288

Merge branch 'master' into dev/join_elimination

64996b5

Update explain_easy.result

48df0c1

Update explain_easy.result

11fe6a7

debug

05bc4eb

lzmhhh123 removed the status/WIP label Oct 24, 2018

zz-jason reviewed Oct 24, 2018

View reviewed changes

address comment

438fafb

winoros reviewed Oct 29, 2018

View reviewed changes

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

lzmhhh123 added 4 commits October 30, 2018 13:29

address comments

27f2b06

Merge branch 'master' into dev/join_elimination

80b9125

fix

1b4903e

Merge branch 'dev/join_elimination' of https://github.com/lzmhhh123/tidb

9842cad

into dev/join_elimination

zz-jason reviewed Nov 1, 2018

View reviewed changes

lzmhhh123 added 2 commits November 2, 2018 14:12

address comments

53d18e4

address comment

2cb071f

zz-jason reviewed Nov 2, 2018

View reviewed changes

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

zz-jason reviewed Nov 2, 2018

View reviewed changes

planner/core/optimizer.go Outdated Show resolved Hide resolved

lzmhhh123 and others added 2 commits November 2, 2018 17:43

change join eliminating rule order

01649a1

Merge branch 'master' into dev/join_elimination

4b4936d

zz-jason added the status/LGT1 Indicates that a PR has LGTM 1. label Nov 2, 2018

Merge branch 'master' into dev/join_elimination

89a6bb7

winoros reviewed Nov 5, 2018

View reviewed changes

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

lzmhhh123 added 3 commits November 5, 2018 19:15

address

155445e

Merge branch 'dev/join_elimination' of https://github.com/lzmhhh123/tidb

129de34

into HEAD

fix count args

91a5cdb

XuHuaiyu reviewed Nov 5, 2018

View reviewed changes

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

add comments

03ad39a

eurekaka reviewed Nov 6, 2018

View reviewed changes

lzmhhh123 added 2 commits November 6, 2018 13:18

improve

d5a4164

address comments

122f438

ci

3c93712

XuHuaiyu reviewed Nov 6, 2018

View reviewed changes

planner/core/rule_join_elimination.go Show resolved Hide resolved

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved

address comments

8fff6e8

XuHuaiyu reviewed Nov 9, 2018

View reviewed changes

zz-jason reviewed Nov 9, 2018

View reviewed changes

fix

684fd29

zz-jason approved these changes Nov 9, 2018

View reviewed changes

Merge branch 'master' into dev/join_elimination

f1de131

zz-jason added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Nov 9, 2018

Merge branch 'master' into dev/join_elimination

71e757d

zz-jason added the status/all tests passed label Nov 9, 2018

zz-jason merged commit cd5ffb3 into pingcap:master Nov 9, 2018

lzmhhh123 assigned lzmhhh123 and unassigned lzmhhh123 Nov 9, 2018

lzmhhh123 deleted the dev/join_elimination branch November 13, 2018 06:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner: support join elimination rule #8021

planner: support join elimination rule #8021

lzmhhh123 commented Oct 23, 2018

lzmhhh123 commented Oct 24, 2018

lzmhhh123 commented Oct 30, 2018

zz-jason commented Nov 2, 2018

zz-jason commented Nov 2, 2018

XuHuaiyu commented Nov 5, 2018

eurekaka Nov 6, 2018

XuHuaiyu commented Nov 6, 2018

XuHuaiyu left a comment

XuHuaiyu commented Nov 9, 2018

zz-jason Nov 9, 2018

zz-jason Nov 9, 2018

lzmhhh123 Nov 9, 2018

zz-jason left a comment

zz-jason commented Nov 9, 2018

planner: support join elimination rule #8021

planner: support join elimination rule #8021

Conversation

lzmhhh123 commented Oct 23, 2018

What problem does this PR solve?

What is changed and how it works?

Check List

lzmhhh123 commented Oct 24, 2018

lzmhhh123 commented Oct 30, 2018

zz-jason commented Nov 2, 2018

zz-jason commented Nov 2, 2018

XuHuaiyu commented Nov 5, 2018

eurekaka Nov 6, 2018

Choose a reason for hiding this comment

XuHuaiyu commented Nov 6, 2018

XuHuaiyu left a comment

Choose a reason for hiding this comment

XuHuaiyu commented Nov 9, 2018

zz-jason Nov 9, 2018

Choose a reason for hiding this comment

zz-jason Nov 9, 2018

Choose a reason for hiding this comment

lzmhhh123 Nov 9, 2018

Choose a reason for hiding this comment

zz-jason left a comment

Choose a reason for hiding this comment

zz-jason commented Nov 9, 2018