explaintest: add explain test for partition pruning #7505

tiancaiamao · 2018-08-27T13:40:59Z

What problem does this PR solve?

Add a explain test for partition pruning.

What is changed and how it works?

TiDB explain result is different from mysql, so we have the explain test.

There are still many SQL statements that could take the advantage of partition pruning in MySQL,
but can't be pruned in TiDB. It's a known issue and I'll resolve it later.

Some statement doesn't work in TiDB as MySQL:

show status like 'Handler_read_rnd_next';
show status like 'Handler_read_key';

The result is generated using command:

./run-tests.sh -r partition_pruning

I manually adjust some test input adding order by as partitioned table doesn't keep result order.

Check List

Tests

Integration test
and
Manual test (add detailed scripts or steps below)

@zz-jason @coocood

shenli · 2018-08-28T01:44:12Z

Is there any issue for this:

There are still many SQL statements that could take the advantage of partition pruning in MySQL,
but can't be pruned in TiDB. It's a known issue and I'll resolve it later.

shenli · 2018-08-28T01:45:13Z

plan/explain.go

@@ -34,6 +34,12 @@ func (p *PhysicalIndexScan) ExplainInfo() string {
 		tblName = p.TableAsName.O
 	}
 	fmt.Fprintf(buffer, "table:%s", tblName)
+	if p.isPartition {
+		if pi := p.Table.GetPartitionInfo(); pi != nil {


Is there any chance that pi is nil?

pi should never be nil here...

shenli · 2018-08-28T01:46:03Z

model/model.go

+			return def.Name.L
+		}
+	}
+	return ""


Should we return error here?

Not necessary.
The only place we need to know the partition name is explain, while explain doesn't care GetNameByID fail.

Can it be changed to notFind? empty strings that don't make sense are bad.

empty string means we have to check the code and fix bug, notFind gives no more information than that.

table: t, partition:

It's a bug!

table: t, partition: notFind

It's a bug!

tiancaiamao · 2018-08-28T07:22:59Z

I've make a issue #7516 for this. @shenli

alivxxx

LGTM

ciscoxll

LGTM.

zz-jason · 2018-10-11T09:37:49Z

@tiancaiamao Please resolve conflicts.

tiancaiamao · 2018-10-11T12:54:17Z

/run-all-tests

zz-jason · 2018-10-11T12:55:34Z

LGTM

zz-jason · 2018-10-11T12:59:27Z

@tiancaiamao please fix ci

tiancaiamao · 2018-10-11T14:36:03Z

/run-all-tests

tiancaiamao · 2018-10-11T15:01:29Z

/run-all-tests

tiancaiamao · 2018-10-11T15:03:46Z

/run-all-tests

tiancaiamao · 2018-10-12T12:04:19Z

@zz-jason

bugfix fixed pingcap#7518 expression: MySQL compatible current_user function (pingcap#7801) plan: propagate constant over outer join (pingcap#7794) - extract `outerCol = const` from join conditions and filter conditions, substitute `outerCol` in join conditions with `const`; - extract `outerCol = innerCol` from join conditions, derive new join conditions based on this column equal condition and `outerCol` related expressions in join conditions and filter conditions; util/timeutil: fix data race caused by forgetting set stats lease to 0 (pingcap#7901) stats: handle ddl event for partition table (pingcap#7903) plan: implement Operand and Pattern of cascades planner. (pingcap#7910) planner: not convert to TableDual if empty range is derived from deferred constants (pingcap#7808) plan: move projEliminate behind aggEliminate (pingcap#7909) admin: fix admin check table bug of byte compare (pingcap#7887) * admin: remove reflect deepEqual stats: fix panic caused by empty histogram (pingcap#7912) plan: fix panic caused by empty schema of LogicalTableDual (pingcap#7906) * fix drop view if exist error (pingcap#7833) executor: refine `explain analyze` (pingcap#7888) executor: add an variable to compatible with MySQL insert for OGG (pingcap#7863) expression: maintain `DeferredExpr` in aggressive constant folding. (pingcap#7915) stats: fix histogram boundaries overflow error (pingcap#7883) ddl:support the definition of `null` change to `not null` using `alter table` (pingcap#7771) * ddl:support the definition of null change to not null using alter table ddl: add check when create table with foreign key. (pingcap#7885) * ddl: add check when create table with foreign key planner: eliminate if null on non null column (pingcap#7924) executor: fix a bug in point get (pingcap#7934) planner, executor: refine ColumnPrune for LogicalUnionAll (pingcap#7930) executor: fix panic when limit is too large (pingcap#7936) ddl: add TiDB version to metrics (pingcap#7902) stats: limit the length of sample values (pingcap#7931) vendor: update tipb (pingcap#7893) planner: support the Group and GroupExpr for the cascades planner (pingcap#7917) store/tikv: log more information when other err occurs (pingcap#7948) types: fix date time parse (pingcap#7933) ddl: just print error message when ddl job is normal to calcel, to eliminate noisy log (pingcap#7875) stats: update delta info for partition table (pingcap#7947) explaintest: add explain test for partition pruning (pingcap#7505) util: move disjoint set to util package (pingcap#7950) util: add PreAlloc4Row and Insert for Chunk and List (pingcap#7916) executor: add the slow log for commit (pingcap#7951) expression: add builtin json_keys (pingcap#7776) privilege: add USAGE in `show grants` for mysql compatibility (pingcap#7955) ddl: fix invailid ddl job panic (pingcap#7940) *: move ast.NewValueExpr to standalone parser_driver package (pingcap#7952) Make the ast package get rid of the dependency of types.Datum server: allow cors http request (pingcap#7939) *: move `Statement` and `RecordSet` from ast to sqlexec package (pingcap#7970) pr suggestion update executor/aggfuncs: split unit tests to corresponding file (pingcap#7993) store/tikv: fix typo (pingcap#7990) executor, planner: clone proj schema for different children in buildProj4Union (pingcap#7999) executor: let information_schema be the first database in ShowDatabases (pingcap#7938) stats: use local feedback for partition table (pingcap#7963) executor: add unit test for aggfuncs (pingcap#7966) server: add log for binary execute statement (pingcap#7987) admin: refine admin check decoder (pingcap#7862) executor: improve wide table insert & update performance (pingcap#7935) ddl: fix reassigned partition id in `truncate table` does not take effect (pingcap#7919) fix reassigned partition id in truncate table does not take effect add changelog for 2.1.0 rc4 (pingcap#8020) *: make parser package dependency as small as possible (pingcap#7989) parser: support `:=` in the `set` syntax (pingcap#8018) According to MySQL document, `set` use the = assignment operator, but the := assignment operator is also permitted stats: garbage collect stats for partition table (pingcap#7962) docs: add the proposal for the column pool (pingcap#7988) expression: refine built-in func truncate to support uint arg (pingcap#8000) stats: support show stats for partition table (pingcap#8023) stats: update error rate for partition table (pingcap#8022) stats: fix estimation for out of range point queries (pingcap#8015) *: move parser to a separate repository (pingcap#8036) executor: fix wrong result when index join on union scan. (pingcap#8031) Do not modify Plan of dataReaderBuilder directly, because it would impact next batch of outer rows, as well as other concurrent inner workers. Instead, build a local child builder to store the child plan. planner: fix a panic of a cached prepared statement with IndexScan (pingcap#8017) *: fix the issue of executing DDL after executing SQL failure in txn (pingcap#8044) * ddl, executor: fix the issue of executing DDL after executing SQL failure in txn add unit test remove debug info add like evaluator case sensitive test ddl, domain: make schema correct after canceling jobs (pingcap#7997) unit test fix code format proposal: maintaining histograms in plan. (pingcap#7605) support _tidb_rowid for table scan range (pingcap#8047) var rename fix

explaintest: add explain test for partition pruning

b7d0ebd

tiancaiamao added the component/test label Aug 27, 2018

shenli reviewed Aug 28, 2018

View reviewed changes

alivxxx reviewed Aug 31, 2018

View reviewed changes

morgo mentioned this pull request Sep 15, 2018

FR: EXPLAIN on partitioned table should describe partitions #7523

Closed

ciscoxll reviewed Oct 11, 2018

View reviewed changes

zz-jason added the status/LGT2 Indicates that a PR has LGTM 2. label Oct 11, 2018

tiancaiamao added 2 commits October 11, 2018 20:19

Merge branch 'master' into explain-partition-pruning

f963156

address comment

e5b98d6

zz-jason previously approved these changes Oct 11, 2018

View reviewed changes

Merge branch 'master' into explain-partition-pruning

2c7d6ed

make golint happy

b5be88c

tiancaiamao dismissed zz-jason’s stale review via b5be88c October 11, 2018 15:03

tiancaiamao added the status/all tests passed label Oct 12, 2018

Merge branch 'master' into explain-partition-pruning

bfb5ce2

zz-jason approved these changes Oct 18, 2018

View reviewed changes

zz-jason added 2 commits October 18, 2018 18:29

Merge branch 'master' into explain-partition-pruning

06fcba4

Merge branch 'master' into explain-partition-pruning

5d01345

zz-jason merged commit 19f5648 into pingcap:master Oct 18, 2018

tiancaiamao deleted the explain-partition-pruning branch October 18, 2018 10:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

explaintest: add explain test for partition pruning #7505

explaintest: add explain test for partition pruning #7505

tiancaiamao commented Aug 27, 2018

shenli commented Aug 28, 2018

shenli Aug 28, 2018

tiancaiamao Aug 28, 2018

shenli Aug 28, 2018

tiancaiamao Aug 28, 2018

ciscoxll Aug 31, 2018

tiancaiamao Aug 31, 2018

tiancaiamao commented Aug 28, 2018

alivxxx left a comment

ciscoxll left a comment

zz-jason commented Oct 11, 2018

tiancaiamao commented Oct 11, 2018

zz-jason commented Oct 11, 2018

zz-jason commented Oct 11, 2018

tiancaiamao commented Oct 11, 2018

tiancaiamao commented Oct 11, 2018

tiancaiamao commented Oct 11, 2018

tiancaiamao commented Oct 12, 2018

explaintest: add explain test for partition pruning #7505

explaintest: add explain test for partition pruning #7505

Conversation

tiancaiamao commented Aug 27, 2018

What problem does this PR solve?

What is changed and how it works?

Check List

shenli commented Aug 28, 2018

shenli Aug 28, 2018

Choose a reason for hiding this comment

tiancaiamao Aug 28, 2018

Choose a reason for hiding this comment

shenli Aug 28, 2018

Choose a reason for hiding this comment

tiancaiamao Aug 28, 2018

Choose a reason for hiding this comment

ciscoxll Aug 31, 2018

Choose a reason for hiding this comment

tiancaiamao Aug 31, 2018

Choose a reason for hiding this comment

tiancaiamao commented Aug 28, 2018

alivxxx left a comment

Choose a reason for hiding this comment

ciscoxll left a comment

Choose a reason for hiding this comment

zz-jason commented Oct 11, 2018

tiancaiamao commented Oct 11, 2018

zz-jason commented Oct 11, 2018

zz-jason commented Oct 11, 2018

tiancaiamao commented Oct 11, 2018

tiancaiamao commented Oct 11, 2018

tiancaiamao commented Oct 11, 2018

tiancaiamao commented Oct 12, 2018