ddl: fix reassigned partition id in `truncate table` does not take effect #7919

ciscoxll · 2018-10-16T12:49:56Z

What problem does this PR solve?

When we executed truncate table multiple times, we found that the assigned partitionID did not take effect.

What is changed and how it works?

Assign a new partition ID to each partition.
Assigning a new ID to a partition should be done by changing the pointer instead of assigning it by copying.

Check List

Tests

Unit test

alivxxx · 2018-10-17T02:53:00Z

ddl/partition.go

@@ -417,3 +417,24 @@ func isRangePartitionColUnsignedBigint(cols []*table.Column, pi *model.Partition
 	}
 	return false
 }
+
+// truncateTableFromReassignPartitionIDs reassign a new partition ids.
+func truncateTableFromReassignPartitionIDs(job *model.Job, t *meta.Meta, tblInfo *model.TableInfo) (ver int64, _ error) {


Why return the ver? It is not used or assigned in this function.

ciscoxll · 2018-10-17T07:39:47Z

@lamxTyler @zimulala PTAL.

ciscoxll · 2018-10-17T08:01:23Z

/run-all-tests

alivxxx

LGTM

ciscoxll · 2018-10-18T06:29:49Z

@tiancaiamao @crazycs520 PTAL.

crazycs520 · 2018-10-18T11:44:20Z

ddl/partition.go

+
+// truncateTableFromReassignPartitionIDs reassign a new partition ids.
+func truncateTableFromReassignPartitionIDs(job *model.Job, t *meta.Meta, tblInfo *model.TableInfo) error {
+	newDefs := make([]model.PartitionDefinition, 0, len(tblInfo.Partition.Definitions))


no need to define a new defs slice.

for i := range tblInfo.Partition.Definitions { var pid int64 pid, err = t.GenGlobalID() if err != nil { job.State = model.JobStateCancelled return ver, errors.Trace(err) } tblInfo.Partition.Definitions[i].ID = pid }

tiancaiamao · 2018-10-19T03:05:52Z

ddl/partition.go

+			job.State = model.JobStateCancelled
+			return errors.Trace(err)
+		}
+		tblInfo.Partition.Definitions[i].ID = pid


So, if an error occur and DDL rollback, model.TableInfo.Partition is partially changed, some id is new and the old ?

@tiancaiamao This is executed in a transaction. All changes will not take effect until the transaction is successfully committed.

@tiancaiamao ~~Only one updateSchemaVersion was executed in onTruncateTable, so if an error occurs and DDL rollback will not be a problem.~~

crazycs520

LGTM

crazycs520

LGTM

crazycs520

LGTM

winkyao

LGTM

winkyao

LGTM

winkyao

LGTM

winkyao

LGTM

winkyao

LGTM

winkyao · 2018-10-22T02:57:05Z

ddl/partition.go

@@ -417,3 +417,16 @@ func isRangePartitionColUnsignedBigint(cols []*table.Column, pi *model.Partition
 	}
 	return false
 }
+
+// truncateTableFromReassignPartitionIDs reassign a new partition ids.
+func truncateTableFromReassignPartitionIDs(job *model.Job, t *meta.Meta, tblInfo *model.TableInfo) error {


truncateTableFromReassignPartitionIDs ---> truncateTableByReassignPartitionIDs

winkyao · 2018-10-22T03:01:10Z

ddl/partition.go

@@ -417,3 +417,16 @@ func isRangePartitionColUnsignedBigint(cols []*table.Column, pi *model.Partition
 	}
 	return false
 }
+
+// truncateTableFromReassignPartitionIDs reassign a new partition ids.
+func truncateTableFromReassignPartitionIDs(job *model.Job, t *meta.Meta, tblInfo *model.TableInfo) error {


truncateTableFromReassignPartitionIDs ---> truncateTableByReassignPartitionIDs

crazycs520 · 2018-10-22T07:42:21Z

/run-all-tests

bugfix fixed pingcap#7518 expression: MySQL compatible current_user function (pingcap#7801) plan: propagate constant over outer join (pingcap#7794) - extract `outerCol = const` from join conditions and filter conditions, substitute `outerCol` in join conditions with `const`; - extract `outerCol = innerCol` from join conditions, derive new join conditions based on this column equal condition and `outerCol` related expressions in join conditions and filter conditions; util/timeutil: fix data race caused by forgetting set stats lease to 0 (pingcap#7901) stats: handle ddl event for partition table (pingcap#7903) plan: implement Operand and Pattern of cascades planner. (pingcap#7910) planner: not convert to TableDual if empty range is derived from deferred constants (pingcap#7808) plan: move projEliminate behind aggEliminate (pingcap#7909) admin: fix admin check table bug of byte compare (pingcap#7887) * admin: remove reflect deepEqual stats: fix panic caused by empty histogram (pingcap#7912) plan: fix panic caused by empty schema of LogicalTableDual (pingcap#7906) * fix drop view if exist error (pingcap#7833) executor: refine `explain analyze` (pingcap#7888) executor: add an variable to compatible with MySQL insert for OGG (pingcap#7863) expression: maintain `DeferredExpr` in aggressive constant folding. (pingcap#7915) stats: fix histogram boundaries overflow error (pingcap#7883) ddl:support the definition of `null` change to `not null` using `alter table` (pingcap#7771) * ddl:support the definition of null change to not null using alter table ddl: add check when create table with foreign key. (pingcap#7885) * ddl: add check when create table with foreign key planner: eliminate if null on non null column (pingcap#7924) executor: fix a bug in point get (pingcap#7934) planner, executor: refine ColumnPrune for LogicalUnionAll (pingcap#7930) executor: fix panic when limit is too large (pingcap#7936) ddl: add TiDB version to metrics (pingcap#7902) stats: limit the length of sample values (pingcap#7931) vendor: update tipb (pingcap#7893) planner: support the Group and GroupExpr for the cascades planner (pingcap#7917) store/tikv: log more information when other err occurs (pingcap#7948) types: fix date time parse (pingcap#7933) ddl: just print error message when ddl job is normal to calcel, to eliminate noisy log (pingcap#7875) stats: update delta info for partition table (pingcap#7947) explaintest: add explain test for partition pruning (pingcap#7505) util: move disjoint set to util package (pingcap#7950) util: add PreAlloc4Row and Insert for Chunk and List (pingcap#7916) executor: add the slow log for commit (pingcap#7951) expression: add builtin json_keys (pingcap#7776) privilege: add USAGE in `show grants` for mysql compatibility (pingcap#7955) ddl: fix invailid ddl job panic (pingcap#7940) *: move ast.NewValueExpr to standalone parser_driver package (pingcap#7952) Make the ast package get rid of the dependency of types.Datum server: allow cors http request (pingcap#7939) *: move `Statement` and `RecordSet` from ast to sqlexec package (pingcap#7970) pr suggestion update executor/aggfuncs: split unit tests to corresponding file (pingcap#7993) store/tikv: fix typo (pingcap#7990) executor, planner: clone proj schema for different children in buildProj4Union (pingcap#7999) executor: let information_schema be the first database in ShowDatabases (pingcap#7938) stats: use local feedback for partition table (pingcap#7963) executor: add unit test for aggfuncs (pingcap#7966) server: add log for binary execute statement (pingcap#7987) admin: refine admin check decoder (pingcap#7862) executor: improve wide table insert & update performance (pingcap#7935) ddl: fix reassigned partition id in `truncate table` does not take effect (pingcap#7919) fix reassigned partition id in truncate table does not take effect add changelog for 2.1.0 rc4 (pingcap#8020) *: make parser package dependency as small as possible (pingcap#7989) parser: support `:=` in the `set` syntax (pingcap#8018) According to MySQL document, `set` use the = assignment operator, but the := assignment operator is also permitted stats: garbage collect stats for partition table (pingcap#7962) docs: add the proposal for the column pool (pingcap#7988) expression: refine built-in func truncate to support uint arg (pingcap#8000) stats: support show stats for partition table (pingcap#8023) stats: update error rate for partition table (pingcap#8022) stats: fix estimation for out of range point queries (pingcap#8015) *: move parser to a separate repository (pingcap#8036) executor: fix wrong result when index join on union scan. (pingcap#8031) Do not modify Plan of dataReaderBuilder directly, because it would impact next batch of outer rows, as well as other concurrent inner workers. Instead, build a local child builder to store the child plan. planner: fix a panic of a cached prepared statement with IndexScan (pingcap#8017) *: fix the issue of executing DDL after executing SQL failure in txn (pingcap#8044) * ddl, executor: fix the issue of executing DDL after executing SQL failure in txn add unit test remove debug info add like evaluator case sensitive test ddl, domain: make schema correct after canceling jobs (pingcap#7997) unit test fix code format proposal: maintaining histograms in plan. (pingcap#7605) support _tidb_rowid for table scan range (pingcap#8047) var rename fix

fix reassigned partition id in truncate table does not take effect

45bf86b

ciscoxll added the component/DDL-need-LGT3 label Oct 16, 2018

alivxxx reviewed Oct 17, 2018

View reviewed changes

Address comment

2fa6b5d

alivxxx reviewed Oct 17, 2018

View reviewed changes

ciscoxll added status/all tests passed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 17, 2018

crazycs520 reviewed Oct 18, 2018

View reviewed changes

Address comment

09b6b23

ciscoxll force-pushed the truncate-table-improve branch from a3c3257 to 09b6b23 Compare October 19, 2018 02:23

tiancaiamao reviewed Oct 19, 2018

View reviewed changes

crazycs520 reviewed Oct 22, 2018

View reviewed changes

winkyao reviewed Oct 22, 2018

View reviewed changes

pingcap deleted a comment from winkyao Oct 22, 2018

Address comment

ff5da6b

ciscoxll added status/LGT3 The PR has already had 3 LGTM. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 23, 2018

crazycs520 approved these changes Oct 23, 2018

View reviewed changes

ciscoxll added 2 commits October 23, 2018 17:50

Merge branch 'master' into truncate-table-improve

50de52c

Merge branch 'master' into truncate-table-improve

63c4cd0

ciscoxll merged commit 7b8caa1 into pingcap:master Oct 23, 2018

ciscoxll deleted the truncate-table-improve branch October 23, 2018 10:14

ciscoxll added the type/2.1 cherry-pick label Oct 29, 2018

winkyao removed the type/2.1 cherry-pick label Oct 29, 2018

This was referenced Oct 30, 2018

ddl: fix reassigned partition id in truncate table does not take effect #8102

Closed

fix reassigned partition id in truncate table does not take effect #8103

Merged

sre-bot added the contribution This PR is from a community contributor. label Dec 18, 2019

you06 added the sig/sql-infra SIG: SQL Infra label Mar 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ddl: fix reassigned partition id in `truncate table` does not take effect #7919

ddl: fix reassigned partition id in `truncate table` does not take effect #7919

ciscoxll commented Oct 16, 2018

alivxxx Oct 17, 2018

ciscoxll commented Oct 17, 2018

ciscoxll commented Oct 17, 2018

alivxxx left a comment

ciscoxll commented Oct 18, 2018

crazycs520 Oct 18, 2018

tiancaiamao Oct 19, 2018

crazycs520 Oct 19, 2018

ciscoxll Oct 19, 2018 •

edited

Loading

crazycs520 left a comment

crazycs520 left a comment

crazycs520 left a comment

winkyao left a comment

winkyao left a comment

winkyao left a comment

winkyao left a comment

winkyao left a comment

winkyao Oct 22, 2018

winkyao Oct 22, 2018

crazycs520 commented Oct 22, 2018

ddl: fix reassigned partition id in truncate table does not take effect #7919

ddl: fix reassigned partition id in truncate table does not take effect #7919

Conversation

ciscoxll commented Oct 16, 2018

What problem does this PR solve?

What is changed and how it works?

Check List

alivxxx Oct 17, 2018

Choose a reason for hiding this comment

ciscoxll commented Oct 17, 2018

ciscoxll commented Oct 17, 2018

alivxxx left a comment

Choose a reason for hiding this comment

ciscoxll commented Oct 18, 2018

crazycs520 Oct 18, 2018

Choose a reason for hiding this comment

tiancaiamao Oct 19, 2018

Choose a reason for hiding this comment

crazycs520 Oct 19, 2018

Choose a reason for hiding this comment

ciscoxll Oct 19, 2018 • edited Loading

Choose a reason for hiding this comment

crazycs520 left a comment

Choose a reason for hiding this comment

crazycs520 left a comment

Choose a reason for hiding this comment

crazycs520 left a comment

Choose a reason for hiding this comment

winkyao left a comment

Choose a reason for hiding this comment

winkyao left a comment

Choose a reason for hiding this comment

winkyao left a comment

Choose a reason for hiding this comment

winkyao left a comment

Choose a reason for hiding this comment

winkyao left a comment

Choose a reason for hiding this comment

winkyao Oct 22, 2018

Choose a reason for hiding this comment

winkyao Oct 22, 2018

Choose a reason for hiding this comment

crazycs520 commented Oct 22, 2018

ddl: fix reassigned partition id in `truncate table` does not take effect #7919

ddl: fix reassigned partition id in `truncate table` does not take effect #7919

ciscoxll Oct 19, 2018 •

edited

Loading