Data Inconsistency when meet TiDB "amend + add column default 'xxx'"

[maxshuang]

What did you do?

1. Setup upstream TIDB and downstream MySQL
2. Create a changfeed
3. Prepare and run two sessions in TiDB as follows:

Session 1:

mysql> show variables like "%amend%";
+-----------------------------------+-------+
| Variable_name                     | Value |
+-----------------------------------+-------+
| tidb_enable_amend_pessimistic_txn | ON    |
+-----------------------------------+-------+
1 row in set (0.01 sec)

mysql> create table t2(id int primary key);
Query OK, 0 rows affected (0.09 sec)

mysql> insert into t2 values(1);
Query OK, 1 row affected (0.00 sec)

mysql> select * from t2;
+----+
| id |
+----+
|  1 |
+----+
1 row in set (0.00 sec)

mysql> begin;
Query OK, 0 rows affected (0.00 sec)

mysql> insert into t2 values(2);
Query OK, 1 row affected (0.00 sec)

// HOLD here until session2 commit DDL

mysql> commit;
Query OK, 0 rows affected (0.01 sec)

Session2:

mysql> select * from t2;
+----+
| id |
+----+
|  1 |
+----+
1 row in set (0.05 sec)

// Exec DDL before session1 commit

mysql> alter table t2 add column c1 datetime default "2020-10-10 10:10:10";
Query OK, 0 rows affected (0.06 sec)
Records: 0  Duplicates: 0  Warnings: 0

// Before Session1 commit, we do select 

mysql> select * from t2;
+----+---------------------+
| id | c1                  |
+----+---------------------+
|  1 | 2020-10-10 10:10:10 |
+----+---------------------+
1 row in set (0.06 sec)

// After Session1 commit, we do select

mysql> select * from t2;
+----+---------------------+
| id | c1                  |
+----+---------------------+
|  1 | 2020-10-10 10:10:10 |
|  2 | 2020-10-10 10:10:10 |
+----+---------------------+
2 rows in set (0.06 sec)

4. Select from MySQL

mysql> select * from t2;
+----+---------------------+
| id | c1                  |
+----+---------------------+
|  1 | 2020-10-10 10:10:10 |
|  2 | NULL |
+----+---------------------+

Root Cause
https://github.com/pingcap/ticdc/blob/master/cdc/entry/mounter.go#L509

Before PR #3846, this bug still exists

What did you expect to see?

data consistence

What did you see instead?

data inconsistency

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

all

TiCDC version (execute cdc version):

all

[VelocityLight]

update