Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Limitation] TiCDC can't not handle DDL as alter table xx add column xx datetime default current_timestamp properly #11368

Open
asddongmen opened this issue Jul 1, 2024 · 4 comments
Labels
affects-5.4 affects-6.1 affects-6.5 affects-7.1 affects-7.5 affects-8.1 area/ticdc Issues or PRs related to TiCDC. report/customer Customers have encountered this bug. severity/major This is a major bug. type/bug This is a bug.

Comments

@asddongmen
Copy link
Contributor

What did you do?

Reproduction Steps

  1. Create a table t1

    create table t1 (id int primary key);
    
  2. Create a changefeed to replicate this table

  3. Insert some data into t1

  4. Pause changefeed (the purpose is to manually create synchronization delay), assuming the current time is time1.

  5. Add a new column to t1

    alter table t1 add column `updatetime` datetime default current_timestamp;
    
  6. Resume the changefeed and wait for the changefeed to replicate the above DDL downstream. Assume the current time is time2.

    When comparing the upstream and downstream data, it will be found that the values of the newly added column updatetime upstream and downstream inconsistent. The difference in this column upstream and downstream is time2 - time1.

Reason: This is because cdc only synchronizes the DDL itself when synchronizing DDLs that will cause destructive changes like add column, and does not synchronize the row changes caused by the DDL. The reason for doing this is to reduce synchronization latency.

Currently, these inconsistent data can only be manually fixed.

What did you expect to see?

No response

What did you see instead?

As described above.

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

(paste TiDB cluster version here)

Upstream TiKV version (execute tikv-server --version):

(paste TiKV version here)

TiCDC version (execute cdc version):

All version.
@asddongmen asddongmen added type/bug This is a bug. area/ticdc Issues or PRs related to TiCDC. labels Jul 1, 2024
@asddongmen asddongmen added type/enhancement This is a enhancement PR affects-6.5 affects-7.1 affects-7.5 affects-8.1 and removed type/bug This is a bug. type/enhancement This is a enhancement PR labels Jul 1, 2024
@asddongmen asddongmen changed the title Product Behavior: Data will inconsistent after TiCDC replicates the DDL as alter table xx add column xx datetime default current_timestamp Product Behavior: TiCDC can't not handle DDL as alter table xx add column xx datetime default current_timestamp properly Jul 1, 2024
@asddongmen asddongmen changed the title Product Behavior: TiCDC can't not handle DDL as alter table xx add column xx datetime default current_timestamp properly Product Limitation: TiCDC can't not handle DDL as alter table xx add column xx datetime default current_timestamp properly Jul 1, 2024
@asddongmen asddongmen changed the title Product Limitation: TiCDC can't not handle DDL as alter table xx add column xx datetime default current_timestamp properly [Limitation] TiCDC can't not handle DDL as alter table xx add column xx datetime default current_timestamp properly Jul 1, 2024
@lance6716
Copy link
Contributor

if it's caused by different value of current_timestamp in upsteam and downstream, it should be able to fix like #7036

@asddongmen
Copy link
Contributor Author

@lance6716 Thank you very much for your useful insights!

@seiya-annie
Copy link

/found customer

@ti-chi-bot ti-chi-bot bot added the report/customer Customers have encountered this bug. label Jul 5, 2024
@flowbehappy
Copy link
Collaborator

flowbehappy commented Jul 12, 2024

Only fix this issue starting from v6.5, due to the limited bandwidth.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-5.4 affects-6.1 affects-6.5 affects-7.1 affects-7.5 affects-8.1 area/ticdc Issues or PRs related to TiCDC. report/customer Customers have encountered this bug. severity/major This is a major bug. type/bug This is a bug.
Projects
Status: Need Triage
Development

No branches or pull requests

5 participants