Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support to check whether the table can be duplicated by CDC #368

Merged
merged 15 commits into from
Mar 25, 2020

Conversation

zier-one
Copy link
Contributor

@zier-one zier-one commented Mar 23, 2020

What problem does this PR solve?

Support to check whether the table can be duplicated by CDC

What is changed and how it works?

we check table in the following step:

  1. creating the changefeed
  2. handling DDL jobs in the owner

Check List

Tests

  • Unit test
  • Integration test

Run those sql in upstream db:

MySQL [test]> create table t1(a int ,b int);
Query OK, 0 rows affected (0.308 sec)

MySQL [test]> create table t2(a int unique key null ,b int);
Query OK, 0 rows affected (0.241 sec)

MySQL [test]> create table t3(a int unique key not null ,b int);
Query OK, 0 rows affected (0.750 sec)

create a changefeed:

[my-devel]➜ bin git:(check_table) ✗ ./cdc cli changefeed create --sink-uri="mysql://root@127.0.0.1:3306/"
[WARN] some tables are not eligible to duplicate, []entry.TableName{entry.TableName{Schema:"test", Table:"t1"}, entry.TableName{Schema:"test", Table:"t2"}}
Could you agree to ignore those tables, and continue to duplicate [Y/N]
N
[my-devel]➜ bin git:(check_table) ✗ ./cdc cli changefeed create --sink-uri="mysql://root@127.0.0.1:3306/" --no-confirm
[WARN] some tables are not eligible to duplicate, []entry.TableName{entry.TableName{Schema:"test", Table:"t1"}, entry.TableName{Schema:"test", Table:"t2"}}
create changefeed ID: 9a4603c1-5dda-4279-9001-ae01acd294b0 info {"sink-uri":"mysql://root@127.0.0.1:3306/","opts":{},"create-time":"2020-03-24T11:43:22.130405373+08:00","start-ts":415503850433675265,"target-ts":0,"admin-job-type":0,"config":{"filter-case-sensitive":false,"filter-rules":null,"ignore-txn-commit-ts":null}}

@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one zier-one added the status/ptal Could you please take a look? label Mar 24, 2020
cdc/owner.go Outdated Show resolved Hide resolved
cmd/client.go Outdated Show resolved Hide resolved
@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one
Copy link
Contributor Author

@amyangfei PTAL again

@zier-one
Copy link
Contributor Author

/run-integration-tests

2 similar comments
@zier-one
Copy link
Contributor Author

/run-integration-tests

@zier-one
Copy link
Contributor Author

/run-integration-tests

Copy link
Contributor

@amyangfei amyangfei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@amyangfei amyangfei added LGT1 and removed status/ptal Could you please take a look? labels Mar 25, 2020
@zier-one zier-one merged commit 516e9fe into pingcap:master Mar 25, 2020
@zier-one zier-one deleted the check_table branch April 10, 2020 05:13
5kbpers pushed a commit to 5kbpers/ticdc that referenced this pull request Aug 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants