Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM's schema tracker doesn't track DDL when we skip some DDLs which will lead to column count unmatch #3239

Closed
lichunzhu opened this issue May 24, 2021 · 1 comment · Fixed by #4178
Assignees
Labels
area/dm Issues or PRs related to DM. subject/new-feature Denotes an issue or pull request adding a new feature.

Comments

@lichunzhu
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do? If possible, provide a recipe for reproducing the error.
    Configure binlog-event-filter to filter some DDLs to downstream database.

  2. What did you expect to see?
    The binlog-event-filter takes effect and the replication can continue automatically.

  3. What did you see instead?
    DM task will pause with error 'column count unmatch'.

Root Cause:

  1. DM will not track the DDLs to skip in the schema tracker. Then DM will face count unmatch errors when DM tries to parse DML events from the upstream binlogs.

  2. If DM tracks these DDLs in the schema tracker, the generated DMLs will contain skipped columns now. Since the DDL is not executed in the downstream, the DML will also fail in the downstream database.

  3. Versions of the cluster

    • DM version (run dmctl -V or dm-worker -V or dm-master -V):
      DM v2.0.3
@Ehco1996 Ehco1996 self-assigned this Jun 29, 2021
@lance6716 lance6716 assigned lance6716 and unassigned Ehco1996 Aug 2, 2021
@okJiang
Copy link
Member

okJiang commented Oct 27, 2021

from comment pingcap/dm#1975 (comment)

a recent ONCALL requires we also track DDL when it's skipped by handle-error skip. please check and fix it in another PR 😂

and change the PR description, don't close the issue if we didn't correcly handle handle-error skip

wait the second PR for closing.

@lonng lonng transferred this issue from pingcap/dm Nov 2, 2021
@lance6716 lance6716 added area/dm Issues or PRs related to DM. subject/new-feature Denotes an issue or pull request adding a new feature. labels Dec 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dm Issues or PRs related to DM. subject/new-feature Denotes an issue or pull request adding a new feature.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants