Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mounter(ticdc): calculate row level checksum for timestmap by using UTC time zone #10564

Merged
merged 46 commits into from
Feb 22, 2024
Merged

mounter(ticdc): calculate row level checksum for timestmap by using UTC time zone #10564

merged 46 commits into from
Feb 22, 2024

Conversation

3AceShowHand
Copy link
Contributor

@3AceShowHand 3AceShowHand commented Jan 30, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #10573

What is changed and how it works?

  • When calculate the checksum for timestamp value, ignore the server tz, always convert it to UTC
  • Simple avro, add UnsignedBigint schema, use it to encode unsigned bigint value.
  • Simple avro, add Timestamp schema, use it to encode timestamp value.

Check List

Tests

  • Unit test
  • Integration test

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note 

fix e2e checksum on the timestamp value failed since does not consider time zone

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 30, 2024
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 31, 2024
@3AceShowHand 3AceShowHand changed the title [DNM]mounter(ticdc): detect e2e checksum correctness mounter(ticdc): calculate row level checksum for timestmap by using UTC time zone Jan 31, 2024
@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/needs-triage-completed and removed do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jan 31, 2024
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 2, 2024
@3AceShowHand
Copy link
Contributor Author

/test dm-compatibility-test

@3AceShowHand
Copy link
Contributor Author

/test dm-integration-test

@3AceShowHand
Copy link
Contributor Author

/test cdc-integration-kafka-test

@3AceShowHand
Copy link
Contributor Author

/retest

@3AceShowHand
Copy link
Contributor Author

/retest

2 similar comments
@3AceShowHand
Copy link
Contributor Author

/retest

@3AceShowHand
Copy link
Contributor Author

/retest

@3AceShowHand
Copy link
Contributor Author

/retest

@3AceShowHand
Copy link
Contributor Author

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 21, 2024
@ti-chi-bot ti-chi-bot bot added the affect-ticdc-config-docs Pull requests that affect TiCDC configuration docs. label Feb 22, 2024
@3AceShowHand 3AceShowHand removed the affect-ticdc-config-docs Pull requests that affect TiCDC configuration docs. label Feb 22, 2024
@3AceShowHand
Copy link
Contributor Author

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 22, 2024
@3AceShowHand
Copy link
Contributor Author

/retest

1 similar comment
@3AceShowHand
Copy link
Contributor Author

/retest

@3AceShowHand
Copy link
Contributor Author

/test dm-integration-test

@3AceShowHand
Copy link
Contributor Author

/retest

@ti-chi-bot ti-chi-bot bot merged commit 71b5a0a into pingcap:master Feb 22, 2024
28 checks passed
@3AceShowHand
Copy link
Contributor Author

/cherry-pick release-7.1
/cherry-pick release-7.5

@ti-chi-bot
Copy link
Member

@3AceShowHand: new pull request created to branch release-7.1: #10645.

In response to this:

/cherry-pick release-7.1
/cherry-pick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Feb 26, 2024
@3AceShowHand @ti-chi-bot
This is an automated cherry-pick of pingcap#10564
e319967 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Feb 26, 2024
Closed
ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Feb 26, 2024
@3AceShowHand @ti-chi-bot
This is an automated cherry-pick of pingcap#10564
3e52aee 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

@3AceShowHand: new pull request created to branch release-7.5: #10646.

In response to this:

/cherry-pick release-7.1
/cherry-pick release-7.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot mentioned this pull request Feb 26, 2024
Merged
@3AceShowHand 3AceShowHand mentioned this pull request Feb 26, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this pull request Apr 28, 2024
@ti-chi-bot
mounter(ticdc): calculate row level checksum for timestmap by using U…
47417d7 
…TC time zone (#10564) (#10646)

close #10573
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@asddongmen asddongmen asddongmen approved these changes

@GMHDBJD GMHDBJD Awaiting requested review from GMHDBJD

Assignees
No one assigned
Labels
approved lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

row level checksum verification failed on timestamp column caused by time zone mistmatch
4 participants
@3AceShowHand @GMHDBJD @ti-chi-bot @asddongmen