Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: mysql-cdc: test add sub source after corruption #27789

Conversation

nrainer-materialize
Copy link
Contributor

@nrainer-materialize nrainer-materialize commented Jun 21, 2024

Create a source, purge the bin log, add a sub source?

This adds testing for #27507.

@nrainer-materialize nrainer-materialize added the T-testing Theme: tests or test infrastructure label Jun 21, 2024
@nrainer-materialize nrainer-materialize self-assigned this Jun 21, 2024
@nrainer-materialize
Copy link
Contributor Author

@petrosagg, is this what we discussed? Should this be working?

SELECT * FROM t3 produces

ERROR:  Source error: u4: mysql server binlog frontier at Antichain { elements: [Partitioned((Interval { lower: 00000000-0000-0000-0000-000000000000, upper: 0a6ab578-2faf-11ef-ac57-0242c0a83002 }, Absent)), Partitioned((Interval { lower: 0a6ab578-2faf-11ef-ac57-0242c0a83003, upper: 0a6ab578-2faf-11ef-ac57-0242c0a83003 }, Active(28))), Partitioned((Interval { lower: 0a6ab578-2faf-11ef-ac57-0242c0a83004, upper: ffffffff-ffff-ffff-ffff-ffffffffffff }, Absent))] } is beyond required frontier Antichain { elements: [Partitioned((Interval { lower: 00000000-0000-0000-0000-000000000000, upper: 0a6ab578-2faf-11ef-ac57-0242c0a83002 }, Absent)), Partitioned((Interval { lower: 0a6ab578-2faf-11ef-ac57-0242c0a83003, upper: 0a6ab578-2faf-11ef-ac57-0242c0a83003 }, Active(27))), Partitioned((Interval { lower: 0a6ab578-2faf-11ef-ac57-0242c0a83004, upper: ffffffff-ffff-ffff-ffff-ffffffffffff }, Absent))] }

@rjobanp
Copy link
Contributor

rjobanp commented Jun 21, 2024

@nrainer-materialize is this working or failing with that error you posted? The CI checks seem to be green on this.

What I think is happening in the failure case is that the source doesn't actually finish the snapshot of the first tables before materialize is killed and the binlog wiped. Then when it's restarted it can't resume from a point that the first subsources needed. I think we should add a SELECT * FROM t1 before we kill MZ and wipe the binlog

@nrainer-materialize
Copy link
Contributor Author

@nrainer-materialize is this working or failing with that error you posted? The CI checks seem to be green on this.

I guess it was failing locally.

What I think is happening in the failure case is that the source doesn't actually finish the snapshot of the first tables before materialize is killed and the binlog wiped. Then when it's restarted it can't resume from a point that the first subsources needed. I think we should add a SELECT * FROM t1 before we kill MZ and wipe the binlog

Will do!

@nrainer-materialize nrainer-materialize merged commit 1d1c907 into MaterializeInc:main Jun 24, 2024
9 checks passed
@nrainer-materialize nrainer-materialize deleted the tests/mysql-separate-gtid-subsource branch June 24, 2024 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-testing Theme: tests or test infrastructure
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants