changefeedccl: fix schema change boundary race condition #137554
Draft
+479
−201
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
andyyang890
force-pushed
the
20241213-refactor-schema-change-frontier
branch
2 times, most recently
from
006e8fa to
9a3efc3
Compare
This patch fixes an issue where a schema change could incorrectly cause a changefeed to fail with an assertion error like `received boundary timestamp ... of type ... before reaching existing boundary of type ...` due to a race condition between aggregators sending resolved spans for multiple schema change descriptor updates to the change frontier. This patch also renames `schemaChangeFrontier` to `resolvedSpanFrontier` to reflect that not all resolved span boundaries correspond to schema changes (e.g. initial-scan-only, end time reached). Its private fields are also now properly encapsulated and new derived struct types for each processor type have been created to handle the processor-specific boundary handling logic. The field and method used for tracking whether a changefeed is considered idle has been extracted from the frontier struct because it is not related to span progress or boundaries. Release note (bug fix): An issue where a schema change could incorrectly cause a changefeed to fail with an assertion error like `received boundary timestamp ... of type ... before reaching existing boundary of type ...` has now been fixed.
andyyang890
force-pushed
the
20241213-refactor-schema-change-frontier
branch
from
9a3efc3 to
85c27fb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch fixes an issue where a schema change could incorrectly
cause a changefeed to fail with an assertion error like
received boundary timestamp ... of type ... before reaching existing boundary of type ...due to a race condition between aggregators sendingresolved spans for multiple schema change descriptor updates to
the change frontier.
This patch also renames
schemaChangeFrontiertoresolvedSpanFrontierto reflect that not all resolved span boundaries correspond to schema
changes (e.g. initial-scan-only, end time reached). Its private fields
are also now properly encapsulated and new derived struct types for each
processor type have been created to handle the processor-specific
boundary handling logic. The field and method used for tracking whether
a changefeed is considered idle has been extracted from the frontier
struct because it is not related to span progress or boundaries.
Release note (bug fix): An issue where a schema change could
incorrectly cause a changefeed to fail with an assertion error
like
received boundary timestamp ... of type ... before reaching existing boundary of type ...has now been fixed.