Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-22.2: changefeedccl: Fix initial scan checkpointing #97049

Merged
merged 1 commit into from
Feb 14, 2023

Commits on Feb 11, 2023

  1. changefeedccl: Fix initial scan checkpointing

    An over than 2 year old change
    (#71848)
    that added support for checkpointing during backfill after schema change,
    inadvertently broke initial scan checkpointing funcitonality
    
    Exacerbating the problem, the existing test
    `TestChangefeedBackfillCheckpoint` continued to work fine.
    Treason why it was passing was because the test was looking
    for a checkpoint whose timestamp matched bacfill timestamp.
    The bug involved incorrect initialize/use of 0 timestamp.
    It just so happens, that after initial scan completes, the
    rangefeed starts, and the very first thing it does is to
    generate a 0 timestamp checkpoint.  So, the test was
    observing this event, and continued to work.
    This PR does not have a dedicated test because the existing
    tests work fine -- provided we ignore 0 timestamp checkpoint,
    which is what this PR does in addition to addressing
    the root cause of the bug.
    
    Informs #96959
    
    Release note (enterprise change): Fix a bug in changefeeds, where
    long running initial scans will fail to generate checkpoint.
    Failure to generate checkpoint is particularly bad if the
    changefeed restarts for whatever reason.  Without checkpoints,
    the changefeed will restart from the beginning, and in the worst
    case, when exporting substantially sized tables, changefeed
    initial scan may have hard time completing.
    Yevgeniy Miretskiy committed Feb 11, 2023
    Configuration menu
    Copy the full SHA
    1426d01 View commit details
    Browse the repository at this point in the history