Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

streaming: ingestion job to rollback to resolved timestamp OnFailOrCancel #57404

Closed
pbardea opened this issue Dec 2, 2020 · 0 comments · Fixed by #58693
Closed

streaming: ingestion job to rollback to resolved timestamp OnFailOrCancel #57404

pbardea opened this issue Dec 2, 2020 · 0 comments · Fixed by #58693
Assignees
@pbardea
Labels
A-disaster-recovery C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-disaster-recovery

Comments

@pbardea
Copy link
Contributor

pbardea commented Dec 2, 2020

Once the ingestion job is maintaining its ingested timestamp (the low-watermark resolved timestamp across all processors, see #57400), the job should be able to rollback to this timestamp when the job is cancelled or fails.

The job will perform a RevertRange request over the entire keyspace on which it has been ingesting. After this change, the ingestion job should be able to ingest a stream and leave the cluster in a consistent state.
@pbardea pbardea added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-disaster-recovery labels Dec 2, 2020
@pbardea pbardea self-assigned this Jan 11, 2021
@pbardea pbardea mentioned this issue Jan 12, 2021
Merged
@craig craig bot closed this as completed in e650213 Jan 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pbardea pbardea

Labels
A-disaster-recovery C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-disaster-recovery
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

streamingccl: support rolling back to a timestamp on job cancel pbardea/cockroach
2 participants
@kenliu @pbardea