Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

streamingccl: add IngestionFrontierProcessor #58672

Merged
merged 1 commit into from
Jan 27, 2021

Commits on Jan 27, 2021

  1. streamingccl: add IngestionFrontierProcessor

    The IngestionFrontierProcessor (IFP) is fed rows from the IngestionProcessor.
    Each row represents a partition and the latest resolved ts event
    received for that partition. The IFP uses the existing util
    `spanFrontier` (used by changefeeds) to keep track of all the rows
    returned by the several ingestion procs, and resolves the minimum
    resolved ts for a particular partition across all procs. This is then
    written to the ingestion job progress indicating that all processors
    have ingested a stream upto a particular resolved ts.
    
    Release note: None
    adityamaru committed Jan 27, 2021
    Configuration menu
    Copy the full SHA
    013a948 View commit details
    Browse the repository at this point in the history