Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

feat(dup): implement procedure load_from_private_log #317

Merged
merged 8 commits into from
Sep 23, 2019

Conversation

neverchanje
Copy link
Contributor

@neverchanje neverchanje commented Sep 19, 2019

load_from_private_log performs the heaviest procedure among the duplication:
loading mutation-logs from disk.

The entire process:

  1. load_from_private_log::run:
    ensure the logs from start_decree are available on disk.

  2. load_from_private_log::find_log_file_to_start:
    select the log file to load.

  3. load_from_private_log::replay_log_block:
    loading in a block by block way. once EOF is hit, switch to the next file.

  4. Once a log block is loaded, the mutations will be passed down to the next stage, aka ship_mutations.

The unit test of this procedure is instrumented by some testing methods:

  1. write and load: write a number of fake mutations to disk and load. The next stage after load_from_private_log is mocked as verification of the loaded logs.

  2. handle real-world logs, not only fake logs: given files that were generated from a real pegasus server.

  3. fault injection: the replaying of logs and file opening are injected with faults in 25% probability. This helps us to test the corner cases.

qinzuoyan
qinzuoyan previously approved these changes Sep 20, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants