This repository has been archived by the owner on Jun 23, 2022. It is now read-only.
feat(dup): implement procedure load_from_private_log #317
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
load_from_private_log
performs the heaviest procedure among the duplication:loading mutation-logs from disk.
The entire process:
load_from_private_log::run
:ensure the logs from
start_decree
are available on disk.load_from_private_log::find_log_file_to_start
:select the log file to load.
load_from_private_log::replay_log_block
:loading in a block by block way. once EOF is hit, switch to the next file.
Once a log block is loaded, the mutations will be passed down to the next stage, aka
ship_mutations
.The unit test of this procedure is instrumented by some testing methods:
write and load: write a number of fake mutations to disk and load. The next stage after
load_from_private_log
is mocked as verification of the loaded logs.handle real-world logs, not only fake logs: given files that were generated from a real pegasus server.
fault injection: the replaying of logs and file opening are injected with faults in 25% probability. This helps us to test the corner cases.