When triggering tree repairs - repair each vnode once #26
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Each vnode keeps track of the ID if the last repair it triggered. If that ID changes, it assumes it has been triggered again and will prompt repair (as long as it is not the filter vnode on the query). This means that for each call to riak_kv_ttaaefs_manager:trigger_tree_repairs/0, each vnode should only repair the tree once. If any faulty segment is not repaired - the next time this node performs a full-sync, the repair will be re-triggered, and each vnode should repair once (and once only again). Note with N nodes in M clusters, when there is a faulty segment there will be N X M fetch_clocks_nval queries for every local full-sync event (and hence trigger of read repairs). Should sync_state become true, repairs will be disabled. to force repairs as an operator - call riak_kv_ttaaefs_manager:trigger_tree_repairs() from remote_console. This will force a read repair only once for each vnode (unless it is disabled by a local full-sync where sync_state=true).
stephenwood24-nhs
approved these changes
Mar 12, 2024
ThomasArts
approved these changes
Mar 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Each vnode keeps track of the ID if the last repair it triggered. If that ID changes, it assumes it has been triggered again and will prompt repair (as long as it is not the filter vnode on the query).
This means that for each call to riak_kv_ttaaefs_manager:trigger_tree_repairs/0, each vnode should only repair the tree once.
If any faulty segment is not repaired - the next time this node performs a full-sync, the repair will be re-triggered, and each vnode should repair once (and once only) again.
Note with N nodes in M clusters, when there is a faulty segment there will be N X M fetch_clocks_nval queries for every local full-sync event (and hence trigger of read repairs). Should sync_state become true after a locally-prompted full-sync, repairs will be disabled.
To force repairs as an operator - call riak_kv_ttaaefs_manager:trigger_tree_repairs() from remote_console. This will force a read repair only once for each vnode (unless it is disabled by a local full-sync where sync_state=true). Do NOT set the riak_kv/aae_fetchclocks_repair environment variable directly.
The additional environment variable riak_kv/aae_fetchclocks_repair_force is no longer used - a sync_state=true full-sync will always disable local tree repair