feat: Shadow tracking #11689
feat: Shadow tracking #11689
Conversation
staffik
force-pushed
the
shadow_tracking
branch
2 times, most recently
from
5fca122 to
e1e1095
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #11689 +/- ##
==========================================
- Coverage 71.81% 71.78% -0.03%
==========================================
Files 790 792 +2
Lines 161945 162586 +641
Branches 161945 162586 +641
==========================================
+ Hits 116294 116710 +416
- Misses 40618 40824 +206
- Partials 5033 5052 +19
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Tested in forknet20, I will add a dedicated pytest. |
| @@ -90,6 +96,9 @@ impl ShardTracker { | |||
| let subset = &schedule[index as usize]; | |||
| Ok(subset.contains(&shard_id)) | |||
| } | |||
| TrackedConfig::ShadowValidator(account_id) => { | |||
| self.epoch_manager.cares_about_shard_in_epoch(*epoch_id, account_id, shard_id) | |||
There was a problem hiding this comment.
can we also give some warning somewhere (potentially here) if the accountid is not eligible to be a chunk validator as all? I was thinking about when loading the config but at that point I am not sure if the node will have the full info about the other validators.
There was a problem hiding this comment.
Logging it here is spammy: resulted in thousands of logs in a very short nayduck test.
Part of: near/near-one-project-tracking#65 An option for non-validator node to track shards of given validator. During stateful -> stateless protocol upgrade a node will track all shards and will require a lot of RAM. After the migration we can move the validator key to a new, smaller node, that does not track all shards. To make it with minimal downtime, the new node needs to have appropriate shards in place and memtries loaded in memory, then we hot swap the validator key without stopping the new node. But before that happen the new node is not a validator and we need a way to tell it which validator's shards it should track.
Part of: near/near-one-project-tracking#65
An option for non-validator node to track shards of given validator.
During stateful -> stateless protocol upgrade a node will track all shards and will require a lot of RAM. After the migration we can move the validator key to a new, smaller node, that does not track all shards.
To make it with minimal downtime, the new node needs to have appropriate shards in place and memtries loaded in memory, then we hot swap the validator key without stopping the new node.
But before that happen the new node is not a validator and we need a way to tell it which validator's shards it should track.