Skip to content

Commit

Permalink
Add flat state specs to resharding NEP (#570)
Browse files Browse the repository at this point in the history
Filling the specification section about flat state.
  • Loading branch information
Trisfald authored Oct 29, 2024
1 parent 47f22f9 commit 8d8e761
Showing 1 changed file with 31 additions and 0 deletions.
31 changes: 31 additions & 0 deletions neps/nep-0568.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,37 @@ post-processing, as long as the chain's view reflects a fully resharded state.

### State Storage - Flat State

Flat State is a collection of key-value pairs stored on disk and each entry
contains a reference to its ShardId. When splitting a shard, every item inside
its Flat State must be correctly reassigned to either one of the new children;
due to technical limitations such operation can't be completed instantaneously.

Flat State main purposes are allowing the creation of State Sync snapshots and
the construction of Mem Tries. Fortunately, these two operations can be delayed
until resharding is completed. Note also that with Mem Tries enabled the chain
can move forward even if the current status of Flat State is not in sync with
the latest block.

For the reason stated above, the chosen strategy is to reshard Flat State in a
long-running background task. The new shards' states must converge with their
Mem Tries representation in a reasonable amount of time.

Splitting a shard's Flat State is performed in multiple steps:
1) A post-processing 'split' task is created during the last block of the old

Check failure on line 92 in neps/nep-0568.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Lists should be surrounded by blank lines [Context: "1) A post-processing 'split' t..."]

neps/nep-0568.md:92 MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "1) A post-processing 'split' t..."]
shard layout, instantaneously.
2) The 'split' task runs in parallel with the chain for a certain amount of
time. Inside this routine every key-value pair belonging to the shard being
split (also called parent shard) is copied into either the left or the right
child Flat State. Entries linked to receipts are handled in a special way.
3) Once the task is completed, the parent shard Flat State is cleaned up. The
children shards Flat States have their state in sync with last block of the
old shard layout.
4) Children shards must apply the delta changes from the first block of the new
shard layout until the final block of the canonical chain. This operation is
done in another background task to avoid slowdowns while processing blocks.
5) Children shards Flat States are now ready and can be used to take State Sync
snapshots and to reload Mem Tries.

### State Storage - State

### Stateless Validation
Expand Down

0 comments on commit 8d8e761

Please sign in to comment.