Store linear blocks in freezer db #5905
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
dapplion
commented
Jun 10, 2024
| let block = store | ||
| .get_hot_blinded_block(&block_root)? | ||
| .ok_or(HotColdDBError::MissingBlockToFreeze(block_root))?; | ||
| store.blinded_block_as_cold_kv_store_ops(&block_root, &block, &mut cold_db_ops)?; |
There was a problem hiding this comment.
Does the state_root_iter yield the same block_root more than once for skipped slots? If yes we are doing extra work here
dapplion
commented
Jun 10, 2024
| upgrade_freezer_database::<T>(&db, &log)?; | ||
|
|
||
| // No hot DB changes | ||
| return Ok(vec![]); |
There was a problem hiding this comment.
@michaelsproul thoughts on migrating all finalized blocks? If we just delete that's going to be a lot of bandwidth on the entire network to backfill. However, migrating all blocks will take a while. Are DB migrations blocking the start routine of Lighthouse?
1 task
michaelsproul
added
optimization
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue Addressed
Part of tree-states Disk Edition ™️
Currently Lighthouse stores blocks by root in the same bucket as unfinalized blocks. Storing finalized blocks in a separate bucket allows faster reads of sequential slot ranges. It also simplifies pruning logic and can allow us to drop the head-tracker which has been a source of bugs in the past:
Proposed Changes
BeaconBlockFrozenand compress them