Blockchain Sync Improvements: Chain Data & Pipeline

[non-fungible-nelson]

Description

Besu needs to be able to sync the chain data as quickly as possible. It also needs to be more aware of the state of the client (i.e. is the worldstate downloaded? Can I begin validator duties?) and differ or parallelize the chain download to improve the sync UX for Besu stakers and RPC providers.

Goals

• Faster time from start to validating by differing and/or improving chain download
• Refactor chain pipeline for post-Merge environment

Acceptance Criteria:

• New pipeline strategy for chain-data
  • Download blockchain in the background (?) Use 8 seconds of free slot time for background download
  • Better post-Merge sync target based on the CL blocks and what we receive from the CL
  • If we have deposit logs (Teku), immediately start validating when worldstate download is complete

Considerations

• Consider implications for private networks (leave fast unchanged, update snap)
  • auto-detect lock-step sync with CL, switch to optimistic head rather than relying on fcU
• Peering for improving sync performance
  • When receiving an empty response, retry logic polls new peers Make all tasks switching tasks #5271
  • Flag peer as useless/demoted

[non-fungible-nelson]

#5373 ref

Need to re-evaluate post-Merge sync strategy. Q3 perhaps...

[non-fungible-nelson]

#5535 has more context