[Merged by Bors] - use state cache to optimise historical state lookup

[int88]

Issue Addressed

#3873

Proposed Changes

add a cache to optimise historical state lookup.

Additional Info

N/A

[int88]

@michaelsproul PTAL :)

[michaelsproul]

Looks like it's heading in the right direction. Have you done any manual testing to see how the performance is? E.g. try requesting a state at slot 8190 in the period, then request the state one slot later. The 2nd request should be almost instant.

Once the change to the low restore point is made I'll do some testing myself

[michaelsproul]

@int88 Do you mind renaming the flag to --historic-state-cache-size? I'd prefer the longer name to clarify that it only caches states from the freezer DB, and to avoid it conflicting with another state cache added in the future (e.g. my branch #3206 adds a state cache for the hot database).

The implementation is looking really good though, thanks for your efforts! 😊

[int88]

@michaelsproul updated :)

[michaelsproul]

I just tested the performance improvement on one of my archive nodes and it was fantastic!

First I loaded state 6336000 which took 65 seconds:

$ time ssz_state.sh 6336000 > /dev/null

Executed in   65.82 secs      fish           external
   usr time    1.31 millis  400.00 micros    0.91 millis
   sys time   15.44 millis  128.00 micros   15.31 millis

Re-loading the same state from the cache took only 250ms!

time ssz_state.sh 6336000 > /dev/null
________________________________________________________
Executed in  259.70 millis    fish           external
   usr time   12.04 millis  219.00 micros   11.82 millis
   sys time   22.69 millis   74.00 micros   22.62 millis

Loading the state at the next slot was also really fast, around 650ms:

time ssz_state.sh 6336001 > /dev/null

________________________________________________________
Executed in  654.11 millis    fish           external
   usr time   13.86 millis  320.00 micros   13.54 millis
   sys time   17.80 millis  111.00 micros   17.69 millis

This is so awesome I think we should add it to the book. I think a section on this page would be appropriate (maybe with a warning that it can use a lot of memory): https://lighthouse-book.sigmaprime.io/advanced_database.html

[int88]

I just tested the performance improvement on one of my archive nodes and it was fantastic!

First I loaded state 6336000 which took 65 seconds:

$ time ssz_state.sh 6336000 > /dev/null

Executed in   65.82 secs      fish           external
   usr time    1.31 millis  400.00 micros    0.91 millis
   sys time   15.44 millis  128.00 micros   15.31 millis

Re-loading the same state from the cache took only 250ms!

time ssz_state.sh 6336000 > /dev/null
________________________________________________________
Executed in  259.70 millis    fish           external
   usr time   12.04 millis  219.00 micros   11.82 millis
   sys time   22.69 millis   74.00 micros   22.62 millis

Loading the state at the next slot was also really fast, around 650ms:

time ssz_state.sh 6336001 > /dev/null

________________________________________________________
Executed in  654.11 millis    fish           external
   usr time   13.86 millis  320.00 micros   13.54 millis
   sys time   17.80 millis  111.00 micros   17.69 millis

This is so awesome I think we should add it to the book. I think a section on this page would be appropriate (maybe with a warning that it can use a lot of memory): https://lighthouse-book.sigmaprime.io/advanced_database.html

Awesome test! And do you mean add the usage of flag historic-state-cache-size and above test result to the advanced_databsae doc?

[michaelsproul]

Awesome test! And do you mean add the usage of flag historic-state-cache-size and above test result to the advanced_databsae doc?

Just the description of the flag I think 😁 Thanks!

[int88]

@michaelsproul updated!

[michaelsproul]

I just tweaked the docs slightly, but happy to merge!

Thank you @int88 for implementing this long-awaited feature!

[michaelsproul]

bors r+

[bors]

Pull request successfully merged into unstable.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

• arbitrary-check
• beacon-chain-tests
• cargo-audit
• cargo-fmt
• cargo-udeps
• cargo-vendor
• check-benchmarks
• check-msrv
• clippy
• compile-with-beta-compiler
• debug-tests-ubuntu
• dockerfile-ubuntu
• doppelganger-protection-test
• ef-tests-ubuntu
• eth1-simulator-ubuntu
• execution-engine-integration-ubuntu
• merge-transition-ubuntu
• no-eth1-simulator-ubuntu
• op-pool-tests
• release-tests-ubuntu
• release-tests-windows
• slasher-tests
• state-transition-vectors-ubuntu
• syncing-simulator-ubuntu