Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: no state in cache for finalized checkpoint state #6620

Merged
merged 4 commits into from
Apr 5, 2024
Merged

fix: no state in cache for finalized checkpoint state #6620

merged 4 commits into from
Apr 5, 2024

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Apr 2, 2024

Motivation

  • Got this error that cannot find finalized checkpoint state in a 0-historical checkpoint state node
Error: No state in cache for finalized checkpoint state epoch #273222 root 0x100589e1cdf61a69a71d3e8dc0d287e0d86df7e422402566d1c1b00ab7f69efc
    at StatesArchiver.archiveState (file:///usr/src/lodestar/packages/beacon-node/src/chain/archiver/archiveStates.ts:91:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at StatesArchiver.maybeArchiveState (file:///usr/src/lodestar/packages/beacon-node/src/chain/archiver/archiveStates.ts:54:7)
    at JobItemQueue.Archiver.processFinalizedCheckpoint [as itemProcessor] (file:///usr/src/lodestar/packages/beacon-node/src/chain/archiver/index.ts:108:7)
  • This is due to the reorg scenario when processed state doesn't know about the unknown root checkpoint state

Description

  • Cover reorg scenario: also persist any unknown root checkpoint states when processing a state
  • Performance note:
    • In normal condition we persist exactly 1 checkpoint state per epoch
    • In reorg scenario we may persist >1 (most likely 2) checkpoint states per epoch

part of #5968

@codecov Codecov
Copy link

codecov bot commented Apr 2, 2024 •
edited
Loading

Codecov Report

Merging #6620 (16f8c7c) into unstable (b81230a) will increase coverage by 0.08%.
Report is 8 commits behind head on unstable.
The diff coverage is n/a.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #6620      +/-   ##
============================================
+ Coverage     61.61%   61.69%   +0.08%     
============================================
  Files           556      556              
  Lines         58567    58820     +253     
  Branches       1859     1887      +28     
============================================
+ Hits          36084    36287     +203     
- Misses        22442    22492      +50     
  Partials         41       41

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 2, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9f75183 Previous: 2ee8ac6 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 579.75 us/op 635.90 us/op 0.91
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 104.07 us/op 156.87 us/op 0.66
BLS verify - blst-native 1.3745 ms/op 1.5386 ms/op 0.89
BLS verifyMultipleSignatures 3 - blst-native 2.9143 ms/op 3.4106 ms/op 0.85
BLS verifyMultipleSignatures 8 - blst-native 6.3072 ms/op 8.1743 ms/op 0.77
BLS verifyMultipleSignatures 32 - blst-native 23.119 ms/op 24.622 ms/op 0.94
BLS verifyMultipleSignatures 64 - blst-native 45.521 ms/op 51.680 ms/op 0.88
BLS verifyMultipleSignatures 128 - blst-native 90.292 ms/op 94.920 ms/op 0.95
BLS deserializing 10000 signatures 980.89 ms/op 1.0269 s/op 0.96
BLS deserializing 100000 signatures 9.2896 s/op 9.8517 s/op 0.94
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.5096 ms/op 1.3831 ms/op 1.09
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.5809 ms/op 1.5545 ms/op 1.02
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4327 ms/op 2.4079 ms/op 1.01
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.6246 ms/op 3.6214 ms/op 1.00
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.9476 ms/op 6.0754 ms/op 0.98
BLS aggregatePubkeys 32 - blst-native 27.199 us/op 26.262 us/op 1.04
BLS aggregatePubkeys 128 - blst-native 109.33 us/op 105.50 us/op 1.04
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 71.046 ms/op 79.768 ms/op 0.89
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 74.669 ms/op 85.846 ms/op 0.87
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 53.766 ms/op 66.391 ms/op 0.81
getSlashingsAndExits - default max 191.25 us/op 310.78 us/op 0.62
getSlashingsAndExits - 2k 454.66 us/op 788.94 us/op 0.58
proposeBlockBody type=full, size=empty 6.8484 ms/op 9.1927 ms/op 0.74
isKnown best case - 1 super set check 483.00 ns/op 703.00 ns/op 0.69
isKnown normal case - 2 super set checks 446.00 ns/op 747.00 ns/op 0.60
isKnown worse case - 16 super set checks 465.00 ns/op 743.00 ns/op 0.63
InMemoryCheckpointStateCache - add get delete 7.2980 us/op 8.2890 us/op 0.88
validate api signedAggregateAndProof - struct 2.9031 ms/op 3.0011 ms/op 0.97
validate gossip signedAggregateAndProof - struct 2.9054 ms/op 3.1596 ms/op 0.92
validate gossip attestation - vc 640000 1.4299 ms/op 1.4949 ms/op 0.96
batch validate gossip attestation - vc 640000 - chunk 32 173.99 us/op 198.61 us/op 0.88
batch validate gossip attestation - vc 640000 - chunk 64 157.05 us/op 187.26 us/op 0.84
batch validate gossip attestation - vc 640000 - chunk 128 143.91 us/op 178.94 us/op 0.80
batch validate gossip attestation - vc 640000 - chunk 256 134.82 us/op 170.16 us/op 0.79
pickEth1Vote - no votes 1.2694 ms/op 1.5500 ms/op 0.82
pickEth1Vote - max votes 7.6158 ms/op 12.311 ms/op 0.62
pickEth1Vote - Eth1Data hashTreeRoot value x2048 17.238 ms/op 25.291 ms/op 0.68
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.271 ms/op 36.598 ms/op 0.64
pickEth1Vote - Eth1Data fastSerialize value x2048 623.96 us/op 814.62 us/op 0.77
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.5102 ms/op 10.063 ms/op 0.45
bytes32 toHexString 527.00 ns/op 907.00 ns/op 0.58
bytes32 Buffer.toString(hex) 318.00 ns/op 364.00 ns/op 0.87
bytes32 Buffer.toString(hex) from Uint8Array 456.00 ns/op 636.00 ns/op 0.72
bytes32 Buffer.toString(hex) + 0x 321.00 ns/op 355.00 ns/op 0.90
Object access 1 prop 0.17200 ns/op 0.26200 ns/op 0.66
Map access 1 prop 0.15800 ns/op 0.20100 ns/op 0.79
Object get x1000 7.4910 ns/op 8.0750 ns/op 0.93
Map get x1000 0.81700 ns/op 0.98700 ns/op 0.83
Object set x1000 52.078 ns/op 79.595 ns/op 0.65
Map set x1000 41.337 ns/op 57.852 ns/op 0.71
Return object 10000 times 0.24190 ns/op 0.27250 ns/op 0.89
Throw Error 10000 times 4.1081 us/op 4.3351 us/op 0.95
fastMsgIdFn sha256 / 200 bytes 3.4620 us/op 3.5980 us/op 0.96
fastMsgIdFn h32 xxhash / 200 bytes 293.00 ns/op 411.00 ns/op 0.71
fastMsgIdFn h64 xxhash / 200 bytes 360.00 ns/op 458.00 ns/op 0.79
fastMsgIdFn sha256 / 1000 bytes 11.739 us/op 12.830 us/op 0.91
fastMsgIdFn h32 xxhash / 1000 bytes 431.00 ns/op 578.00 ns/op 0.75
fastMsgIdFn h64 xxhash / 1000 bytes 435.00 ns/op 538.00 ns/op 0.81
fastMsgIdFn sha256 / 10000 bytes 106.39 us/op 108.44 us/op 0.98
fastMsgIdFn h32 xxhash / 10000 bytes 1.9780 us/op 2.3610 us/op 0.84
fastMsgIdFn h64 xxhash / 10000 bytes 1.3650 us/op 1.5600 us/op 0.88
send data - 1000 256B messages 19.353 ms/op 27.022 ms/op 0.72
send data - 1000 512B messages 27.447 ms/op 26.623 ms/op 1.03
send data - 1000 1024B messages 39.474 ms/op 55.109 ms/op 0.72
send data - 1000 1200B messages 37.999 ms/op 47.579 ms/op 0.80
send data - 1000 2048B messages 46.519 ms/op 56.963 ms/op 0.82
send data - 1000 4096B messages 43.893 ms/op 55.197 ms/op 0.80
send data - 1000 16384B messages 125.89 ms/op 142.80 ms/op 0.88
send data - 1000 65536B messages 434.40 ms/op 520.01 ms/op 0.84
enrSubnets - fastDeserialize 64 bits 1.4680 us/op 2.1960 us/op 0.67
enrSubnets - ssz BitVector 64 bits 500.00 ns/op 620.00 ns/op 0.81
enrSubnets - fastDeserialize 4 bits 207.00 ns/op 264.00 ns/op 0.78
enrSubnets - ssz BitVector 4 bits 509.00 ns/op 563.00 ns/op 0.90
prioritizePeers score -10:0 att 32-0.1 sync 2-0 120.92 us/op 116.94 us/op 1.03
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 147.69 us/op 146.31 us/op 1.01
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 197.19 us/op 213.83 us/op 0.92
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 342.38 us/op 346.55 us/op 0.99
prioritizePeers score 0:0 att 64-1 sync 4-1 390.55 us/op 397.18 us/op 0.98
array of 16000 items push then shift 1.7417 us/op 1.9067 us/op 0.91
LinkedList of 16000 items push then shift 9.5060 ns/op 9.8350 ns/op 0.97
array of 16000 items push then pop 112.35 ns/op 119.94 ns/op 0.94
LinkedList of 16000 items push then pop 9.2760 ns/op 10.031 ns/op 0.92
array of 24000 items push then shift 2.6911 us/op 2.7503 us/op 0.98
LinkedList of 24000 items push then shift 9.9630 ns/op 9.7220 ns/op 1.02
array of 24000 items push then pop 154.46 ns/op 181.11 ns/op 0.85
LinkedList of 24000 items push then pop 9.2270 ns/op 10.172 ns/op 0.91
intersect bitArray bitLen 8 6.0060 ns/op 6.8120 ns/op 0.88
intersect array and set length 8 75.409 ns/op 99.295 ns/op 0.76
intersect bitArray bitLen 128 38.603 ns/op 47.184 ns/op 0.82
intersect array and set length 128 1.0810 us/op 1.3795 us/op 0.78
bitArray.getTrueBitIndexes() bitLen 128 1.7730 us/op 2.2160 us/op 0.80
bitArray.getTrueBitIndexes() bitLen 248 3.1140 us/op 3.8150 us/op 0.82
bitArray.getTrueBitIndexes() bitLen 512 6.1010 us/op 7.4960 us/op 0.81
Buffer.concat 32 items 1.1290 us/op 1.2720 us/op 0.89
Uint8Array.set 32 items 1.9580 us/op 2.7140 us/op 0.72
Set add up to 64 items then delete first 4.9951 us/op 5.5983 us/op 0.89
OrderedSet add up to 64 items then delete first 7.1033 us/op 7.5992 us/op 0.93
Set add up to 64 items then delete last 5.1308 us/op 5.8359 us/op 0.88
OrderedSet add up to 64 items then delete last 7.2364 us/op 7.2591 us/op 1.00
Set add up to 64 items then delete middle 5.1750 us/op 5.6003 us/op 0.92
OrderedSet add up to 64 items then delete middle 8.3629 us/op 8.8022 us/op 0.95
Set add up to 128 items then delete first 10.757 us/op 11.433 us/op 0.94
OrderedSet add up to 128 items then delete first 14.538 us/op 15.866 us/op 0.92
Set add up to 128 items then delete last 11.084 us/op 11.346 us/op 0.98
OrderedSet add up to 128 items then delete last 15.330 us/op 14.976 us/op 1.02
Set add up to 128 items then delete middle 11.017 us/op 11.097 us/op 0.99
OrderedSet add up to 128 items then delete middle 20.799 us/op 21.645 us/op 0.96
Set add up to 256 items then delete first 22.701 us/op 24.956 us/op 0.91
OrderedSet add up to 256 items then delete first 30.896 us/op 32.207 us/op 0.96
Set add up to 256 items then delete last 21.839 us/op 23.340 us/op 0.94
OrderedSet add up to 256 items then delete last 26.016 us/op 31.580 us/op 0.82
Set add up to 256 items then delete middle 21.402 us/op 23.460 us/op 0.91
OrderedSet add up to 256 items then delete middle 51.707 us/op 54.533 us/op 0.95
transfer serialized Status (84 B) 1.8970 us/op 2.2110 us/op 0.86
copy serialized Status (84 B) 1.4700 us/op 1.5400 us/op 0.95
transfer serialized SignedVoluntaryExit (112 B) 2.0920 us/op 2.2180 us/op 0.94
copy serialized SignedVoluntaryExit (112 B) 1.4850 us/op 1.5520 us/op 0.96
transfer serialized ProposerSlashing (416 B) 2.1500 us/op 3.2050 us/op 0.67
copy serialized ProposerSlashing (416 B) 2.1810 us/op 2.5730 us/op 0.85
transfer serialized Attestation (485 B) 2.8680 us/op 3.1690 us/op 0.91
copy serialized Attestation (485 B) 2.1490 us/op 2.7530 us/op 0.78
transfer serialized AttesterSlashing (33232 B) 2.1520 us/op 2.5490 us/op 0.84
copy serialized AttesterSlashing (33232 B) 7.8530 us/op 9.1410 us/op 0.86
transfer serialized Small SignedBeaconBlock (128000 B) 2.7830 us/op 3.1960 us/op 0.87
copy serialized Small SignedBeaconBlock (128000 B) 22.184 us/op 29.294 us/op 0.76
transfer serialized Avg SignedBeaconBlock (200000 B) 3.0130 us/op 4.3600 us/op 0.69
copy serialized Avg SignedBeaconBlock (200000 B) 27.000 us/op 37.946 us/op 0.71
transfer serialized BlobsSidecar (524380 B) 3.4760 us/op 6.3850 us/op 0.54
copy serialized BlobsSidecar (524380 B) 102.41 us/op 123.36 us/op 0.83
transfer serialized Big SignedBeaconBlock (1000000 B) 2.8840 us/op 4.9770 us/op 0.58
copy serialized Big SignedBeaconBlock (1000000 B) 167.93 us/op 206.45 us/op 0.81
pass gossip attestations to forkchoice per slot 4.1288 ms/op 4.7318 ms/op 0.87
forkChoice updateHead vc 100000 bc 64 eq 0 731.08 us/op 732.94 us/op 1.00
forkChoice updateHead vc 600000 bc 64 eq 0 4.7161 ms/op 5.8280 ms/op 0.81
forkChoice updateHead vc 1000000 bc 64 eq 0 7.3823 ms/op 8.1325 ms/op 0.91
forkChoice updateHead vc 600000 bc 320 eq 0 4.4427 ms/op 4.5894 ms/op 0.97
forkChoice updateHead vc 600000 bc 1200 eq 0 4.6176 ms/op 4.8134 ms/op 0.96
forkChoice updateHead vc 600000 bc 7200 eq 0 5.5239 ms/op 6.1421 ms/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 1000 11.907 ms/op 12.447 ms/op 0.96
forkChoice updateHead vc 600000 bc 64 eq 10000 13.876 ms/op 12.804 ms/op 1.08
forkChoice updateHead vc 600000 bc 64 eq 300000 22.920 ms/op 19.633 ms/op 1.17
computeDeltas 500000 validators 300 proto nodes 7.2657 ms/op 7.8805 ms/op 0.92
computeDeltas 500000 validators 1200 proto nodes 7.1644 ms/op 7.3759 ms/op 0.97
computeDeltas 500000 validators 7200 proto nodes 6.7525 ms/op 7.1460 ms/op 0.94
computeDeltas 750000 validators 300 proto nodes 9.9171 ms/op 10.598 ms/op 0.94
computeDeltas 750000 validators 1200 proto nodes 9.7759 ms/op 11.205 ms/op 0.87
computeDeltas 750000 validators 7200 proto nodes 9.7654 ms/op 11.545 ms/op 0.85
computeDeltas 1400000 validators 300 proto nodes 19.327 ms/op 22.431 ms/op 0.86
computeDeltas 1400000 validators 1200 proto nodes 19.258 ms/op 22.095 ms/op 0.87
computeDeltas 1400000 validators 7200 proto nodes 20.139 ms/op 22.178 ms/op 0.91
computeDeltas 2100000 validators 300 proto nodes 28.651 ms/op 31.726 ms/op 0.90
computeDeltas 2100000 validators 1200 proto nodes 29.555 ms/op 30.904 ms/op 0.96
computeDeltas 2100000 validators 7200 proto nodes 29.190 ms/op 30.670 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei normalcase 3.0837 ms/op 4.4424 ms/op 0.69
altair processAttestation - 250000 vs - 7PWei worstcase 4.0691 ms/op 6.4465 ms/op 0.63
altair processAttestation - setStatus - 1/6 committees join 161.60 us/op 170.25 us/op 0.95
altair processAttestation - setStatus - 1/3 committees join 304.06 us/op 323.30 us/op 0.94
altair processAttestation - setStatus - 1/2 committees join 429.56 us/op 461.13 us/op 0.93
altair processAttestation - setStatus - 2/3 committees join 496.11 us/op 558.28 us/op 0.89
altair processAttestation - setStatus - 4/5 committees join 690.01 us/op 855.60 us/op 0.81
altair processAttestation - setStatus - 100% committees join 835.48 us/op 865.05 us/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase 9.2820 ms/op 9.7398 ms/op 0.95
altair processBlock - 250000 vs - 7PWei normalcase hashState 35.994 ms/op 35.677 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase 46.731 ms/op 46.996 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase hashState 101.32 ms/op 92.600 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei normalcase 3.1673 ms/op 2.9742 ms/op 1.06
phase0 processBlock - 250000 vs - 7PWei worstcase 34.006 ms/op 32.169 ms/op 1.06
altair processEth1Data - 250000 vs - 7PWei normalcase 683.55 us/op 660.04 us/op 1.04
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 13.950 us/op 17.194 us/op 0.81
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 66.897 us/op 85.960 us/op 0.78
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 16.501 us/op 27.861 us/op 0.59
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 18.205 us/op 18.754 us/op 0.97
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 217.86 us/op 211.13 us/op 1.03
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.6244 ms/op 1.5379 ms/op 1.06
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.2006 ms/op 1.9650 ms/op 1.12
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.8607 ms/op 2.0842 ms/op 0.89
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.6320 ms/op 4.3331 ms/op 1.07
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.9078 ms/op 2.8687 ms/op 1.01
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 6.2239 ms/op 6.4625 ms/op 0.96
Tree 40 250000 create 542.23 ms/op 442.90 ms/op 1.22
Tree 40 250000 get(125000) 214.76 ns/op 218.28 ns/op 0.98
Tree 40 250000 set(125000) 1.3529 us/op 3.3396 us/op 0.41
Tree 40 250000 toArray() 24.283 ms/op 33.760 ms/op 0.72
Tree 40 250000 iterate all - toArray() + loop 25.779 ms/op 33.689 ms/op 0.77
Tree 40 250000 iterate all - get(i) 76.822 ms/op 97.083 ms/op 0.79
MutableVector 250000 create 15.438 ms/op 25.615 ms/op 0.60
MutableVector 250000 get(125000) 6.9110 ns/op 8.2590 ns/op 0.84
MutableVector 250000 set(125000) 297.46 ns/op 336.89 ns/op 0.88
MutableVector 250000 toArray() 3.7673 ms/op 5.0121 ms/op 0.75
MutableVector 250000 iterate all - toArray() + loop 3.8750 ms/op 5.4798 ms/op 0.71
MutableVector 250000 iterate all - get(i) 1.5852 ms/op 2.3554 ms/op 0.67
Array 250000 create 3.3112 ms/op 4.0200 ms/op 0.82
Array 250000 clone - spread 1.2989 ms/op 1.7114 ms/op 0.76
Array 250000 get(125000) 1.1200 ns/op 1.2310 ns/op 0.91
Array 250000 set(125000) 4.2040 ns/op 4.3450 ns/op 0.97
Array 250000 iterate all - loop 169.13 us/op 178.58 us/op 0.95
effectiveBalanceIncrements clone Uint8Array 300000 41.370 us/op 35.683 us/op 1.16
effectiveBalanceIncrements clone MutableVector 300000 406.00 ns/op 392.00 ns/op 1.04
effectiveBalanceIncrements rw all Uint8Array 300000 204.87 us/op 222.58 us/op 0.92
effectiveBalanceIncrements rw all MutableVector 300000 117.56 ms/op 100.80 ms/op 1.17
phase0 afterProcessEpoch - 250000 vs - 7PWei 120.85 ms/op 132.47 ms/op 0.91
phase0 beforeProcessEpoch - 250000 vs - 7PWei 62.631 ms/op 41.464 ms/op 1.51
altair processEpoch - mainnet_e81889 529.01 ms/op 612.31 ms/op 0.86
mainnet_e81889 - altair beforeProcessEpoch 86.552 ms/op 100.38 ms/op 0.86
mainnet_e81889 - altair processJustificationAndFinalization 15.621 us/op 19.199 us/op 0.81
mainnet_e81889 - altair processInactivityUpdates 6.4709 ms/op 7.5425 ms/op 0.86
mainnet_e81889 - altair processRewardsAndPenalties 55.640 ms/op 65.058 ms/op 0.86
mainnet_e81889 - altair processRegistryUpdates 2.2820 us/op 3.2560 us/op 0.70
mainnet_e81889 - altair processSlashings 478.00 ns/op 565.00 ns/op 0.85
mainnet_e81889 - altair processEth1DataReset 715.00 ns/op 567.00 ns/op 1.26
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4389 ms/op 1.9332 ms/op 0.74
mainnet_e81889 - altair processSlashingsReset 3.8850 us/op 4.8270 us/op 0.80
mainnet_e81889 - altair processRandaoMixesReset 6.7800 us/op 6.8220 us/op 0.99
mainnet_e81889 - altair processHistoricalRootsUpdate 1.2650 us/op 1.0740 us/op 1.18
mainnet_e81889 - altair processParticipationFlagUpdates 3.6600 us/op 2.0420 us/op 1.79
mainnet_e81889 - altair processSyncCommitteeUpdates 1.5020 us/op 653.00 ns/op 2.30
mainnet_e81889 - altair afterProcessEpoch 125.06 ms/op 123.50 ms/op 1.01
capella processEpoch - mainnet_e217614 2.2255 s/op 2.0958 s/op 1.06
mainnet_e217614 - capella beforeProcessEpoch 521.83 ms/op 512.10 ms/op 1.02
mainnet_e217614 - capella processJustificationAndFinalization 16.131 us/op 15.602 us/op 1.03
mainnet_e217614 - capella processInactivityUpdates 19.882 ms/op 18.666 ms/op 1.07
mainnet_e217614 - capella processRewardsAndPenalties 516.74 ms/op 569.49 ms/op 0.91
mainnet_e217614 - capella processRegistryUpdates 15.773 us/op 23.592 us/op 0.67
mainnet_e217614 - capella processSlashings 720.00 ns/op 644.00 ns/op 1.12
mainnet_e217614 - capella processEth1DataReset 593.00 ns/op 592.00 ns/op 1.00
mainnet_e217614 - capella processEffectiveBalanceUpdates 17.253 ms/op 14.787 ms/op 1.17
mainnet_e217614 - capella processSlashingsReset 4.3180 us/op 4.7330 us/op 0.91
mainnet_e217614 - capella processRandaoMixesReset 6.0880 us/op 4.9650 us/op 1.23
mainnet_e217614 - capella processHistoricalRootsUpdate 848.00 ns/op 904.00 ns/op 0.94
mainnet_e217614 - capella processParticipationFlagUpdates 3.5400 us/op 1.6110 us/op 2.20
mainnet_e217614 - capella afterProcessEpoch 370.85 ms/op 330.97 ms/op 1.12
phase0 processEpoch - mainnet_e58758 487.86 ms/op 557.95 ms/op 0.87
mainnet_e58758 - phase0 beforeProcessEpoch 152.94 ms/op 161.24 ms/op 0.95
mainnet_e58758 - phase0 processJustificationAndFinalization 16.872 us/op 30.728 us/op 0.55
mainnet_e58758 - phase0 processRewardsAndPenalties 49.498 ms/op 62.771 ms/op 0.79
mainnet_e58758 - phase0 processRegistryUpdates 9.9340 us/op 20.725 us/op 0.48
mainnet_e58758 - phase0 processSlashings 460.00 ns/op 851.00 ns/op 0.54
mainnet_e58758 - phase0 processEth1DataReset 504.00 ns/op 1.2920 us/op 0.39
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.1136 ms/op 1.3681 ms/op 0.81
mainnet_e58758 - phase0 processSlashingsReset 3.0520 us/op 4.4510 us/op 0.69
mainnet_e58758 - phase0 processRandaoMixesReset 6.5160 us/op 6.5650 us/op 0.99
mainnet_e58758 - phase0 processHistoricalRootsUpdate 924.00 ns/op 967.00 ns/op 0.96
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.8570 us/op 7.1000 us/op 0.54
mainnet_e58758 - phase0 afterProcessEpoch 106.55 ms/op 114.29 ms/op 0.93
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.4265 ms/op 1.8884 ms/op 0.76
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5131 ms/op 1.9984 ms/op 0.76
altair processInactivityUpdates - 250000 normalcase 32.884 ms/op 50.250 ms/op 0.65
altair processInactivityUpdates - 250000 worstcase 24.236 ms/op 58.848 ms/op 0.41
phase0 processRegistryUpdates - 250000 normalcase 12.566 us/op 21.220 us/op 0.59
phase0 processRegistryUpdates - 250000 badcase_full_deposits 430.08 us/op 454.13 us/op 0.95
phase0 processRegistryUpdates - 250000 worstcase 0.5 144.29 ms/op 170.08 ms/op 0.85
altair processRewardsAndPenalties - 250000 normalcase 58.021 ms/op 114.88 ms/op 0.51
altair processRewardsAndPenalties - 250000 worstcase 76.757 ms/op 71.461 ms/op 1.07
phase0 getAttestationDeltas - 250000 normalcase 10.500 ms/op 13.534 ms/op 0.78
phase0 getAttestationDeltas - 250000 worstcase 10.998 ms/op 12.466 ms/op 0.88
phase0 processSlashings - 250000 worstcase 107.28 us/op 131.56 us/op 0.82
altair processSyncCommitteeUpdates - 250000 173.75 ms/op 190.98 ms/op 0.91
BeaconState.hashTreeRoot - No change 457.00 ns/op 749.00 ns/op 0.61
BeaconState.hashTreeRoot - 1 full validator 170.90 us/op 152.85 us/op 1.12
BeaconState.hashTreeRoot - 32 full validator 1.8840 ms/op 1.5488 ms/op 1.22
BeaconState.hashTreeRoot - 512 full validator 20.281 ms/op 17.619 ms/op 1.15
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 211.56 us/op 188.18 us/op 1.12
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.8237 ms/op 2.5827 ms/op 1.09
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 33.303 ms/op 29.838 ms/op 1.12
BeaconState.hashTreeRoot - 1 balances 163.42 us/op 141.01 us/op 1.16
BeaconState.hashTreeRoot - 32 balances 1.6461 ms/op 1.4258 ms/op 1.15
BeaconState.hashTreeRoot - 512 balances 14.056 ms/op 15.466 ms/op 0.91
BeaconState.hashTreeRoot - 250000 balances 230.25 ms/op 220.10 ms/op 1.05
aggregationBits - 2048 els - zipIndexesInBitList 32.535 us/op 42.329 us/op 0.77
byteArrayEquals 32 76.755 ns/op 79.015 ns/op 0.97
Buffer.compare 32 57.093 ns/op 62.165 ns/op 0.92
byteArrayEquals 1024 2.0999 us/op 2.1892 us/op 0.96
Buffer.compare 1024 74.139 ns/op 77.372 ns/op 0.96
byteArrayEquals 16384 33.408 us/op 35.261 us/op 0.95
Buffer.compare 16384 277.39 ns/op 284.83 ns/op 0.97
byteArrayEquals 123687377 249.46 ms/op 261.10 ms/op 0.96
Buffer.compare 123687377 6.8172 ms/op 8.6782 ms/op 0.79
byteArrayEquals 32 - diff last byte 74.589 ns/op 77.274 ns/op 0.97
Buffer.compare 32 - diff last byte 58.625 ns/op 63.197 ns/op 0.93
byteArrayEquals 1024 - diff last byte 2.1155 us/op 2.1144 us/op 1.00
Buffer.compare 1024 - diff last byte 75.373 ns/op 77.786 ns/op 0.97
byteArrayEquals 16384 - diff last byte 35.337 us/op 33.585 us/op 1.05
Buffer.compare 16384 - diff last byte 289.32 ns/op 276.28 ns/op 1.05
byteArrayEquals 123687377 - diff last byte 253.24 ms/op 293.98 ms/op 0.86
Buffer.compare 123687377 - diff last byte 7.4835 ms/op 9.7728 ms/op 0.77
byteArrayEquals 32 - random bytes 6.2180 ns/op 7.1860 ns/op 0.87
Buffer.compare 32 - random bytes 62.214 ns/op 70.141 ns/op 0.89
byteArrayEquals 1024 - random bytes 5.5630 ns/op 7.0490 ns/op 0.79
Buffer.compare 1024 - random bytes 63.035 ns/op 70.393 ns/op 0.90
byteArrayEquals 16384 - random bytes 5.6030 ns/op 6.5060 ns/op 0.86
Buffer.compare 16384 - random bytes 62.004 ns/op 76.472 ns/op 0.81
byteArrayEquals 123687377 - random bytes 9.4200 ns/op 11.090 ns/op 0.85
Buffer.compare 123687377 - random bytes 73.920 ns/op 81.050 ns/op 0.91
regular array get 100000 times 47.653 us/op 48.257 us/op 0.99
wrappedArray get 100000 times 46.785 us/op 47.691 us/op 0.98
arrayWithProxy get 100000 times 15.238 ms/op 15.441 ms/op 0.99
ssz.Root.equals 59.661 ns/op 56.264 ns/op 1.06
byteArrayEquals 61.670 ns/op 55.652 ns/op 1.11
Buffer.compare 13.046 ns/op 13.218 ns/op 0.99
shuffle list - 16384 els 9.8054 ms/op 8.9683 ms/op 1.09
shuffle list - 250000 els 130.89 ms/op 144.49 ms/op 0.91
processSlot - 1 slots 19.681 us/op 19.180 us/op 1.03
processSlot - 32 slots 3.5829 ms/op 3.8006 ms/op 0.94
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 63.210 ms/op 71.322 ms/op 0.89
getCommitteeAssignments - req 1 vs - 250000 vc 2.7904 ms/op 2.9561 ms/op 0.94
getCommitteeAssignments - req 100 vs - 250000 vc 4.0496 ms/op 4.4318 ms/op 0.91
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3928 ms/op 4.6910 ms/op 0.94
findModifiedValidators - 10000 modified validators 373.30 ms/op 518.56 ms/op 0.72
findModifiedValidators - 1000 modified validators 223.14 ms/op 298.90 ms/op 0.75
findModifiedValidators - 100 modified validators 187.25 ms/op 245.99 ms/op 0.76
findModifiedValidators - 10 modified validators 200.54 ms/op 239.49 ms/op 0.84
findModifiedValidators - 1 modified validators 188.86 ms/op 223.01 ms/op 0.85
findModifiedValidators - no difference 218.20 ms/op 261.02 ms/op 0.84
compare ViewDUs 4.6297 s/op 5.6094 s/op 0.83
compare each validator Uint8Array 1.6001 s/op 1.7758 s/op 0.90
compare ViewDU to Uint8Array 1.4122 s/op 1.3483 s/op 1.05
migrate state 1000000 validators, 24 modified, 0 new 888.64 ms/op 729.41 ms/op 1.22
migrate state 1000000 validators, 1700 modified, 1000 new 1.2065 s/op 1.0226 s/op 1.18
migrate state 1000000 validators, 3400 modified, 2000 new 1.3947 s/op 1.2632 s/op 1.10
migrate state 1500000 validators, 24 modified, 0 new 833.65 ms/op 802.54 ms/op 1.04
migrate state 1500000 validators, 1700 modified, 1000 new 1.1016 s/op 1.0722 s/op 1.03
migrate state 1500000 validators, 3400 modified, 2000 new 1.2889 s/op 1.3577 s/op 0.95
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.6400 ns/op 5.3000 ns/op 0.88
state getBlockRootAtSlot - 250000 vs - 7PWei 639.05 ns/op 604.79 ns/op 1.06
computeProposers - vc 250000 9.2704 ms/op 11.025 ms/op 0.84
computeEpochShuffling - vc 250000 128.75 ms/op 134.72 ms/op 0.96
getNextSyncCommittee - vc 250000 154.19 ms/op 175.41 ms/op 0.88
computeSigningRoot for AttestationData 24.095 us/op 27.808 us/op 0.87
hash AttestationData serialized data then Buffer.toString(base64) 2.4596 us/op 2.4114 us/op 1.02
toHexString serialized data 1.1100 us/op 1.1646 us/op 0.95
Buffer.toString(base64) 238.79 ns/op 237.49 ns/op 1.01

by benchmarkbot/action

@philknows philknows added this to the v1.18.0 milestone Apr 3, 2024
@twoeths
Copy link
Contributor Author

twoeths commented Apr 3, 2024

e2e tests passed, ready for review

@twoeths twoeths marked this pull request as ready for review April 3, 2024 05:43
@twoeths twoeths requested a review from a team as a code owner April 3, 2024 05:43
@ensi321
Copy link
Contributor

ensi321 commented Apr 5, 2024

Looks good overall. Minor comments

@twoeths twoeths merged commit e202592 into unstable Apr 5, 2024
20 checks passed
@twoeths twoeths deleted the tuyen/n_historical_state_store_unknown_root_cp branch April 5, 2024 09:54
twoeths
twoeths commented Apr 5, 2024
View reviewed changes
packages/beacon-node/src/chain/stateCache/persistentCheckpointsCache.ts
@@ -652,7 +652,7 @@ export class PersistentCheckpointStateCache implements CheckpointStateCache {
const prevEpochRoot = toHexString(getBlockRootAtSlot(state, epochBoundarySlot - 1));

// for each epoch, usually there are 2 rootHexes respective to the 2 checkpoint states: Previous Root Checkpoint State and Current Root Checkpoint State
const cpRootHexes = new Set(this.epochIndex.get(epoch) ?? []);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewkeil this addressed your comment during offline review

@twoeths twoeths mentioned this pull request Apr 23, 2024
Merged
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.18.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ensi321 ensi321 ensi321 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.18.0
Development

Successfully merging this pull request may close these issues.
4 participants
@twoeths @ensi321 @wemeetagain @philknows