Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: remove unfinalized pubkey cache #7230

Merged
merged 3 commits into from
Nov 25, 2024
Merged

feat: remove unfinalized pubkey cache #7230

merged 3 commits into from
Nov 25, 2024

Conversation

ensi321
Copy link
Contributor

@ensi321 ensi321 commented Nov 17, 2024 •
edited
Loading

Background

The original EIP-6110 proposes to process deposits in per-slot block processing which requires clients to store unfinalized validators and their indices in pubkey cache. Our solution was to introduce unfinalizedPubkey2index in EpochCache to separate unfinalized validators with finalized validators in case of re-org.

ethereum/consensus-specs#3818 updates EIP-6110 to take advantage of pending deposit queue from EIP-7251 such that

  1. Deposits are added to pending deposit queue during block processing
  2. Pending deposits are dequeued during epoch processing
  3. Only deposits with slot that has been finalized will be processed.

As such, re-org will not impact/change validator index assignment of a newly joined validator and thus there is no need for Lodestar to maintain unfinalizedPubkey2index.

Description

This PR contains the following:

  • Remove unfinalizedPubkey2index from EpochCache and related perf and memory test
  • Remove historicalValidatorLengths from EpochCache
  • Remove immutable js dependency
  • Any new validator pubkey will be added to EpochCache.pubkey2index
  • Remove related metrics

Follow up of #7131

@ensi321 ensi321 mentioned this pull request Nov 17, 2024
Open
12 tasks
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 17, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: cd8f392 Previous: 25c2ee5 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.9667 ms/op 1.9392 ms/op 1.01
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 72.258 us/op 43.709 us/op 1.65
BLS verify - blst 925.05 us/op 758.45 us/op 1.22
BLS verifyMultipleSignatures 3 - blst 1.4023 ms/op 1.1392 ms/op 1.23
BLS verifyMultipleSignatures 8 - blst 1.7826 ms/op 1.6111 ms/op 1.11
BLS verifyMultipleSignatures 32 - blst 4.7383 ms/op 4.7233 ms/op 1.00
BLS verifyMultipleSignatures 64 - blst 8.4176 ms/op 8.8395 ms/op 0.95
BLS verifyMultipleSignatures 128 - blst 16.862 ms/op 16.762 ms/op 1.01
BLS deserializing 10000 signatures 636.02 ms/op 667.45 ms/op 0.95
BLS deserializing 100000 signatures 6.3070 s/op 6.8311 s/op 0.92
BLS verifyMultipleSignatures - same message - 3 - blst 936.65 us/op 941.34 us/op 1.00
BLS verifyMultipleSignatures - same message - 8 - blst 1.0709 ms/op 1.0598 ms/op 1.01
BLS verifyMultipleSignatures - same message - 32 - blst 1.7016 ms/op 1.7221 ms/op 0.99
BLS verifyMultipleSignatures - same message - 64 - blst 2.5196 ms/op 2.6352 ms/op 0.96
BLS verifyMultipleSignatures - same message - 128 - blst 4.0987 ms/op 4.4194 ms/op 0.93
BLS aggregatePubkeys 32 - blst 18.022 us/op 19.713 us/op 0.91
BLS aggregatePubkeys 128 - blst 63.563 us/op 70.130 us/op 0.91
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 60.718 ms/op 52.783 ms/op 1.15
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 61.608 ms/op 61.743 ms/op 1.00
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 34.877 ms/op 38.793 ms/op 0.90
getSlashingsAndExits - default max 82.526 us/op 98.766 us/op 0.84
getSlashingsAndExits - 2k 292.63 us/op 350.39 us/op 0.84
proposeBlockBody type=full, size=empty 5.6755 ms/op 6.5631 ms/op 0.86
isKnown best case - 1 super set check 498.00 ns/op 329.00 ns/op 1.51
isKnown normal case - 2 super set checks 493.00 ns/op 336.00 ns/op 1.47
isKnown worse case - 16 super set checks 458.00 ns/op 303.00 ns/op 1.51
InMemoryCheckpointStateCache - add get delete 3.3860 us/op 3.0060 us/op 1.13
validate api signedAggregateAndProof - struct 1.7681 ms/op 1.3824 ms/op 1.28
validate gossip signedAggregateAndProof - struct 1.6317 ms/op 1.4387 ms/op 1.13
batch validate gossip attestation - vc 640000 - chunk 32 128.94 us/op 123.57 us/op 1.04
batch validate gossip attestation - vc 640000 - chunk 64 108.61 us/op 108.26 us/op 1.00
batch validate gossip attestation - vc 640000 - chunk 128 100.86 us/op 109.69 us/op 0.92
batch validate gossip attestation - vc 640000 - chunk 256 111.62 us/op 103.58 us/op 1.08
pickEth1Vote - no votes 1.0653 ms/op 1.1594 ms/op 0.92
pickEth1Vote - max votes 8.3033 ms/op 7.0794 ms/op 1.17
pickEth1Vote - Eth1Data hashTreeRoot value x2048 19.256 ms/op 17.201 ms/op 1.12
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 27.861 ms/op 22.483 ms/op 1.24
pickEth1Vote - Eth1Data fastSerialize value x2048 501.52 us/op 539.55 us/op 0.93
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.0869 ms/op 5.0929 ms/op 0.61
bytes32 toHexString 973.00 ns/op 435.00 ns/op 2.24
bytes32 Buffer.toString(hex) 628.00 ns/op 255.00 ns/op 2.46
bytes32 Buffer.toString(hex) from Uint8Array 717.00 ns/op 358.00 ns/op 2.00
bytes32 Buffer.toString(hex) + 0x 525.00 ns/op 251.00 ns/op 2.09
Object access 1 prop 0.40500 ns/op 0.14000 ns/op 2.89
Map access 1 prop 0.33800 ns/op 0.13200 ns/op 2.56
Object get x1000 5.3140 ns/op 5.8530 ns/op 0.91
Map get x1000 5.9830 ns/op 6.3010 ns/op 0.95
Object set x1000 52.301 ns/op 45.077 ns/op 1.16
Map set x1000 35.779 ns/op 28.940 ns/op 1.24
Return object 10000 times 0.31380 ns/op 0.30030 ns/op 1.04
Throw Error 10000 times 3.0570 us/op 3.3739 us/op 0.91
toHex 157.27 ns/op 176.27 ns/op 0.89
Buffer.from 149.54 ns/op 163.55 ns/op 0.91
shared Buffer 88.982 ns/op 103.34 ns/op 0.86
fastMsgIdFn sha256 / 200 bytes 2.2390 us/op 2.3440 us/op 0.96
fastMsgIdFn h32 xxhash / 200 bytes 577.00 ns/op 287.00 ns/op 2.01
fastMsgIdFn h64 xxhash / 200 bytes 562.00 ns/op 288.00 ns/op 1.95
fastMsgIdFn sha256 / 1000 bytes 6.5230 us/op 7.4140 us/op 0.88
fastMsgIdFn h32 xxhash / 1000 bytes 714.00 ns/op 414.00 ns/op 1.72
fastMsgIdFn h64 xxhash / 1000 bytes 564.00 ns/op 351.00 ns/op 1.61
fastMsgIdFn sha256 / 10000 bytes 50.467 us/op 64.026 us/op 0.79
fastMsgIdFn h32 xxhash / 10000 bytes 2.0670 us/op 1.9210 us/op 1.08
fastMsgIdFn h64 xxhash / 10000 bytes 1.3940 us/op 1.2330 us/op 1.13
send data - 1000 256B messages 14.130 ms/op 13.525 ms/op 1.04
send data - 1000 512B messages 20.180 ms/op 17.269 ms/op 1.17
send data - 1000 1024B messages 32.264 ms/op 27.303 ms/op 1.18
send data - 1000 1200B messages 32.005 ms/op 19.474 ms/op 1.64
send data - 1000 2048B messages 39.216 ms/op 31.897 ms/op 1.23
send data - 1000 4096B messages 38.444 ms/op 31.012 ms/op 1.24
send data - 1000 16384B messages 86.646 ms/op 67.051 ms/op 1.29
send data - 1000 65536B messages 296.24 ms/op 199.23 ms/op 1.49
enrSubnets - fastDeserialize 64 bits 1.1700 us/op 1.0980 us/op 1.07
enrSubnets - ssz BitVector 64 bits 576.00 ns/op 372.00 ns/op 1.55
enrSubnets - fastDeserialize 4 bits 377.00 ns/op 162.00 ns/op 2.33
enrSubnets - ssz BitVector 4 bits 557.00 ns/op 372.00 ns/op 1.50
prioritizePeers score -10:0 att 32-0.1 sync 2-0 179.60 us/op 152.48 us/op 1.18
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 153.79 us/op 152.39 us/op 1.01
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 359.43 us/op 246.72 us/op 1.46
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 576.06 us/op 414.05 us/op 1.39
prioritizePeers score 0:0 att 64-1 sync 4-1 520.83 us/op 601.56 us/op 0.87
array of 16000 items push then shift 1.3060 us/op 1.6888 us/op 0.77
LinkedList of 16000 items push then shift 6.4680 ns/op 7.6010 ns/op 0.85
array of 16000 items push then pop 107.88 ns/op 125.49 ns/op 0.86
LinkedList of 16000 items push then pop 6.4000 ns/op 7.4100 ns/op 0.86
array of 24000 items push then shift 1.8517 us/op 2.4679 us/op 0.75
LinkedList of 24000 items push then shift 7.2480 ns/op 7.6400 ns/op 0.95
array of 24000 items push then pop 162.35 ns/op 166.54 ns/op 0.97
LinkedList of 24000 items push then pop 6.4550 ns/op 7.4470 ns/op 0.87
intersect bitArray bitLen 8 5.3860 ns/op 6.5870 ns/op 0.82
intersect array and set length 8 42.279 ns/op 46.634 ns/op 0.91
intersect bitArray bitLen 128 26.302 ns/op 29.705 ns/op 0.89
intersect array and set length 128 770.82 ns/op 687.28 ns/op 1.12
bitArray.getTrueBitIndexes() bitLen 128 2.4310 us/op 1.7350 us/op 1.40
bitArray.getTrueBitIndexes() bitLen 248 4.0450 us/op 3.5170 us/op 1.15
bitArray.getTrueBitIndexes() bitLen 512 9.3150 us/op 7.1450 us/op 1.30
Buffer.concat 32 items 1.0640 us/op 974.00 ns/op 1.09
Uint8Array.set 32 items 1.8870 us/op 1.6920 us/op 1.12
Buffer.copy 2.1500 us/op 1.9060 us/op 1.13
Uint8Array.set - with subarray 3.0960 us/op 2.8260 us/op 1.10
Uint8Array.set - without subarray 2.0450 us/op 1.5680 us/op 1.30
getUint32 - dataview 417.00 ns/op 264.00 ns/op 1.58
getUint32 - manual 362.00 ns/op 172.00 ns/op 2.10
Set add up to 64 items then delete first 1.7655 us/op 2.2436 us/op 0.79
OrderedSet add up to 64 items then delete first 2.9295 us/op 3.4037 us/op 0.86
Set add up to 64 items then delete last 2.2810 us/op 2.5686 us/op 0.89
OrderedSet add up to 64 items then delete last 3.2257 us/op 4.0455 us/op 0.80
Set add up to 64 items then delete middle 2.2631 us/op 2.6041 us/op 0.87
OrderedSet add up to 64 items then delete middle 4.5865 us/op 5.5001 us/op 0.83
Set add up to 128 items then delete first 3.9730 us/op 5.2212 us/op 0.76
OrderedSet add up to 128 items then delete first 5.9049 us/op 7.4580 us/op 0.79
Set add up to 128 items then delete last 3.9516 us/op 5.4177 us/op 0.73
OrderedSet add up to 128 items then delete last 6.1544 us/op 7.9275 us/op 0.78
Set add up to 128 items then delete middle 3.9273 us/op 4.9760 us/op 0.79
OrderedSet add up to 128 items then delete middle 11.996 us/op 13.781 us/op 0.87
Set add up to 256 items then delete first 7.7942 us/op 10.782 us/op 0.72
OrderedSet add up to 256 items then delete first 11.734 us/op 15.661 us/op 0.75
Set add up to 256 items then delete last 7.7861 us/op 10.499 us/op 0.74
OrderedSet add up to 256 items then delete last 12.775 us/op 15.875 us/op 0.80
Set add up to 256 items then delete middle 7.6181 us/op 10.205 us/op 0.75
OrderedSet add up to 256 items then delete middle 34.702 us/op 40.557 us/op 0.86
transfer serialized Status (84 B) 1.4290 us/op 1.3640 us/op 1.05
copy serialized Status (84 B) 1.2990 us/op 1.2390 us/op 1.05
transfer serialized SignedVoluntaryExit (112 B) 1.5470 us/op 1.6320 us/op 0.95
copy serialized SignedVoluntaryExit (112 B) 1.3160 us/op 1.3280 us/op 0.99
transfer serialized ProposerSlashing (416 B) 2.1340 us/op 1.8810 us/op 1.13
copy serialized ProposerSlashing (416 B) 1.8460 us/op 1.9410 us/op 0.95
transfer serialized Attestation (485 B) 2.0730 us/op 1.8700 us/op 1.11
copy serialized Attestation (485 B) 2.0530 us/op 1.8110 us/op 1.13
transfer serialized AttesterSlashing (33232 B) 2.7700 us/op 2.2660 us/op 1.22
copy serialized AttesterSlashing (33232 B) 8.7320 us/op 5.4710 us/op 1.60
transfer serialized Small SignedBeaconBlock (128000 B) 3.6340 us/op 3.2060 us/op 1.13
copy serialized Small SignedBeaconBlock (128000 B) 23.509 us/op 19.032 us/op 1.24
transfer serialized Avg SignedBeaconBlock (200000 B) 3.8920 us/op 4.5530 us/op 0.85
copy serialized Avg SignedBeaconBlock (200000 B) 33.866 us/op 27.905 us/op 1.21
transfer serialized BlobsSidecar (524380 B) 5.2700 us/op 3.1540 us/op 1.67
copy serialized BlobsSidecar (524380 B) 143.48 us/op 82.555 us/op 1.74
transfer serialized Big SignedBeaconBlock (1000000 B) 3.9240 us/op 2.6440 us/op 1.48
copy serialized Big SignedBeaconBlock (1000000 B) 145.39 us/op 222.11 us/op 0.65
pass gossip attestations to forkchoice per slot 2.5342 ms/op 2.8148 ms/op 0.90
forkChoice updateHead vc 100000 bc 64 eq 0 345.98 us/op 444.89 us/op 0.78
forkChoice updateHead vc 600000 bc 64 eq 0 2.3422 ms/op 2.8132 ms/op 0.83
forkChoice updateHead vc 1000000 bc 64 eq 0 3.7829 ms/op 4.8467 ms/op 0.78
forkChoice updateHead vc 600000 bc 320 eq 0 2.3595 ms/op 2.8516 ms/op 0.83
forkChoice updateHead vc 600000 bc 1200 eq 0 2.3153 ms/op 2.8909 ms/op 0.80
forkChoice updateHead vc 600000 bc 7200 eq 0 3.2627 ms/op 3.3225 ms/op 0.98
forkChoice updateHead vc 600000 bc 64 eq 1000 9.3441 ms/op 10.344 ms/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 10000 9.4508 ms/op 10.717 ms/op 0.88
forkChoice updateHead vc 600000 bc 64 eq 300000 14.809 ms/op 13.503 ms/op 1.10
computeDeltas 500000 validators 300 proto nodes 3.3886 ms/op 3.9685 ms/op 0.85
computeDeltas 500000 validators 1200 proto nodes 3.4506 ms/op 3.9902 ms/op 0.86
computeDeltas 500000 validators 7200 proto nodes 3.3182 ms/op 4.1140 ms/op 0.81
computeDeltas 750000 validators 300 proto nodes 4.9513 ms/op 5.9279 ms/op 0.84
computeDeltas 750000 validators 1200 proto nodes 4.8617 ms/op 5.9474 ms/op 0.82
computeDeltas 750000 validators 7200 proto nodes 4.7873 ms/op 5.9845 ms/op 0.80
computeDeltas 1400000 validators 300 proto nodes 8.9892 ms/op 11.322 ms/op 0.79
computeDeltas 1400000 validators 1200 proto nodes 9.3201 ms/op 11.371 ms/op 0.82
computeDeltas 1400000 validators 7200 proto nodes 8.9601 ms/op 11.196 ms/op 0.80
computeDeltas 2100000 validators 300 proto nodes 13.628 ms/op 17.038 ms/op 0.80
computeDeltas 2100000 validators 1200 proto nodes 13.765 ms/op 16.589 ms/op 0.83
computeDeltas 2100000 validators 7200 proto nodes 13.547 ms/op 16.361 ms/op 0.83
altair processAttestation - 250000 vs - 7PWei normalcase 1.4325 ms/op 1.9029 ms/op 0.75
altair processAttestation - 250000 vs - 7PWei worstcase 2.1310 ms/op 2.5582 ms/op 0.83
altair processAttestation - setStatus - 1/6 committees join 64.685 us/op 87.042 us/op 0.74
altair processAttestation - setStatus - 1/3 committees join 132.69 us/op 181.14 us/op 0.73
altair processAttestation - setStatus - 1/2 committees join 180.19 us/op 243.90 us/op 0.74
altair processAttestation - setStatus - 2/3 committees join 247.46 us/op 319.95 us/op 0.77
altair processAttestation - setStatus - 4/5 committees join 386.17 us/op 448.97 us/op 0.86
altair processAttestation - setStatus - 100% committees join 459.66 us/op 543.43 us/op 0.85
altair processBlock - 250000 vs - 7PWei normalcase 4.7798 ms/op 5.8064 ms/op 0.82
altair processBlock - 250000 vs - 7PWei normalcase hashState 25.437 ms/op 25.345 ms/op 1.00
altair processBlock - 250000 vs - 7PWei worstcase 33.036 ms/op 31.729 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase hashState 75.459 ms/op 61.498 ms/op 1.23
phase0 processBlock - 250000 vs - 7PWei normalcase 1.8889 ms/op 1.6901 ms/op 1.12
phase0 processBlock - 250000 vs - 7PWei worstcase 20.430 ms/op 19.515 ms/op 1.05
altair processEth1Data - 250000 vs - 7PWei normalcase 257.33 us/op 312.05 us/op 0.82
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 4.0870 us/op 5.2620 us/op 0.78
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 29.445 us/op 34.342 us/op 0.86
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 4.4730 us/op 8.6090 us/op 0.52
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 4.6840 us/op 5.5600 us/op 0.84
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 122.22 us/op 128.74 us/op 0.95
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.3928 ms/op 1.0314 ms/op 1.35
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.0767 ms/op 1.5189 ms/op 0.71
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.1809 ms/op 1.4575 ms/op 0.81
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.8572 ms/op 3.6539 ms/op 0.78
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.2412 ms/op 1.5573 ms/op 0.80
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 2.8010 ms/op 3.8880 ms/op 0.72
Tree 40 250000 create 188.15 ms/op 214.76 ms/op 0.88
Tree 40 250000 get(125000) 111.06 ns/op 148.03 ns/op 0.75
Tree 40 250000 set(125000) 523.87 ns/op 732.45 ns/op 0.72
Tree 40 250000 toArray() 11.854 ms/op 20.196 ms/op 0.59
Tree 40 250000 iterate all - toArray() + loop 11.457 ms/op 17.037 ms/op 0.67
Tree 40 250000 iterate all - get(i) 44.232 ms/op 50.518 ms/op 0.88
Array 250000 create 2.5896 ms/op 2.9219 ms/op 0.89
Array 250000 clone - spread 1.1789 ms/op 1.3226 ms/op 0.89
Array 250000 get(125000) 0.61800 ns/op 0.42200 ns/op 1.46
Array 250000 set(125000) 0.59400 ns/op 0.47100 ns/op 1.26
Array 250000 iterate all - loop 84.600 us/op 105.36 us/op 0.80
phase0 afterProcessEpoch - 250000 vs - 7PWei 42.559 ms/op 49.106 ms/op 0.87
Array.fill - length 1000000 2.4218 ms/op 3.2852 ms/op 0.74
Array push - length 1000000 14.865 ms/op 16.757 ms/op 0.89
Array.get 0.26056 ns/op 0.27177 ns/op 0.96
Uint8Array.get 0.34174 ns/op 0.42495 ns/op 0.80
phase0 beforeProcessEpoch - 250000 vs - 7PWei 13.442 ms/op 15.898 ms/op 0.85
altair processEpoch - mainnet_e81889 333.80 ms/op 283.49 ms/op 1.18
mainnet_e81889 - altair beforeProcessEpoch 17.480 ms/op 18.433 ms/op 0.95
mainnet_e81889 - altair processJustificationAndFinalization 12.847 us/op 12.057 us/op 1.07
mainnet_e81889 - altair processInactivityUpdates 4.4699 ms/op 5.5356 ms/op 0.81
mainnet_e81889 - altair processRewardsAndPenalties 51.324 ms/op 34.793 ms/op 1.48
mainnet_e81889 - altair processRegistryUpdates 2.6410 us/op 1.8780 us/op 1.41
mainnet_e81889 - altair processSlashings 779.00 ns/op 417.00 ns/op 1.87
mainnet_e81889 - altair processEth1DataReset 609.00 ns/op 322.00 ns/op 1.89
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.7743 ms/op 1.3700 ms/op 2.03
mainnet_e81889 - altair processSlashingsReset 3.2350 us/op 4.2280 us/op 0.77
mainnet_e81889 - altair processRandaoMixesReset 4.0470 us/op 4.2280 us/op 0.96
mainnet_e81889 - altair processHistoricalRootsUpdate 661.00 ns/op 517.00 ns/op 1.28
mainnet_e81889 - altair processParticipationFlagUpdates 4.6050 us/op 1.9420 us/op 2.37
mainnet_e81889 - altair processSyncCommitteeUpdates 820.00 ns/op 401.00 ns/op 2.04
mainnet_e81889 - altair afterProcessEpoch 53.736 ms/op 50.178 ms/op 1.07
capella processEpoch - mainnet_e217614 1.3854 s/op 1.0220 s/op 1.36
mainnet_e217614 - capella beforeProcessEpoch 89.230 ms/op 64.682 ms/op 1.38
mainnet_e217614 - capella processJustificationAndFinalization 15.010 us/op 15.222 us/op 0.99
mainnet_e217614 - capella processInactivityUpdates 24.085 ms/op 15.879 ms/op 1.52
mainnet_e217614 - capella processRewardsAndPenalties 247.17 ms/op 216.90 ms/op 1.14
mainnet_e217614 - capella processRegistryUpdates 12.118 us/op 11.935 us/op 1.02
mainnet_e217614 - capella processSlashings 797.00 ns/op 348.00 ns/op 2.29
mainnet_e217614 - capella processEth1DataReset 849.00 ns/op 308.00 ns/op 2.76
mainnet_e217614 - capella processEffectiveBalanceUpdates 16.690 ms/op 12.134 ms/op 1.38
mainnet_e217614 - capella processSlashingsReset 4.1730 us/op 2.9550 us/op 1.41
mainnet_e217614 - capella processRandaoMixesReset 3.9280 us/op 4.0780 us/op 0.96
mainnet_e217614 - capella processHistoricalRootsUpdate 649.00 ns/op 498.00 ns/op 1.30
mainnet_e217614 - capella processParticipationFlagUpdates 1.7410 us/op 1.8180 us/op 0.96
mainnet_e217614 - capella afterProcessEpoch 106.25 ms/op 120.14 ms/op 0.88
phase0 processEpoch - mainnet_e58758 323.48 ms/op 339.14 ms/op 0.95
mainnet_e58758 - phase0 beforeProcessEpoch 78.822 ms/op 72.867 ms/op 1.08
mainnet_e58758 - phase0 processJustificationAndFinalization 13.380 us/op 15.301 us/op 0.87
mainnet_e58758 - phase0 processRewardsAndPenalties 34.884 ms/op 29.402 ms/op 1.19
mainnet_e58758 - phase0 processRegistryUpdates 6.8050 us/op 7.7050 us/op 0.88
mainnet_e58758 - phase0 processSlashings 728.00 ns/op 347.00 ns/op 2.10
mainnet_e58758 - phase0 processEth1DataReset 407.00 ns/op 366.00 ns/op 1.11
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0219 ms/op 1.1944 ms/op 0.86
mainnet_e58758 - phase0 processSlashingsReset 2.9600 us/op 3.4230 us/op 0.86
mainnet_e58758 - phase0 processRandaoMixesReset 2.9670 us/op 3.7040 us/op 0.80
mainnet_e58758 - phase0 processHistoricalRootsUpdate 708.00 ns/op 297.00 ns/op 2.38
mainnet_e58758 - phase0 processParticipationRecordUpdates 2.7740 us/op 2.9940 us/op 0.93
mainnet_e58758 - phase0 afterProcessEpoch 36.346 ms/op 41.917 ms/op 0.87
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.8293 ms/op 1.3928 ms/op 1.31
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.1006 ms/op 2.3783 ms/op 0.88
altair processInactivityUpdates - 250000 normalcase 17.115 ms/op 14.981 ms/op 1.14
altair processInactivityUpdates - 250000 worstcase 17.194 ms/op 15.010 ms/op 1.15
phase0 processRegistryUpdates - 250000 normalcase 2.9450 us/op 5.8030 us/op 0.51
phase0 processRegistryUpdates - 250000 badcase_full_deposits 306.27 us/op 222.01 us/op 1.38
phase0 processRegistryUpdates - 250000 worstcase 0.5 109.08 ms/op 104.72 ms/op 1.04
altair processRewardsAndPenalties - 250000 normalcase 44.328 ms/op 30.106 ms/op 1.47
altair processRewardsAndPenalties - 250000 worstcase 47.103 ms/op 36.437 ms/op 1.29
phase0 getAttestationDeltas - 250000 normalcase 6.1271 ms/op 6.9900 ms/op 0.88
phase0 getAttestationDeltas - 250000 worstcase 6.1660 ms/op 6.9305 ms/op 0.89
phase0 processSlashings - 250000 worstcase 61.161 us/op 76.705 us/op 0.80
altair processSyncCommitteeUpdates - 250000 102.75 ms/op 117.36 ms/op 0.88
BeaconState.hashTreeRoot - No change 442.00 ns/op 258.00 ns/op 1.71
BeaconState.hashTreeRoot - 1 full validator 107.00 us/op 122.00 us/op 0.88
BeaconState.hashTreeRoot - 32 full validator 1.2114 ms/op 1.4034 ms/op 0.86
BeaconState.hashTreeRoot - 512 full validator 10.807 ms/op 9.0865 ms/op 1.19
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 97.434 us/op 123.58 us/op 0.79
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.3194 ms/op 1.5124 ms/op 0.87
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 25.569 ms/op 22.417 ms/op 1.14
BeaconState.hashTreeRoot - 1 balances 128.27 us/op 88.375 us/op 1.45
BeaconState.hashTreeRoot - 32 balances 1.3264 ms/op 837.26 us/op 1.58
BeaconState.hashTreeRoot - 512 balances 6.6738 ms/op 7.6953 ms/op 0.87
BeaconState.hashTreeRoot - 250000 balances 174.39 ms/op 167.88 ms/op 1.04
aggregationBits - 2048 els - zipIndexesInBitList 23.801 us/op 46.494 us/op 0.51
byteArrayEquals 32 47.426 ns/op 54.998 ns/op 0.86
Buffer.compare 32 15.173 ns/op 16.872 ns/op 0.90
byteArrayEquals 1024 1.2477 us/op 1.5492 us/op 0.81
Buffer.compare 1024 24.765 ns/op 24.852 ns/op 1.00
byteArrayEquals 16384 19.787 us/op 24.642 us/op 0.80
Buffer.compare 16384 201.24 ns/op 200.25 ns/op 1.00
byteArrayEquals 123687377 149.46 ms/op 186.36 ms/op 0.80
Buffer.compare 123687377 6.1243 ms/op 6.0616 ms/op 1.01
byteArrayEquals 32 - diff last byte 46.228 ns/op 51.453 ns/op 0.90
Buffer.compare 32 - diff last byte 15.402 ns/op 17.051 ns/op 0.90
byteArrayEquals 1024 - diff last byte 1.2254 us/op 1.5802 us/op 0.78
Buffer.compare 1024 - diff last byte 24.623 ns/op 24.890 ns/op 0.99
byteArrayEquals 16384 - diff last byte 19.396 us/op 25.174 us/op 0.77
Buffer.compare 16384 - diff last byte 189.70 ns/op 201.68 ns/op 0.94
byteArrayEquals 123687377 - diff last byte 152.43 ms/op 187.06 ms/op 0.81
Buffer.compare 123687377 - diff last byte 6.7129 ms/op 6.4409 ms/op 1.04
byteArrayEquals 32 - random bytes 4.8720 ns/op 5.0780 ns/op 0.96
Buffer.compare 32 - random bytes 15.148 ns/op 16.796 ns/op 0.90
byteArrayEquals 1024 - random bytes 4.7360 ns/op 5.0420 ns/op 0.94
Buffer.compare 1024 - random bytes 15.153 ns/op 16.782 ns/op 0.90
byteArrayEquals 16384 - random bytes 4.8770 ns/op 5.0460 ns/op 0.97
Buffer.compare 16384 - random bytes 15.074 ns/op 16.930 ns/op 0.89
byteArrayEquals 123687377 - random bytes 7.5800 ns/op 6.3900 ns/op 1.19
Buffer.compare 123687377 - random bytes 17.960 ns/op 18.170 ns/op 0.99
regular array get 100000 times 29.792 us/op 33.013 us/op 0.90
wrappedArray get 100000 times 29.967 us/op 32.552 us/op 0.92
arrayWithProxy get 100000 times 9.6652 ms/op 13.081 ms/op 0.74
ssz.Root.equals 42.671 ns/op 45.150 ns/op 0.95
byteArrayEquals 41.067 ns/op 44.493 ns/op 0.92
Buffer.compare 8.9050 ns/op 10.150 ns/op 0.88
processSlot - 1 slots 11.612 us/op 13.188 us/op 0.88
processSlot - 32 slots 3.5177 ms/op 2.9112 ms/op 1.21
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 39.022 ms/op 36.615 ms/op 1.07
getCommitteeAssignments - req 1 vs - 250000 vc 1.8240 ms/op 2.0394 ms/op 0.89
getCommitteeAssignments - req 100 vs - 250000 vc 3.5863 ms/op 3.9935 ms/op 0.90
getCommitteeAssignments - req 1000 vs - 250000 vc 3.8119 ms/op 4.2468 ms/op 0.90
findModifiedValidators - 10000 modified validators 241.77 ms/op 240.57 ms/op 1.00
findModifiedValidators - 1000 modified validators 163.92 ms/op 165.93 ms/op 0.99
findModifiedValidators - 100 modified validators 149.94 ms/op 146.62 ms/op 1.02
findModifiedValidators - 10 modified validators 155.30 ms/op 141.86 ms/op 1.09
findModifiedValidators - 1 modified validators 151.28 ms/op 137.65 ms/op 1.10
findModifiedValidators - no difference 186.42 ms/op 145.45 ms/op 1.28
compare ViewDUs 3.3919 s/op 2.8751 s/op 1.18
compare each validator Uint8Array 1.4478 s/op 1.6409 s/op 0.88
compare ViewDU to Uint8Array 1.0096 s/op 937.43 ms/op 1.08
migrate state 1000000 validators, 24 modified, 0 new 662.32 ms/op 729.44 ms/op 0.91
migrate state 1000000 validators, 1700 modified, 1000 new 946.95 ms/op 967.07 ms/op 0.98
migrate state 1000000 validators, 3400 modified, 2000 new 1.4715 s/op 1.1626 s/op 1.27
migrate state 1500000 validators, 24 modified, 0 new 866.05 ms/op 713.82 ms/op 1.21
migrate state 1500000 validators, 1700 modified, 1000 new 1.1223 s/op 946.18 ms/op 1.19
migrate state 1500000 validators, 3400 modified, 2000 new 1.4161 s/op 1.1576 s/op 1.22
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.0700 ns/op 4.1800 ns/op 1.69
state getBlockRootAtSlot - 250000 vs - 7PWei 534.62 ns/op 693.33 ns/op 0.77
computeProposers - vc 250000 6.2736 ms/op 6.6118 ms/op 0.95
computeEpochShuffling - vc 250000 37.310 ms/op 40.555 ms/op 0.92
getNextSyncCommittee - vc 250000 135.53 ms/op 116.23 ms/op 1.17
computeSigningRoot for AttestationData 23.136 us/op 22.905 us/op 1.01
hash AttestationData serialized data then Buffer.toString(base64) 1.3313 us/op 1.5285 us/op 0.87
toHexString serialized data 872.34 ns/op 837.86 ns/op 1.04
Buffer.toString(base64) 189.08 ns/op 181.12 ns/op 1.04
nodejs block root to RootHex using toHex 162.43 ns/op 175.36 ns/op 0.93
nodejs block root to RootHex using toRootHex 97.615 ns/op 88.733 ns/op 1.10
browser block root to RootHex using the deprecated toHexString 260.81 ns/op 217.99 ns/op 1.20
browser block root to RootHex using toHex 232.56 ns/op 174.69 ns/op 1.33
browser block root to RootHex using toRootHex 157.20 ns/op 156.66 ns/op 1.00

by benchmarkbot/action

@codecov Codecov
Copy link

codecov bot commented Nov 19, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 49.14%. Comparing base (6039f58) to head (090417f).
Report is 3 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7230      +/-   ##
============================================
+ Coverage     49.08%   49.14%   +0.06%     
============================================
  Files           600      600              
  Lines         40241    40121     -120     
  Branches       2110     2093      -17     
============================================
- Hits          19753    19719      -34     
+ Misses        20450    20364      -86     
  Partials         38       38
---- 🚨 Try these New Features:

@ensi321 ensi321 marked this pull request as ready for review November 20, 2024 07:52
@ensi321 ensi321 requested a review from a team as a code owner November 20, 2024 07:52
twoeths
twoeths approved these changes Nov 22, 2024
View reviewed changes
Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ensi321 ensi321 merged commit b78cb92 into unstable Nov 25, 2024
20 checks passed
@ensi321 ensi321 deleted the nc/remove-immutable branch November 25, 2024 10:20
wemeetagain added a commit that referenced this pull request Dec 20, 2024
@wemeetagain @nflaig
@nazarhussain @g11tech @matthewkeil @ensi321 @twoeths @philknows @dependabot @acolytec3 @varunguleriaCodes @ClockworkYuzu
chore: release v1.24.0 (#7325)
11f35a8 
* feat: add keymanager endpoint to retrieve proposer config (#7210)

* feat: add keymanager endpoint to retrieve proposer config

* Do not return empty builder config

* Check all builder proposer config values

* Fix settings builder config if undefined

* Fix builder config parsing

* Use ssz type to handle json serialization

Default parsing can't handle BigInt

* Revert "Use ssz type to handle json serialization"

This reverts commit 01fcea7.

* Fix boost factor json serialization

* Remove unused import

* Update test data

* Update proposer config test

* feat: add mekong network option (#7212)

* chore: fix import order with biome syntax (#7211)

Fix import order

* fix: consistently validate pubkey and throw 404 if not found (#7214)

* Throw error if pubkey is unknown when getting graffiti

* Consistently validate pubkey and throw 404 if not found

* fix: only return local keys from /eth/v1/keystores (#7215)

* fix: only return local keys from /eth/v1/keystores

* Fix fetching remote keys in node assertion

* feat: add and use getBlobsV1 to expedite gossip import (#7134)

* hookup the getblobs api to get bob and proof data from el

remove unused

fix import

metrics overhault, test, debugging testing, some feeback

fix

add nethermind bug dicussion link

fix

resolve conflicts

* deblobs timeout

* fix metric

* chore: revert async aggregate with randomness (#7218)

Revert "feat: asyncAggregateWithRandomness (#7204)"

This reverts commit e31d535.

* fix: update config for relaunched mekong network (#7220)

* fix: light client generating `LightClientUpdate` with wrong length of branches (#7187)

* initial commit

* Rewrite SyncCommitteeWitnessRepository

* Fix finality branch

* Update unit test

* fix e2e

* Review PR

---------

Co-authored-by: Nico Flaig <nflaig@protonmail.com>

* fix: archive finalized state when shutting down beacon node (#7221)

* Fix typo

* feat: remove unfinalized pubkey cache (#7230)

* Remove unfinalized pubkey cache

* lint

* Fix unit test

* chore: skip web3_provider unit tests (#7252)

* fix: prune checkpoint states at syncing time (#7241)

* fix: prune checkpoint states at syncing time

* fix: lint

* fix: check-types in test

* fix: sync cached isCompoundingValidatorArr at epoch transition (#7247)

* fix: handle outOfRangeData when range sync Deneb (#7249)

* fix: handle outOfRangeData for beaconBlocksMaybeBlobsByRange()

* fix: lint

* fix: archiveBlocks - handle deneb outOfRangeData block

* fix: sync cached balance when adding new validator to registry (#7255)

* fix: sync cached balance when adding new validator to registry

* chore: add more comments

* fix: remove persisted checkpoint states from the previous run at startup

* fix: do not throw error when trying to prune missing directory (#7257)

* docs: update documentation Oct 2024 (#7178)

* docs update oct 2024 init

* Reconfig quickstart nav and minor fixes

* fix lint

* spelling fixes

* minor fixes and add to wordlist

* prettier fix

* add to wordlist

* sort wordlist

* modify dominance to include lighthouse

* fix typescript casing and add recommendation

* add selection and boost_factor with keymanager notice

* update wordlist

* remove builder enabled and add keymanager api

* spelling

---------

Co-authored-by: Nico Flaig <nflaig@protonmail.com>

* chore(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /docs (#7268)

Bumps [cross-spawn](https://github.com/moxystudio/node-cross-spawn) from 7.0.3 to 7.0.6.
- [Changelog](https://github.com/moxystudio/node-cross-spawn/blob/master/CHANGELOG.md)
- [Commits](moxystudio/node-cross-spawn@v7.0.3...v7.0.6)

---
updated-dependencies:
- dependency-name: cross-spawn
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: add error log to notifier if execution client auth failed (#7239)

* feat: add error log to notifier if execution client auth failed

* Update packages/beacon-node/src/node/notifier.ts

---------

Co-authored-by: NC <17676176+ensi321@users.noreply.github.com>

* docs: display rcConfig flag on CLI reference page (#7270)

* docs: display rcConfig flag on CLI reference page

* Update word list

* chore: remove prettier as default formatter for all file types (#7275)

* chore: unhide flags relevant for devnets / testing (#7271)

* feat: debug too many shuffling promises (#7251)

* feat: add asyncShufflingCalculation to StateTransitionOpts

* feat: add asyncShufflingCalculation to all regen / processSlots consumers

* fix: default to false for async shuffling and remove unnecessary props

* fix: remove unnecessary flags from stateTransition

* feat: implement conditional build of shuffling for prepareNextSlot

* fix: spec test bug where shufflingCache is present from BeaconChain constructor

* feat: sync build next shuffling if not queued async

* fix: use getSync to pull next shuffling correctly

* docs: add comment to prepareNextSlot

* refactor: rename StateCloneOpts to StateRegenerationOpts

* feat: pass asyncShufflingCalculation through to afterProcessEpoch and refactor conditional to run purely sync

* docs: add issue number to comment

* chore: lint

* chore: unpin nodejs version from 22.4 (#6982)

* Revert "chore: pin nodejs version to 22.4 (#6964)"

This reverts commit f20484b.

* Don't revert formatting changes

---------

Co-authored-by: Nico Flaig <nflaig@protonmail.com>
Co-authored-by: Cayman <caymannava@gmail.com>

* chore: update bootnodes file url for holesky and sepolia (#7276)

* feat: add `debug_getHistoricalSummaries` endpoint (#7245)

* feat: add new getHistoricalSummaries endpoint to debug namespace

* Add JSON response

* Restructure to use stateId and add proof to response

* add test scaffolding

* Address feedback

* Move getHistoricalSummaries to lodestar namespace

* add lodestar namespace unit test

* update route name to lodestar namespace

* cast state object as Capella state

* Lint

* json properties need to be lower case

* Make it v1 since it's now part of lodestar namespace

* Group with other /lodestar endpoints

* Simplify beacon node impl

* Rename return type

* Update test description

* Fix variable name

---------

Co-authored-by: Nico Flaig <nflaig@protonmail.com>

* chore: log sync committee signature errors as `error` (#7283)

* fix: update engine_getClientVersionV1 commit encoding (#7282)

* fix: check pubkey or validator index known to a state (#7284)

* fix: check pubkey or validator index known to a state

* chore: add more comments

* feat: lodestar script setup (#7254)

* feat: lodestar_setup

* feat: script_updates + docs

* feat: script_addition_in_docs + command_update

* Remove duplicate script from docs folder

* Minor script updates

* Update script to prepare docs and ignore copied file

* Update installation page

* Wording

---------

Co-authored-by: Nico Flaig <nflaig@protonmail.com>

* feat: add terminal-sized Electra giraffe banner (#7286)

* Create giraffeBanners.ts

* Wire in banner

* Fix file name

* lint

* Address @nflaig's comment

---------

Co-authored-by: NC <17676176+ensi321@users.noreply.github.com>

* chore: pin nodejs version to 22.4 (#7291)

Revert "chore: unpin nodejs version from 22.4 (#6982)"

This reverts commit 69ae688.

* feat: expose `DOMAIN_APPLICATION_MASK` in config/spec api (#7296)

* feat: expose DOMAIN_APPLICATION_MASK in config/spec api

* Lint

* feat: make `MAX_REQUEST_BLOB_SIDECARS` and `MAX_BLOBS_PER_BLOCK` configurable (#7294)

* Init commit

* Fix check-types

* Add comment on how MAX_REQUEST_BLOB_SIDECARS is calculated

* Ensure proper config object is passed

* Address comment

---------

Co-authored-by: Nico Flaig <nflaig@protonmail.com>

* feat: use `BLOB_SIDECAR_SUBNET_COUNT` to configure blob subnets (#7297)

feat: use BLOB_SIDECAR_SUBNET_COUNT to configure blob subnets

* chore: log sync aggregate participants when producing beacon block body (#7300)

* chore: log sync aggregate participants when producing beacon block body

* Use isForkLightClient instead of ForkSeq

* Fix produce block unit tests

* chore: print graffiti when producing beacon block body (#7303)

* fix: warn if engine / builder failed to produce block within cutoff time (#7305)

* feat: add kzg commitment length check when validating gossip blocks (#7302)

* feat: add blob sidecar index check (#7313)

Validate blobSidecar index

* fix: fix blob sidecar index check (#7315)

Fix index check

* chore: fix format of printed graffiti from hex to utf-8 (#7306)

* chore: fix format of printed graffiti from hex to utf-8

* Use Buffer.from no copy with offset

* docs: batch commit typos and update contributor readme (#7312)

* batch commit typos and update contributor readme

* update donation text

Co-authored-by: Nico Flaig <nflaig@protonmail.com>

* correct spelling

Co-authored-by: Nico Flaig <nflaig@protonmail.com>

---------

Co-authored-by: Nico Flaig <nflaig@protonmail.com>

* chore: remove trailing null bytes from printed graffiti (#7320)

* chore: remove trailing null bytes from printed graffiti

* Use replaceAll instead of regex

* chore: unpin nodejs version from 22.4 (#7324)

Revert "chore: pin nodejs version to 22.4 (#7291)"

This reverts commit 99794d3.

* chore: bump package versions to 1.24.0

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
Co-authored-by: Nazar Hussain <nazarhussain@gmail.com>
Co-authored-by: g11tech <develop@g11tech.io>
Co-authored-by: Matthew Keil <me@matthewkeil.com>
Co-authored-by: NC <17676176+ensi321@users.noreply.github.com>
Co-authored-by: twoeths <tuyen@chainsafe.io>
Co-authored-by: Phil Ngo <58080811+philknows@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: acolytec3 <17355484+acolytec3@users.noreply.github.com>
Co-authored-by: Varun Guleria <152203177+varunguleriaCodes@users.noreply.github.com>
Co-authored-by: ClockworkYuzu <yvetteheh@gmail.com>
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.24.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ensi321 @wemeetagain @twoeths