fix: prune oldest slots first in aggregator tracker

[nflaig]

Motivation

Due to the fact the we are pruning based on FIFO it might happen that we prune a slot which is in the future and should not yet be pruned.

This is completely random as is depends on the order of which the validator client submits the subscriptions, current epoch vs. next epoch first, and the order of the items themselves. For example at the start of epoch 0, the client submits epoch 1 first, then epoch 0 and if there is an aggregator for each slot (likely with a lot of keys) then the cache is already full (64 items). Then at the start of epoch 1, some clients only submit for epoch 2 (next epoch) but this means we start pruning items of epoch 1, but that's the current epoch which should not yet be pruned.

While Lodestar calls prepareBeaconCommitteeSubnet always for current and next epoch, other clients like Nimbus, Teku and Vouch do not, which might lead to the issue that Lodestar does not prepare an aggregated attestation (#6631, #6419), as at the time when the attestations are submitted via submitPoolAttestations the aggregator check will return false.

Description

Prune oldest slots first in aggregator tracker to ensure we only prune slots that are in the past.

Alternative solution

We could also increase the max cache slots to 3 epochs

Closes #6631
Closes #6419

[codecov]

Codecov Report

Merging #6668 (3022480) into unstable (669239b) will decrease coverage by 0.02%.
Report is 1 commits behind head on unstable.
The diff coverage is 25.92%.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #6668      +/-   ##
============================================
- Coverage     61.81%   61.80%   -0.02%     
============================================
  Files           556      556              
  Lines         59050    59073      +23     
  Branches       1898     1898              
============================================
+ Hits          36502    36508       +6     
- Misses        22505    22522      +17     
  Partials         43       43              

[github-actions]

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite	Current: dd1ed1d	Previous: ae984f0	Ratio
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15	7.6100 us/op	26.507 us/op	0.29

Full benchmark results

Benchmark suite	Current: dd1ed1d	Previous: ae984f0	Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	574.29 us/op	679.83 us/op	0.84
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	57.678 us/op	107.00 us/op	0.54
BLS verify - blst-native	1.1329 ms/op	1.3826 ms/op	0.82
BLS verifyMultipleSignatures 3 - blst-native	2.4200 ms/op	2.8539 ms/op	0.85
BLS verifyMultipleSignatures 8 - blst-native	5.4899 ms/op	6.2462 ms/op	0.88
BLS verifyMultipleSignatures 32 - blst-native	19.698 ms/op	22.893 ms/op	0.86
BLS verifyMultipleSignatures 64 - blst-native	38.879 ms/op	46.461 ms/op	0.84
BLS verifyMultipleSignatures 128 - blst-native	78.840 ms/op	94.462 ms/op	0.83
BLS deserializing 10000 signatures	813.98 ms/op	968.93 ms/op	0.84
BLS deserializing 100000 signatures	8.6397 s/op	8.9732 s/op	0.96
BLS verifyMultipleSignatures - same message - 3 - blst-native	1.2093 ms/op	1.3743 ms/op	0.88
BLS verifyMultipleSignatures - same message - 8 - blst-native	1.5152 ms/op	1.5601 ms/op	0.97
BLS verifyMultipleSignatures - same message - 32 - blst-native	2.2158 ms/op	2.3996 ms/op	0.92
BLS verifyMultipleSignatures - same message - 64 - blst-native	4.3056 ms/op	3.4012 ms/op	1.27
BLS verifyMultipleSignatures - same message - 128 - blst-native	5.1736 ms/op	5.7714 ms/op	0.90
BLS aggregatePubkeys 32 - blst-native	24.209 us/op	26.284 us/op	0.92
BLS aggregatePubkeys 128 - blst-native	90.954 us/op	100.33 us/op	0.91
notSeenSlots=1 numMissedVotes=1 numBadVotes=10	59.216 ms/op	64.205 ms/op	0.92
notSeenSlots=1 numMissedVotes=0 numBadVotes=4	49.598 ms/op	59.169 ms/op	0.84
notSeenSlots=2 numMissedVotes=1 numBadVotes=10	26.655 ms/op	53.250 ms/op	0.50
getSlashingsAndExits - default max	137.16 us/op	207.45 us/op	0.66
getSlashingsAndExits - 2k	292.55 us/op	597.64 us/op	0.49
proposeBlockBody type=full, size=empty	3.9308 ms/op	8.0017 ms/op	0.49
isKnown best case - 1 super set check	322.00 ns/op	740.00 ns/op	0.44
isKnown normal case - 2 super set checks	347.00 ns/op	739.00 ns/op	0.47
isKnown worse case - 16 super set checks	349.00 ns/op	649.00 ns/op	0.54
InMemoryCheckpointStateCache - add get delete	5.0430 us/op	8.7280 us/op	0.58
validate api signedAggregateAndProof - struct	2.3572 ms/op	3.0190 ms/op	0.78
validate gossip signedAggregateAndProof - struct	2.3540 ms/op	2.9503 ms/op	0.80
validate gossip attestation - vc 640000	1.1391 ms/op	1.4169 ms/op	0.80
batch validate gossip attestation - vc 640000 - chunk 32	142.33 us/op	169.53 us/op	0.84
batch validate gossip attestation - vc 640000 - chunk 64	123.66 us/op	148.73 us/op	0.83
batch validate gossip attestation - vc 640000 - chunk 128	119.38 us/op	139.53 us/op	0.86
batch validate gossip attestation - vc 640000 - chunk 256	118.41 us/op	127.81 us/op	0.93
pickEth1Vote - no votes	889.56 us/op	1.1556 ms/op	0.77
pickEth1Vote - max votes	12.416 ms/op	9.1164 ms/op	1.36
pickEth1Vote - Eth1Data hashTreeRoot value x2048	19.816 ms/op	15.981 ms/op	1.24
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	28.196 ms/op	24.409 ms/op	1.16
pickEth1Vote - Eth1Data fastSerialize value x2048	445.07 us/op	597.39 us/op	0.75
pickEth1Vote - Eth1Data fastSerialize tree x2048	4.5239 ms/op	4.2496 ms/op	1.06
bytes32 toHexString	469.00 ns/op	481.00 ns/op	0.98
bytes32 Buffer.toString(hex)	322.00 ns/op	284.00 ns/op	1.13
bytes32 Buffer.toString(hex) from Uint8Array	435.00 ns/op	411.00 ns/op	1.06
bytes32 Buffer.toString(hex) + 0x	328.00 ns/op	288.00 ns/op	1.14
Object access 1 prop	0.21300 ns/op	0.15900 ns/op	1.34
Map access 1 prop	0.22200 ns/op	0.14800 ns/op	1.50
Object get x1000	5.6530 ns/op	7.2450 ns/op	0.78
Map get x1000	0.74900 ns/op	0.73400 ns/op	1.02
Object set x1000	27.964 ns/op	48.062 ns/op	0.58
Map set x1000	19.463 ns/op	38.120 ns/op	0.51
Return object 10000 times	0.23200 ns/op	0.23690 ns/op	0.98
Throw Error 10000 times	2.7726 us/op	3.8225 us/op	0.73
fastMsgIdFn sha256 / 200 bytes	2.1480 us/op	3.2360 us/op	0.66
fastMsgIdFn h32 xxhash / 200 bytes	354.00 ns/op	277.00 ns/op	1.28
fastMsgIdFn h64 xxhash / 200 bytes	370.00 ns/op	345.00 ns/op	1.07
fastMsgIdFn sha256 / 1000 bytes	6.4880 us/op	11.231 us/op	0.58
fastMsgIdFn h32 xxhash / 1000 bytes	482.00 ns/op	395.00 ns/op	1.22
fastMsgIdFn h64 xxhash / 1000 bytes	453.00 ns/op	413.00 ns/op	1.10
fastMsgIdFn sha256 / 10000 bytes	53.281 us/op	102.94 us/op	0.52
fastMsgIdFn h32 xxhash / 10000 bytes	1.8920 us/op	1.8900 us/op	1.00
fastMsgIdFn h64 xxhash / 10000 bytes	1.3130 us/op	1.2900 us/op	1.02
send data - 1000 256B messages	12.185 ms/op	18.299 ms/op	0.67
send data - 1000 512B messages	14.388 ms/op	25.403 ms/op	0.57
send data - 1000 1024B messages	23.585 ms/op	39.722 ms/op	0.59
send data - 1000 1200B messages	26.803 ms/op	40.113 ms/op	0.67
send data - 1000 2048B messages	38.499 ms/op	50.297 ms/op	0.77
send data - 1000 4096B messages	33.258 ms/op	44.953 ms/op	0.74
send data - 1000 16384B messages	92.520 ms/op	120.57 ms/op	0.77
send data - 1000 65536B messages	455.96 ms/op	460.66 ms/op	0.99
enrSubnets - fastDeserialize 64 bits	997.00 ns/op	1.2750 us/op	0.78
enrSubnets - ssz BitVector 64 bits	460.00 ns/op	432.00 ns/op	1.06
enrSubnets - fastDeserialize 4 bits	208.00 ns/op	175.00 ns/op	1.19
enrSubnets - ssz BitVector 4 bits	472.00 ns/op	437.00 ns/op	1.08
prioritizePeers score -10:0 att 32-0.1 sync 2-0	147.27 us/op	217.84 us/op	0.68
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	177.75 us/op	273.90 us/op	0.65
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	228.58 us/op	342.17 us/op	0.67
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	364.19 us/op	594.39 us/op	0.61
prioritizePeers score 0:0 att 64-1 sync 4-1	419.33 us/op	730.03 us/op	0.57
array of 16000 items push then shift	1.2975 us/op	1.6575 us/op	0.78
LinkedList of 16000 items push then shift	5.9350 ns/op	9.0870 ns/op	0.65
array of 16000 items push then pop	70.436 ns/op	87.564 ns/op	0.80
LinkedList of 16000 items push then pop	5.6490 ns/op	8.8930 ns/op	0.64
array of 24000 items push then shift	1.8914 us/op	2.4132 us/op	0.78
LinkedList of 24000 items push then shift	5.9170 ns/op	9.0720 ns/op	0.65
array of 24000 items push then pop	91.603 ns/op	120.35 ns/op	0.76
LinkedList of 24000 items push then pop	5.8180 ns/op	8.8380 ns/op	0.66
intersect bitArray bitLen 8	5.0280 ns/op	5.9310 ns/op	0.85
intersect array and set length 8	50.984 ns/op	65.118 ns/op	0.78
intersect bitArray bitLen 128	29.510 ns/op	36.181 ns/op	0.82
intersect array and set length 128	697.69 ns/op	904.72 ns/op	0.77
bitArray.getTrueBitIndexes() bitLen 128	1.2410 us/op	1.6080 us/op	0.77
bitArray.getTrueBitIndexes() bitLen 248	1.9530 us/op	2.6890 us/op	0.73
bitArray.getTrueBitIndexes() bitLen 512	4.4580 us/op	5.2060 us/op	0.86
Buffer.concat 32 items	1.0240 us/op	1.1760 us/op	0.87
Uint8Array.set 32 items	2.0030 us/op	1.9210 us/op	1.04
Set add up to 64 items then delete first	1.7382 us/op	4.3733 us/op	0.40
OrderedSet add up to 64 items then delete first	2.7749 us/op	5.5538 us/op	0.50
Set add up to 64 items then delete last	2.2371 us/op	4.6900 us/op	0.48
OrderedSet add up to 64 items then delete last	3.2180 us/op	5.8771 us/op	0.55
Set add up to 64 items then delete middle	2.0789 us/op	4.8264 us/op	0.43
OrderedSet add up to 64 items then delete middle	4.3262 us/op	7.1851 us/op	0.60
Set add up to 128 items then delete first	3.9533 us/op	9.6574 us/op	0.41
OrderedSet add up to 128 items then delete first	5.9562 us/op	13.130 us/op	0.45
Set add up to 128 items then delete last	4.0120 us/op	9.5139 us/op	0.42
OrderedSet add up to 128 items then delete last	6.2848 us/op	11.967 us/op	0.53
Set add up to 128 items then delete middle	3.8518 us/op	9.3372 us/op	0.41
OrderedSet add up to 128 items then delete middle	13.193 us/op	17.978 us/op	0.73
Set add up to 256 items then delete first	8.6348 us/op	19.790 us/op	0.44
OrderedSet add up to 256 items then delete first	12.368 us/op	28.492 us/op	0.43
Set add up to 256 items then delete last	8.1181 us/op	19.309 us/op	0.42
OrderedSet add up to 256 items then delete last	12.131 us/op	24.277 us/op	0.50
Set add up to 256 items then delete middle	7.6769 us/op	19.030 us/op	0.40
OrderedSet add up to 256 items then delete middle	31.037 us/op	47.200 us/op	0.66
transfer serialized Status (84 B)	1.4220 us/op	1.7500 us/op	0.81
copy serialized Status (84 B)	1.1560 us/op	1.3280 us/op	0.87
transfer serialized SignedVoluntaryExit (112 B)	1.5090 us/op	1.9950 us/op	0.76
copy serialized SignedVoluntaryExit (112 B)	1.1910 us/op	1.6570 us/op	0.72
transfer serialized ProposerSlashing (416 B)	1.5070 us/op	3.2420 us/op	0.46
copy serialized ProposerSlashing (416 B)	1.2610 us/op	2.9820 us/op	0.42
transfer serialized Attestation (485 B)	1.5030 us/op	3.4460 us/op	0.44
copy serialized Attestation (485 B)	1.2670 us/op	3.1520 us/op	0.40
transfer serialized AttesterSlashing (33232 B)	1.5410 us/op	3.3010 us/op	0.47
copy serialized AttesterSlashing (33232 B)	3.2100 us/op	6.4470 us/op	0.50
transfer serialized Small SignedBeaconBlock (128000 B)	1.9220 us/op	3.4130 us/op	0.56
copy serialized Small SignedBeaconBlock (128000 B)	8.8460 us/op	16.319 us/op	0.54
transfer serialized Avg SignedBeaconBlock (200000 B)	2.2910 us/op	3.7740 us/op	0.61
copy serialized Avg SignedBeaconBlock (200000 B)	12.279 us/op	23.907 us/op	0.51
transfer serialized BlobsSidecar (524380 B)	2.7310 us/op	3.8790 us/op	0.70
copy serialized BlobsSidecar (524380 B)	72.456 us/op	189.82 us/op	0.38
transfer serialized Big SignedBeaconBlock (1000000 B)	2.5960 us/op	4.0430 us/op	0.64
copy serialized Big SignedBeaconBlock (1000000 B)	147.21 us/op	159.69 us/op	0.92
pass gossip attestations to forkchoice per slot	2.9151 ms/op	4.6671 ms/op	0.62
forkChoice updateHead vc 100000 bc 64 eq 0	417.50 us/op	700.41 us/op	0.60
forkChoice updateHead vc 600000 bc 64 eq 0	2.8988 ms/op	5.0507 ms/op	0.57
forkChoice updateHead vc 1000000 bc 64 eq 0	4.9862 ms/op	7.3799 ms/op	0.68
forkChoice updateHead vc 600000 bc 320 eq 0	2.6364 ms/op	4.3554 ms/op	0.61
forkChoice updateHead vc 600000 bc 1200 eq 0	2.7044 ms/op	4.4869 ms/op	0.60
forkChoice updateHead vc 600000 bc 7200 eq 0	3.5324 ms/op	5.9469 ms/op	0.59
forkChoice updateHead vc 600000 bc 64 eq 1000	9.9704 ms/op	11.548 ms/op	0.86
forkChoice updateHead vc 600000 bc 64 eq 10000	9.8454 ms/op	12.239 ms/op	0.80
forkChoice updateHead vc 600000 bc 64 eq 300000	11.990 ms/op	16.697 ms/op	0.72
computeDeltas 500000 validators 300 proto nodes	3.2046 ms/op	7.0034 ms/op	0.46
computeDeltas 500000 validators 1200 proto nodes	3.1388 ms/op	6.9346 ms/op	0.45
computeDeltas 500000 validators 7200 proto nodes	3.1020 ms/op	6.5988 ms/op	0.47
computeDeltas 750000 validators 300 proto nodes	4.6584 ms/op	10.166 ms/op	0.46
computeDeltas 750000 validators 1200 proto nodes	4.9929 ms/op	10.070 ms/op	0.50
computeDeltas 750000 validators 7200 proto nodes	4.8343 ms/op	10.077 ms/op	0.48
computeDeltas 1400000 validators 300 proto nodes	9.2437 ms/op	19.209 ms/op	0.48
computeDeltas 1400000 validators 1200 proto nodes	9.6813 ms/op	19.126 ms/op	0.51
computeDeltas 1400000 validators 7200 proto nodes	9.1849 ms/op	19.163 ms/op	0.48
computeDeltas 2100000 validators 300 proto nodes	13.836 ms/op	28.235 ms/op	0.49
computeDeltas 2100000 validators 1200 proto nodes	13.692 ms/op	27.520 ms/op	0.50
computeDeltas 2100000 validators 7200 proto nodes	13.813 ms/op	28.962 ms/op	0.48
altair processAttestation - 250000 vs - 7PWei normalcase	1.5315 ms/op	2.7419 ms/op	0.56
altair processAttestation - 250000 vs - 7PWei worstcase	2.3041 ms/op	4.1149 ms/op	0.56
altair processAttestation - setStatus - 1/6 committees join	100.62 us/op	206.03 us/op	0.49
altair processAttestation - setStatus - 1/3 committees join	197.58 us/op	375.34 us/op	0.53
altair processAttestation - setStatus - 1/2 committees join	281.69 us/op	522.27 us/op	0.54
altair processAttestation - setStatus - 2/3 committees join	370.86 us/op	672.60 us/op	0.55
altair processAttestation - setStatus - 4/5 committees join	475.17 us/op	885.73 us/op	0.54
altair processAttestation - setStatus - 100% committees join	554.80 us/op	1.0630 ms/op	0.52
altair processBlock - 250000 vs - 7PWei normalcase	7.9560 ms/op	9.4320 ms/op	0.84
altair processBlock - 250000 vs - 7PWei normalcase hashState	28.270 ms/op	40.689 ms/op	0.69
altair processBlock - 250000 vs - 7PWei worstcase	33.429 ms/op	55.116 ms/op	0.61
altair processBlock - 250000 vs - 7PWei worstcase hashState	82.422 ms/op	126.97 ms/op	0.65
phase0 processBlock - 250000 vs - 7PWei normalcase	2.3853 ms/op	4.1865 ms/op	0.57
phase0 processBlock - 250000 vs - 7PWei worstcase	26.070 ms/op	38.385 ms/op	0.68
altair processEth1Data - 250000 vs - 7PWei normalcase	390.38 us/op	803.69 us/op	0.49
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15	7.6100 us/op	26.507 us/op	0.29
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219	50.490 us/op	88.056 us/op	0.57
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42	20.602 us/op	32.963 us/op	0.63
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18	12.144 us/op	24.755 us/op	0.49
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020	168.21 us/op	240.26 us/op	0.70
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777	1.1013 ms/op	1.8372 ms/op	0.60
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384	971.55 us/op	2.7300 ms/op	0.36
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384	1.5512 ms/op	2.6109 ms/op	0.59
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384	2.1720 ms/op	5.2719 ms/op	0.41
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384	1.4879 ms/op	3.5065 ms/op	0.42
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384	3.7964 ms/op	7.4469 ms/op	0.51
Tree 40 250000 create	254.60 ms/op	782.38 ms/op	0.33
Tree 40 250000 get(125000)	112.79 ns/op	260.80 ns/op	0.43
Tree 40 250000 set(125000)	827.93 ns/op	3.2939 us/op	0.25
Tree 40 250000 toArray()	16.119 ms/op	33.072 ms/op	0.49
Tree 40 250000 iterate all - toArray() + loop	19.374 ms/op	32.464 ms/op	0.60
Tree 40 250000 iterate all - get(i)	47.671 ms/op	87.995 ms/op	0.54
MutableVector 250000 create	10.346 ms/op	19.581 ms/op	0.53
MutableVector 250000 get(125000)	6.1960 ns/op	7.7060 ns/op	0.80
MutableVector 250000 set(125000)	255.72 ns/op	690.85 ns/op	0.37
MutableVector 250000 toArray()	2.5728 ms/op	5.3295 ms/op	0.48
MutableVector 250000 iterate all - toArray() + loop	3.7276 ms/op	5.5741 ms/op	0.67
MutableVector 250000 iterate all - get(i)	1.4452 ms/op	1.6992 ms/op	0.85
Array 250000 create	2.4475 ms/op	4.4676 ms/op	0.55
Array 250000 clone - spread	1.2748 ms/op	1.8013 ms/op	0.71
Array 250000 get(125000)	1.1150 ns/op	1.4220 ns/op	0.78
Array 250000 set(125000)	1.3250 ns/op	6.0350 ns/op	0.22
Array 250000 iterate all - loop	160.23 us/op	192.60 us/op	0.83
effectiveBalanceIncrements clone Uint8Array 300000	45.976 us/op	69.487 us/op	0.66
effectiveBalanceIncrements clone MutableVector 300000	506.00 ns/op	540.00 ns/op	0.94
effectiveBalanceIncrements rw all Uint8Array 300000	189.19 us/op	225.03 us/op	0.84
effectiveBalanceIncrements rw all MutableVector 300000	84.741 ms/op	161.94 ms/op	0.52
phase0 afterProcessEpoch - 250000 vs - 7PWei	78.372 ms/op	138.33 ms/op	0.57
phase0 beforeProcessEpoch - 250000 vs - 7PWei	45.530 ms/op	51.168 ms/op	0.89
altair processEpoch - mainnet_e81889	467.11 ms/op	600.59 ms/op	0.78
mainnet_e81889 - altair beforeProcessEpoch	72.182 ms/op	101.39 ms/op	0.71
mainnet_e81889 - altair processJustificationAndFinalization	12.339 us/op	17.748 us/op	0.70
mainnet_e81889 - altair processInactivityUpdates	5.9704 ms/op	8.5947 ms/op	0.69
mainnet_e81889 - altair processRewardsAndPenalties	71.920 ms/op	66.367 ms/op	1.08
mainnet_e81889 - altair processRegistryUpdates	1.9180 us/op	2.8010 us/op	0.68
mainnet_e81889 - altair processSlashings	573.00 ns/op	502.00 ns/op	1.14
mainnet_e81889 - altair processEth1DataReset	637.00 ns/op	818.00 ns/op	0.78
mainnet_e81889 - altair processEffectiveBalanceUpdates	1.0277 ms/op	1.6723 ms/op	0.61
mainnet_e81889 - altair processSlashingsReset	3.0140 us/op	4.9530 us/op	0.61
mainnet_e81889 - altair processRandaoMixesReset	4.4820 us/op	8.5010 us/op	0.53
mainnet_e81889 - altair processHistoricalRootsUpdate	825.00 ns/op	1.2940 us/op	0.64
mainnet_e81889 - altair processParticipationFlagUpdates	1.3990 us/op	3.7510 us/op	0.37
mainnet_e81889 - altair processSyncCommitteeUpdates	621.00 ns/op	924.00 ns/op	0.67
mainnet_e81889 - altair afterProcessEpoch	80.622 ms/op	131.21 ms/op	0.61
capella processEpoch - mainnet_e217614	1.9435 s/op	3.5070 s/op	0.55
mainnet_e217614 - capella beforeProcessEpoch	500.14 ms/op	750.51 ms/op	0.67
mainnet_e217614 - capella processJustificationAndFinalization	11.402 us/op	35.132 us/op	0.32
mainnet_e217614 - capella processInactivityUpdates	16.921 ms/op	33.453 ms/op	0.51
mainnet_e217614 - capella processRewardsAndPenalties	441.71 ms/op	799.51 ms/op	0.55
mainnet_e217614 - capella processRegistryUpdates	17.956 us/op	46.184 us/op	0.39
mainnet_e217614 - capella processSlashings	636.00 ns/op	1.7450 us/op	0.36
mainnet_e217614 - capella processEth1DataReset	613.00 ns/op	2.1040 us/op	0.29
mainnet_e217614 - capella processEffectiveBalanceUpdates	3.5748 ms/op	7.2041 ms/op	0.50
mainnet_e217614 - capella processSlashingsReset	3.5570 us/op	9.3440 us/op	0.38
mainnet_e217614 - capella processRandaoMixesReset	4.1180 us/op	15.891 us/op	0.26
mainnet_e217614 - capella processHistoricalRootsUpdate	740.00 ns/op	1.8200 us/op	0.41
mainnet_e217614 - capella processParticipationFlagUpdates	1.2860 us/op	5.8550 us/op	0.22
mainnet_e217614 - capella afterProcessEpoch	216.55 ms/op	427.01 ms/op	0.51
phase0 processEpoch - mainnet_e58758	440.25 ms/op	714.70 ms/op	0.62
mainnet_e58758 - phase0 beforeProcessEpoch	138.50 ms/op	191.59 ms/op	0.72
mainnet_e58758 - phase0 processJustificationAndFinalization	14.158 us/op	33.445 us/op	0.42
mainnet_e58758 - phase0 processRewardsAndPenalties	60.982 ms/op	62.011 ms/op	0.98
mainnet_e58758 - phase0 processRegistryUpdates	9.1270 us/op	24.904 us/op	0.37
mainnet_e58758 - phase0 processSlashings	719.00 ns/op	1.8740 us/op	0.38
mainnet_e58758 - phase0 processEth1DataReset	634.00 ns/op	1.2460 us/op	0.51
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	855.88 us/op	1.7601 ms/op	0.49
mainnet_e58758 - phase0 processSlashingsReset	2.6510 us/op	5.4440 us/op	0.49
mainnet_e58758 - phase0 processRandaoMixesReset	2.4940 us/op	12.913 us/op	0.19
mainnet_e58758 - phase0 processHistoricalRootsUpdate	567.00 ns/op	1.2930 us/op	0.44
mainnet_e58758 - phase0 processParticipationRecordUpdates	2.4170 us/op	7.5960 us/op	0.32
mainnet_e58758 - phase0 afterProcessEpoch	71.025 ms/op	119.59 ms/op	0.59
phase0 processEffectiveBalanceUpdates - 250000 normalcase	1.0989 ms/op	1.7889 ms/op	0.61
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	1.2421 ms/op	2.9182 ms/op	0.43
altair processInactivityUpdates - 250000 normalcase	27.035 ms/op	31.878 ms/op	0.85
altair processInactivityUpdates - 250000 worstcase	28.911 ms/op	33.416 ms/op	0.87
phase0 processRegistryUpdates - 250000 normalcase	8.8710 us/op	22.024 us/op	0.40
phase0 processRegistryUpdates - 250000 badcase_full_deposits	412.07 us/op	723.80 us/op	0.57
phase0 processRegistryUpdates - 250000 worstcase 0.5	140.88 ms/op	217.15 ms/op	0.65
altair processRewardsAndPenalties - 250000 normalcase	59.986 ms/op	89.324 ms/op	0.67
altair processRewardsAndPenalties - 250000 worstcase	57.838 ms/op	85.414 ms/op	0.68
phase0 getAttestationDeltas - 250000 normalcase	6.0920 ms/op	14.978 ms/op	0.41
phase0 getAttestationDeltas - 250000 worstcase	6.3040 ms/op	14.747 ms/op	0.43
phase0 processSlashings - 250000 worstcase	85.828 us/op	149.09 us/op	0.58
altair processSyncCommitteeUpdates - 250000	142.61 ms/op	186.25 ms/op	0.77
BeaconState.hashTreeRoot - No change	649.00 ns/op	820.00 ns/op	0.79
BeaconState.hashTreeRoot - 1 full validator	132.58 us/op	165.36 us/op	0.80
BeaconState.hashTreeRoot - 32 full validator	1.1355 ms/op	1.7205 ms/op	0.66
BeaconState.hashTreeRoot - 512 full validator	13.583 ms/op	21.641 ms/op	0.63
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	176.42 us/op	206.03 us/op	0.86
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	2.9090 ms/op	2.6467 ms/op	1.10
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	31.155 ms/op	38.020 ms/op	0.82
BeaconState.hashTreeRoot - 1 balances	151.86 us/op	175.68 us/op	0.86
BeaconState.hashTreeRoot - 32 balances	1.6601 ms/op	1.5925 ms/op	1.04
BeaconState.hashTreeRoot - 512 balances	15.829 ms/op	15.784 ms/op	1.00
BeaconState.hashTreeRoot - 250000 balances	209.09 ms/op	242.48 ms/op	0.86
aggregationBits - 2048 els - zipIndexesInBitList	26.359 us/op	48.298 us/op	0.55
byteArrayEquals 32	71.281 ns/op	115.05 ns/op	0.62
Buffer.compare 32	38.109 ns/op	67.474 ns/op	0.56
byteArrayEquals 1024	1.7786 us/op	2.2824 us/op	0.78
Buffer.compare 1024	52.863 ns/op	93.152 ns/op	0.57
byteArrayEquals 16384	29.050 us/op	45.140 us/op	0.64
Buffer.compare 16384	222.41 ns/op	283.07 ns/op	0.79
byteArrayEquals 123687377	223.38 ms/op	319.75 ms/op	0.70
Buffer.compare 123687377	7.2964 ms/op	12.145 ms/op	0.60
byteArrayEquals 32 - diff last byte	67.166 ns/op	92.297 ns/op	0.73
Buffer.compare 32 - diff last byte	40.444 ns/op	73.255 ns/op	0.55
byteArrayEquals 1024 - diff last byte	1.8474 us/op	2.5725 us/op	0.72
Buffer.compare 1024 - diff last byte	49.070 ns/op	86.800 ns/op	0.57
byteArrayEquals 16384 - diff last byte	27.976 us/op	37.036 us/op	0.76
Buffer.compare 16384 - diff last byte	241.40 ns/op	300.36 ns/op	0.80
byteArrayEquals 123687377 - diff last byte	221.34 ms/op	317.83 ms/op	0.70
Buffer.compare 123687377 - diff last byte	7.3449 ms/op	13.285 ms/op	0.55
byteArrayEquals 32 - random bytes	6.0290 ns/op	8.5920 ns/op	0.70
Buffer.compare 32 - random bytes	42.699 ns/op	87.609 ns/op	0.49
byteArrayEquals 1024 - random bytes	5.9960 ns/op	8.7870 ns/op	0.68
Buffer.compare 1024 - random bytes	37.893 ns/op	75.831 ns/op	0.50
byteArrayEquals 16384 - random bytes	6.3090 ns/op	8.7550 ns/op	0.72
Buffer.compare 16384 - random bytes	39.313 ns/op	76.354 ns/op	0.51
byteArrayEquals 123687377 - random bytes	18.910 ns/op	21.340 ns/op	0.89
Buffer.compare 123687377 - random bytes	53.480 ns/op	90.070 ns/op	0.59
regular array get 100000 times	44.070 us/op	65.254 us/op	0.68
wrappedArray get 100000 times	43.632 us/op	59.678 us/op	0.73
arrayWithProxy get 100000 times	10.877 ms/op	16.892 ms/op	0.64
ssz.Root.equals	59.713 ns/op	64.385 ns/op	0.93
byteArrayEquals	58.981 ns/op	63.602 ns/op	0.93
Buffer.compare	10.489 ns/op	14.672 ns/op	0.71
shuffle list - 16384 els	6.4855 ms/op	9.5762 ms/op	0.68
shuffle list - 250000 els	94.769 ms/op	145.73 ms/op	0.65
processSlot - 1 slots	21.082 us/op	26.589 us/op	0.79
processSlot - 32 slots	3.4734 ms/op	5.6918 ms/op	0.61
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	62.913 ms/op	72.778 ms/op	0.86
getCommitteeAssignments - req 1 vs - 250000 vc	3.0765 ms/op	2.9460 ms/op	1.04
getCommitteeAssignments - req 100 vs - 250000 vc	3.9981 ms/op	4.1339 ms/op	0.97
getCommitteeAssignments - req 1000 vs - 250000 vc	5.1752 ms/op	4.7432 ms/op	1.09
findModifiedValidators - 10000 modified validators	417.60 ms/op	483.54 ms/op	0.86
findModifiedValidators - 1000 modified validators	327.34 ms/op	311.92 ms/op	1.05
findModifiedValidators - 100 modified validators	266.68 ms/op	305.58 ms/op	0.87
findModifiedValidators - 10 modified validators	221.98 ms/op	245.66 ms/op	0.90
findModifiedValidators - 1 modified validators	241.46 ms/op	252.45 ms/op	0.96
findModifiedValidators - no difference	260.20 ms/op	227.26 ms/op	1.14
compare ViewDUs	5.3624 s/op	5.5458 s/op	0.97
compare each validator Uint8Array	2.0609 s/op	2.0277 s/op	1.02
compare ViewDU to Uint8Array	1.4261 s/op	1.5043 s/op	0.95
migrate state 1000000 validators, 24 modified, 0 new	743.74 ms/op	871.52 ms/op	0.85
migrate state 1000000 validators, 1700 modified, 1000 new	913.41 ms/op	1.3018 s/op	0.70
migrate state 1000000 validators, 3400 modified, 2000 new	1.2004 s/op	1.5074 s/op	0.80
migrate state 1500000 validators, 24 modified, 0 new	769.78 ms/op	881.46 ms/op	0.87
migrate state 1500000 validators, 1700 modified, 1000 new	973.56 ms/op	1.4174 s/op	0.69
migrate state 1500000 validators, 3400 modified, 2000 new	1.1078 s/op	1.6414 s/op	0.67
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei	4.9400 ns/op	7.7500 ns/op	0.64
state getBlockRootAtSlot - 250000 vs - 7PWei	464.83 ns/op	1.2254 us/op	0.38
computeProposers - vc 250000	6.3401 ms/op	13.980 ms/op	0.45
computeEpochShuffling - vc 250000	88.615 ms/op	147.31 ms/op	0.60
getNextSyncCommittee - vc 250000	132.53 ms/op	211.70 ms/op	0.63
computeSigningRoot for AttestationData	24.119 us/op	37.229 us/op	0.65
hash AttestationData serialized data then Buffer.toString(base64)	1.2964 us/op	2.7138 us/op	0.48
toHexString serialized data	892.10 ns/op	1.6882 us/op	0.53
Buffer.toString(base64)	169.77 ns/op	291.46 ns/op	0.58

by benchmarkbot/action

[jeluard]

It might be nice to have unit tests to ensure no regression are introduced to those kind of expectations.

[jeluard]

The alternative could be to keep subnetAggregatorsBySlot sorted at insertion time? We might not have the necessary data structure yet.

[nflaig]

I had three options in mind

• sort keys when pruning (as implemented)
• switch to pruneBySlot logic, I am not sure if this strategy is correct here, although it could keep the cache size lower, maybe @tuyennhv has some thoughts
• as you suggested @jeluard always have keys sorted, I think we wanna keep a map for quick lookup times, although might not matter much since the array would have max 64 items

[jeluard]

Current approach is probably fine given the sizes involved, probably better not to over engineer yet :)

[nflaig]

I overlooked this yesterday, we have a ordered map

lodestar/packages/beacon-node/src/util/map.ts

Line 6 in ae984f0

export class OrderedMap<K, V> {

This one keeps track of the keys which is the tradeoff I guess

[nflaig]

Alternative using an ordered map unstable...nflaig/ordered-map-aggregator-tracker

[nflaig]

turns out our "ordered" map isn't actually ordered... added units tests now

[twoeths]

Current approach is probably fine given the sizes involved, probably better not to over engineer yet :)

I agree with this considering the cache size is just 64, may want to add a note for the reasoning we choose this approach

[nflaig]

It might be nice to have unit tests to ensure no regression are introduced to those kind of expectations.

Could add unit tests for aggregator tracker but the logic is pretty simple, the problem with this issue is that it involves multiple parts of the code base, so only good way too test this is in an e2e or even better a sim test.

[jeluard]

@tuyennhv sorry I misclicked and re-requested your review

[nflaig]

Keeping as draft for now, happy to elaborate on different approaches, and probably good idea to have a minimal set of units tests on aggregator tracker.

[wemeetagain]

🎉 This PR is included in v1.18.0 🎉