Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: move 4844 tests to sim tests #6622

Merged
merged 4 commits into from
Apr 9, 2024
Merged

test: move 4844 tests to sim tests #6622

merged 4 commits into from
Apr 9, 2024

Conversation

nazarhussain
Copy link
Contributor

Motivation

Cover more scenarios in the sim tests.

Description

  • Create support for blob transaction in sim tests
  • Add support for the blob assertion to deneb tests
  • Refactor syncing logic to make more re-useable

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner April 2, 2024 10:18
@nazarhussain nazarhussain self-assigned this Apr 2, 2024
Copy link

codecov bot commented Apr 2, 2024

Codecov Report

Merging #6622 (5d1f509) into unstable (b1f4e7e) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff            @@
##           unstable    #6622   +/-   ##
=========================================
  Coverage     61.69%   61.69%           
=========================================
  Files           556      556           
  Lines         58820    58820           
  Branches       1887     1887           
=========================================
  Hits          36287    36287           
  Misses        22492    22492           
  Partials         41       41           

Copy link
Contributor

github-actions bot commented Apr 2, 2024

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 6dfecdd Previous: 65927c6 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 563.29 us/op 627.21 us/op 0.90
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 47.916 us/op 52.724 us/op 0.91
BLS verify - blst-native 1.1323 ms/op 1.0981 ms/op 1.03
BLS verifyMultipleSignatures 3 - blst-native 2.3802 ms/op 2.3436 ms/op 1.02
BLS verifyMultipleSignatures 8 - blst-native 5.2813 ms/op 5.1807 ms/op 1.02
BLS verifyMultipleSignatures 32 - blst-native 19.647 ms/op 18.565 ms/op 1.06
BLS verifyMultipleSignatures 64 - blst-native 38.335 ms/op 36.253 ms/op 1.06
BLS verifyMultipleSignatures 128 - blst-native 76.144 ms/op 71.694 ms/op 1.06
BLS deserializing 10000 signatures 819.41 ms/op 804.89 ms/op 1.02
BLS deserializing 100000 signatures 8.6513 s/op 8.1518 s/op 1.06
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.1249 ms/op 1.1271 ms/op 1.00
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.2713 ms/op 1.4424 ms/op 0.88
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4326 ms/op 2.2853 ms/op 1.06
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.8308 ms/op 3.3356 ms/op 1.15
BLS verifyMultipleSignatures - same message - 128 - blst-native 7.0498 ms/op 6.5747 ms/op 1.07
BLS aggregatePubkeys 32 - blst-native 22.374 us/op 23.924 us/op 0.94
BLS aggregatePubkeys 128 - blst-native 87.856 us/op 85.914 us/op 1.02
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 52.042 ms/op 47.677 ms/op 1.09
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 56.983 ms/op 49.961 ms/op 1.14
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 26.588 ms/op 27.696 ms/op 0.96
getSlashingsAndExits - default max 120.27 us/op 161.55 us/op 0.74
getSlashingsAndExits - 2k 351.51 us/op 443.96 us/op 0.79
proposeBlockBody type=full, size=empty 4.3045 ms/op 4.5914 ms/op 0.94
isKnown best case - 1 super set check 551.00 ns/op 398.00 ns/op 1.38
isKnown normal case - 2 super set checks 660.00 ns/op 421.00 ns/op 1.57
isKnown worse case - 16 super set checks 850.00 ns/op 405.00 ns/op 2.10
InMemoryCheckpointStateCache - add get delete 8.7900 us/op 4.6020 us/op 1.91
validate api signedAggregateAndProof - struct 2.6163 ms/op 2.4607 ms/op 1.06
validate gossip signedAggregateAndProof - struct 2.5616 ms/op 2.4515 ms/op 1.04
validate gossip attestation - vc 640000 1.2855 ms/op 1.1988 ms/op 1.07
batch validate gossip attestation - vc 640000 - chunk 32 165.80 us/op 158.30 us/op 1.05
batch validate gossip attestation - vc 640000 - chunk 64 170.29 us/op 141.68 us/op 1.20
batch validate gossip attestation - vc 640000 - chunk 128 141.85 us/op 127.15 us/op 1.12
batch validate gossip attestation - vc 640000 - chunk 256 131.93 us/op 115.90 us/op 1.14
pickEth1Vote - no votes 836.02 us/op 936.38 us/op 0.89
pickEth1Vote - max votes 5.3141 ms/op 7.9667 ms/op 0.67
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.708 ms/op 14.376 ms/op 0.81
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.396 ms/op 18.842 ms/op 0.98
pickEth1Vote - Eth1Data fastSerialize value x2048 424.84 us/op 481.14 us/op 0.88
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.4119 ms/op 4.5691 ms/op 0.97
bytes32 toHexString 451.00 ns/op 459.00 ns/op 0.98
bytes32 Buffer.toString(hex) 311.00 ns/op 307.00 ns/op 1.01
bytes32 Buffer.toString(hex) from Uint8Array 424.00 ns/op 436.00 ns/op 0.97
bytes32 Buffer.toString(hex) + 0x 312.00 ns/op 323.00 ns/op 0.97
Object access 1 prop 0.20200 ns/op 0.20400 ns/op 0.99
Map access 1 prop 0.20600 ns/op 0.19400 ns/op 1.06
Object get x1000 5.2370 ns/op 5.5740 ns/op 0.94
Map get x1000 0.73300 ns/op 0.76300 ns/op 0.96
Object set x1000 24.685 ns/op 25.135 ns/op 0.98
Map set x1000 17.231 ns/op 17.652 ns/op 0.98
Return object 10000 times 0.23370 ns/op 0.23530 ns/op 0.99
Throw Error 10000 times 2.7423 us/op 2.8088 us/op 0.98
fastMsgIdFn sha256 / 200 bytes 1.9840 us/op 2.0080 us/op 0.99
fastMsgIdFn h32 xxhash / 200 bytes 293.00 ns/op 303.00 ns/op 0.97
fastMsgIdFn h64 xxhash / 200 bytes 373.00 ns/op 365.00 ns/op 1.02
fastMsgIdFn sha256 / 1000 bytes 6.1340 us/op 6.3400 us/op 0.97
fastMsgIdFn h32 xxhash / 1000 bytes 404.00 ns/op 433.00 ns/op 0.93
fastMsgIdFn h64 xxhash / 1000 bytes 414.00 ns/op 432.00 ns/op 0.96
fastMsgIdFn sha256 / 10000 bytes 53.062 us/op 53.854 us/op 0.99
fastMsgIdFn h32 xxhash / 10000 bytes 1.7710 us/op 1.8870 us/op 0.94
fastMsgIdFn h64 xxhash / 10000 bytes 1.2230 us/op 1.3090 us/op 0.93
send data - 1000 256B messages 11.400 ms/op 12.103 ms/op 0.94
send data - 1000 512B messages 14.158 ms/op 15.396 ms/op 0.92
send data - 1000 1024B messages 23.539 ms/op 26.786 ms/op 0.88
send data - 1000 1200B messages 26.061 ms/op 29.708 ms/op 0.88
send data - 1000 2048B messages 33.552 ms/op 36.774 ms/op 0.91
send data - 1000 4096B messages 31.104 ms/op 32.875 ms/op 0.95
send data - 1000 16384B messages 92.669 ms/op 83.741 ms/op 1.11
send data - 1000 65536B messages 400.70 ms/op 381.21 ms/op 1.05
enrSubnets - fastDeserialize 64 bits 1.0330 us/op 1.0530 us/op 0.98
enrSubnets - ssz BitVector 64 bits 437.00 ns/op 421.00 ns/op 1.04
enrSubnets - fastDeserialize 4 bits 211.00 ns/op 198.00 ns/op 1.07
enrSubnets - ssz BitVector 4 bits 441.00 ns/op 418.00 ns/op 1.06
prioritizePeers score -10:0 att 32-0.1 sync 2-0 66.808 us/op 64.341 us/op 1.04
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 75.025 us/op 81.577 us/op 0.92
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 108.64 us/op 110.38 us/op 0.98
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 189.45 us/op 182.43 us/op 1.04
prioritizePeers score 0:0 att 64-1 sync 4-1 214.11 us/op 201.20 us/op 1.06
array of 16000 items push then shift 1.3081 us/op 1.2241 us/op 1.07
LinkedList of 16000 items push then shift 7.3090 ns/op 6.6740 ns/op 1.10
array of 16000 items push then pop 61.571 ns/op 63.924 ns/op 0.96
LinkedList of 16000 items push then pop 5.8020 ns/op 5.5940 ns/op 1.04
array of 24000 items push then shift 1.9076 us/op 1.8556 us/op 1.03
LinkedList of 24000 items push then shift 5.9720 ns/op 6.3310 ns/op 0.94
array of 24000 items push then pop 84.141 ns/op 91.359 ns/op 0.92
LinkedList of 24000 items push then pop 5.7430 ns/op 5.4800 ns/op 1.05
intersect bitArray bitLen 8 4.7170 ns/op 4.4120 ns/op 1.07
intersect array and set length 8 47.887 ns/op 45.416 ns/op 1.05
intersect bitArray bitLen 128 29.315 ns/op 27.831 ns/op 1.05
intersect array and set length 128 683.31 ns/op 640.17 ns/op 1.07
bitArray.getTrueBitIndexes() bitLen 128 1.2160 us/op 1.2540 us/op 0.97
bitArray.getTrueBitIndexes() bitLen 248 1.9160 us/op 2.0270 us/op 0.95
bitArray.getTrueBitIndexes() bitLen 512 4.0480 us/op 3.8350 us/op 1.06
Buffer.concat 32 items 842.00 ns/op 804.00 ns/op 1.05
Uint8Array.set 32 items 1.6150 us/op 1.7020 us/op 0.95
Set add up to 64 items then delete first 1.7932 us/op 1.7324 us/op 1.04
OrderedSet add up to 64 items then delete first 2.7408 us/op 2.6818 us/op 1.02
Set add up to 64 items then delete last 2.0398 us/op 1.9877 us/op 1.03
OrderedSet add up to 64 items then delete last 3.0054 us/op 2.9536 us/op 1.02
Set add up to 64 items then delete middle 1.9928 us/op 1.9893 us/op 1.00
OrderedSet add up to 64 items then delete middle 4.3727 us/op 4.1535 us/op 1.05
Set add up to 128 items then delete first 3.9417 us/op 3.9718 us/op 0.99
OrderedSet add up to 128 items then delete first 6.1937 us/op 6.1361 us/op 1.01
Set add up to 128 items then delete last 3.8026 us/op 3.8273 us/op 0.99
OrderedSet add up to 128 items then delete last 5.7311 us/op 5.6706 us/op 1.01
Set add up to 128 items then delete middle 3.7878 us/op 3.7961 us/op 1.00
OrderedSet add up to 128 items then delete middle 10.672 us/op 10.674 us/op 1.00
Set add up to 256 items then delete first 7.9289 us/op 7.6738 us/op 1.03
OrderedSet add up to 256 items then delete first 12.392 us/op 12.209 us/op 1.01
Set add up to 256 items then delete last 7.4470 us/op 7.4449 us/op 1.00
OrderedSet add up to 256 items then delete last 11.543 us/op 11.296 us/op 1.02
Set add up to 256 items then delete middle 7.4275 us/op 7.4236 us/op 1.00
OrderedSet add up to 256 items then delete middle 31.758 us/op 30.767 us/op 1.03
transfer serialized Status (84 B) 1.3730 us/op 1.4070 us/op 0.98
copy serialized Status (84 B) 1.0790 us/op 1.1300 us/op 0.95
transfer serialized SignedVoluntaryExit (112 B) 1.4500 us/op 1.5360 us/op 0.94
copy serialized SignedVoluntaryExit (112 B) 1.2030 us/op 1.1990 us/op 1.00
transfer serialized ProposerSlashing (416 B) 1.6180 us/op 2.0300 us/op 0.80
copy serialized ProposerSlashing (416 B) 1.3410 us/op 1.9290 us/op 0.70
transfer serialized Attestation (485 B) 1.5740 us/op 2.4910 us/op 0.63
copy serialized Attestation (485 B) 1.8030 us/op 1.9890 us/op 0.91
transfer serialized AttesterSlashing (33232 B) 2.1370 us/op 2.6960 us/op 0.79
copy serialized AttesterSlashing (33232 B) 4.2710 us/op 4.2570 us/op 1.00
transfer serialized Small SignedBeaconBlock (128000 B) 2.2040 us/op 2.6450 us/op 0.83
copy serialized Small SignedBeaconBlock (128000 B) 8.6870 us/op 9.0720 us/op 0.96
transfer serialized Avg SignedBeaconBlock (200000 B) 2.0280 us/op 2.5440 us/op 0.80
copy serialized Avg SignedBeaconBlock (200000 B) 13.443 us/op 12.661 us/op 1.06
transfer serialized BlobsSidecar (524380 B) 2.6910 us/op 3.0400 us/op 0.89
copy serialized BlobsSidecar (524380 B) 101.14 us/op 70.538 us/op 1.43
transfer serialized Big SignedBeaconBlock (1000000 B) 2.8040 us/op 3.3050 us/op 0.85
copy serialized Big SignedBeaconBlock (1000000 B) 322.27 us/op 133.64 us/op 2.41
pass gossip attestations to forkchoice per slot 3.0736 ms/op 2.6196 ms/op 1.17
forkChoice updateHead vc 100000 bc 64 eq 0 455.87 us/op 415.70 us/op 1.10
forkChoice updateHead vc 600000 bc 64 eq 0 3.7241 ms/op 2.7159 ms/op 1.37
forkChoice updateHead vc 1000000 bc 64 eq 0 4.5991 ms/op 4.2181 ms/op 1.09
forkChoice updateHead vc 600000 bc 320 eq 0 2.6338 ms/op 2.5639 ms/op 1.03
forkChoice updateHead vc 600000 bc 1200 eq 0 2.7710 ms/op 2.5952 ms/op 1.07
forkChoice updateHead vc 600000 bc 7200 eq 0 3.9010 ms/op 3.2708 ms/op 1.19
forkChoice updateHead vc 600000 bc 64 eq 1000 10.200 ms/op 9.9033 ms/op 1.03
forkChoice updateHead vc 600000 bc 64 eq 10000 10.253 ms/op 9.7852 ms/op 1.05
forkChoice updateHead vc 600000 bc 64 eq 300000 16.246 ms/op 11.889 ms/op 1.37
computeDeltas 500000 validators 300 proto nodes 3.5571 ms/op 3.0280 ms/op 1.17
computeDeltas 500000 validators 1200 proto nodes 3.2386 ms/op 2.8487 ms/op 1.14
computeDeltas 500000 validators 7200 proto nodes 3.1241 ms/op 2.8616 ms/op 1.09
computeDeltas 750000 validators 300 proto nodes 4.6803 ms/op 4.3352 ms/op 1.08
computeDeltas 750000 validators 1200 proto nodes 4.7883 ms/op 4.3885 ms/op 1.09
computeDeltas 750000 validators 7200 proto nodes 4.6698 ms/op 4.3698 ms/op 1.07
computeDeltas 1400000 validators 300 proto nodes 8.8890 ms/op 9.1265 ms/op 0.97
computeDeltas 1400000 validators 1200 proto nodes 8.8203 ms/op 8.8408 ms/op 1.00
computeDeltas 1400000 validators 7200 proto nodes 8.9724 ms/op 8.9838 ms/op 1.00
computeDeltas 2100000 validators 300 proto nodes 13.605 ms/op 13.738 ms/op 0.99
computeDeltas 2100000 validators 1200 proto nodes 13.386 ms/op 13.104 ms/op 1.02
computeDeltas 2100000 validators 7200 proto nodes 13.310 ms/op 13.057 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei normalcase 1.8187 ms/op 1.5979 ms/op 1.14
altair processAttestation - 250000 vs - 7PWei worstcase 2.4660 ms/op 2.3178 ms/op 1.06
altair processAttestation - setStatus - 1/6 committees join 106.76 us/op 102.57 us/op 1.04
altair processAttestation - setStatus - 1/3 committees join 206.64 us/op 201.31 us/op 1.03
altair processAttestation - setStatus - 1/2 committees join 311.85 us/op 284.35 us/op 1.10
altair processAttestation - setStatus - 2/3 committees join 380.55 us/op 376.00 us/op 1.01
altair processAttestation - setStatus - 4/5 committees join 520.03 us/op 501.34 us/op 1.04
altair processAttestation - setStatus - 100% committees join 605.37 us/op 557.44 us/op 1.09
altair processBlock - 250000 vs - 7PWei normalcase 6.2247 ms/op 6.8625 ms/op 0.91
altair processBlock - 250000 vs - 7PWei normalcase hashState 27.619 ms/op 28.261 ms/op 0.98
altair processBlock - 250000 vs - 7PWei worstcase 34.481 ms/op 33.239 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase hashState 87.256 ms/op 82.467 ms/op 1.06
phase0 processBlock - 250000 vs - 7PWei normalcase 2.1518 ms/op 2.2847 ms/op 0.94
phase0 processBlock - 250000 vs - 7PWei worstcase 23.289 ms/op 23.524 ms/op 0.99
altair processEth1Data - 250000 vs - 7PWei normalcase 274.59 us/op 267.90 us/op 1.03
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 8.0870 us/op 3.5450 us/op 2.28
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 53.678 us/op 30.893 us/op 1.74
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 17.876 us/op 14.036 us/op 1.27
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 6.4980 us/op 11.690 us/op 0.56
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 172.67 us/op 162.20 us/op 1.06
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 843.86 us/op 1.0509 ms/op 0.80
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.4508 ms/op 1.4481 ms/op 1.00
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.2444 ms/op 1.3716 ms/op 0.91
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.0234 ms/op 2.0847 ms/op 0.97
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.4638 ms/op 1.5561 ms/op 0.94
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.2789 ms/op 3.7570 ms/op 1.14
Tree 40 250000 create 243.82 ms/op 272.35 ms/op 0.90
Tree 40 250000 get(125000) 114.06 ns/op 112.79 ns/op 1.01
Tree 40 250000 set(125000) 770.90 ns/op 745.81 ns/op 1.03
Tree 40 250000 toArray() 19.954 ms/op 9.6558 ms/op 2.07
Tree 40 250000 iterate all - toArray() + loop 19.274 ms/op 9.7725 ms/op 1.97
Tree 40 250000 iterate all - get(i) 50.765 ms/op 38.406 ms/op 1.32
MutableVector 250000 create 10.736 ms/op 11.826 ms/op 0.91
MutableVector 250000 get(125000) 6.0090 ns/op 5.3960 ns/op 1.11
MutableVector 250000 set(125000) 213.97 ns/op 198.22 ns/op 1.08
MutableVector 250000 toArray() 2.2026 ms/op 2.4298 ms/op 0.91
MutableVector 250000 iterate all - toArray() + loop 2.5704 ms/op 2.5369 ms/op 1.01
MutableVector 250000 iterate all - get(i) 1.3469 ms/op 1.2656 ms/op 1.06
Array 250000 create 2.1175 ms/op 2.0940 ms/op 1.01
Array 250000 clone - spread 1.0486 ms/op 1.2352 ms/op 0.85
Array 250000 get(125000) 1.0210 ns/op 1.0810 ns/op 0.94
Array 250000 set(125000) 1.2340 ns/op 1.2850 ns/op 0.96
Array 250000 iterate all - loop 156.46 us/op 153.35 us/op 1.02
effectiveBalanceIncrements clone Uint8Array 300000 12.721 us/op 15.277 us/op 0.83
effectiveBalanceIncrements clone MutableVector 300000 387.00 ns/op 460.00 ns/op 0.84
effectiveBalanceIncrements rw all Uint8Array 300000 186.91 us/op 183.50 us/op 1.02
effectiveBalanceIncrements rw all MutableVector 300000 62.273 ms/op 67.841 ms/op 0.92
phase0 afterProcessEpoch - 250000 vs - 7PWei 77.517 ms/op 72.473 ms/op 1.07
phase0 beforeProcessEpoch - 250000 vs - 7PWei 45.361 ms/op 37.120 ms/op 1.22
altair processEpoch - mainnet_e81889 375.22 ms/op 412.66 ms/op 0.91
mainnet_e81889 - altair beforeProcessEpoch 72.020 ms/op 64.647 ms/op 1.11
mainnet_e81889 - altair processJustificationAndFinalization 12.790 us/op 6.7260 us/op 1.90
mainnet_e81889 - altair processInactivityUpdates 4.2496 ms/op 4.6855 ms/op 0.91
mainnet_e81889 - altair processRewardsAndPenalties 71.387 ms/op 73.016 ms/op 0.98
mainnet_e81889 - altair processRegistryUpdates 2.1870 us/op 2.0870 us/op 1.05
mainnet_e81889 - altair processSlashings 567.00 ns/op 645.00 ns/op 0.88
mainnet_e81889 - altair processEth1DataReset 584.00 ns/op 692.00 ns/op 0.84
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.0299 ms/op 1.6411 ms/op 0.63
mainnet_e81889 - altair processSlashingsReset 2.6130 us/op 2.4710 us/op 1.06
mainnet_e81889 - altair processRandaoMixesReset 3.9060 us/op 2.7560 us/op 1.42
mainnet_e81889 - altair processHistoricalRootsUpdate 735.00 ns/op 638.00 ns/op 1.15
mainnet_e81889 - altair processParticipationFlagUpdates 1.5330 us/op 1.4870 us/op 1.03
mainnet_e81889 - altair processSyncCommitteeUpdates 625.00 ns/op 657.00 ns/op 0.95
mainnet_e81889 - altair afterProcessEpoch 82.216 ms/op 80.077 ms/op 1.03
capella processEpoch - mainnet_e217614 1.5683 s/op 1.6515 s/op 0.95
mainnet_e217614 - capella beforeProcessEpoch 417.75 ms/op 389.68 ms/op 1.07
mainnet_e217614 - capella processJustificationAndFinalization 9.0940 us/op 10.729 us/op 0.85
mainnet_e217614 - capella processInactivityUpdates 15.153 ms/op 15.139 ms/op 1.00
mainnet_e217614 - capella processRewardsAndPenalties 471.75 ms/op 450.04 ms/op 1.05
mainnet_e217614 - capella processRegistryUpdates 14.336 us/op 15.221 us/op 0.94
mainnet_e217614 - capella processSlashings 545.00 ns/op 675.00 ns/op 0.81
mainnet_e217614 - capella processEth1DataReset 568.00 ns/op 551.00 ns/op 1.03
mainnet_e217614 - capella processEffectiveBalanceUpdates 3.3829 ms/op 4.3070 ms/op 0.79
mainnet_e217614 - capella processSlashingsReset 2.2580 us/op 1.4520 us/op 1.56
mainnet_e217614 - capella processRandaoMixesReset 3.0860 us/op 1.8480 us/op 1.67
mainnet_e217614 - capella processHistoricalRootsUpdate 603.00 ns/op 475.00 ns/op 1.27
mainnet_e217614 - capella processParticipationFlagUpdates 1.2050 us/op 768.00 ns/op 1.57
mainnet_e217614 - capella afterProcessEpoch 206.61 ms/op 234.59 ms/op 0.88
phase0 processEpoch - mainnet_e58758 417.69 ms/op 404.99 ms/op 1.03
mainnet_e58758 - phase0 beforeProcessEpoch 123.98 ms/op 99.742 ms/op 1.24
mainnet_e58758 - phase0 processJustificationAndFinalization 15.987 us/op 8.6810 us/op 1.84
mainnet_e58758 - phase0 processRewardsAndPenalties 57.553 ms/op 63.211 ms/op 0.91
mainnet_e58758 - phase0 processRegistryUpdates 9.6970 us/op 10.079 us/op 0.96
mainnet_e58758 - phase0 processSlashings 599.00 ns/op 704.00 ns/op 0.85
mainnet_e58758 - phase0 processEth1DataReset 566.00 ns/op 598.00 ns/op 0.95
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.4194 ms/op 1.3495 ms/op 1.05
mainnet_e58758 - phase0 processSlashingsReset 3.9460 us/op 2.6880 us/op 1.47
mainnet_e58758 - phase0 processRandaoMixesReset 3.5380 us/op 2.9380 us/op 1.20
mainnet_e58758 - phase0 processHistoricalRootsUpdate 534.00 ns/op 606.00 ns/op 0.88
mainnet_e58758 - phase0 processParticipationRecordUpdates 2.3830 us/op 2.9170 us/op 0.82
mainnet_e58758 - phase0 afterProcessEpoch 71.983 ms/op 66.995 ms/op 1.07
phase0 processEffectiveBalanceUpdates - 250000 normalcase 964.96 us/op 938.87 us/op 1.03
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.0907 ms/op 1.0607 ms/op 1.03
altair processInactivityUpdates - 250000 normalcase 23.486 ms/op 24.855 ms/op 0.94
altair processInactivityUpdates - 250000 worstcase 24.356 ms/op 24.400 ms/op 1.00
phase0 processRegistryUpdates - 250000 normalcase 3.8430 us/op 6.5080 us/op 0.59
phase0 processRegistryUpdates - 250000 badcase_full_deposits 395.18 us/op 213.89 us/op 1.85
phase0 processRegistryUpdates - 250000 worstcase 0.5 102.39 ms/op 120.99 ms/op 0.85
altair processRewardsAndPenalties - 250000 normalcase 69.707 ms/op 70.843 ms/op 0.98
altair processRewardsAndPenalties - 250000 worstcase 58.756 ms/op 67.671 ms/op 0.87
phase0 getAttestationDeltas - 250000 normalcase 5.5559 ms/op 5.5798 ms/op 1.00
phase0 getAttestationDeltas - 250000 worstcase 7.7819 ms/op 5.6775 ms/op 1.37
phase0 processSlashings - 250000 worstcase 86.125 us/op 75.182 us/op 1.15
altair processSyncCommitteeUpdates - 250000 116.31 ms/op 109.88 ms/op 1.06
BeaconState.hashTreeRoot - No change 517.00 ns/op 361.00 ns/op 1.43
BeaconState.hashTreeRoot - 1 full validator 125.15 us/op 151.63 us/op 0.83
BeaconState.hashTreeRoot - 32 full validator 1.2889 ms/op 1.5924 ms/op 0.81
BeaconState.hashTreeRoot - 512 full validator 15.395 ms/op 16.425 ms/op 0.94
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 175.01 us/op 153.51 us/op 1.14
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.1081 ms/op 1.7795 ms/op 1.18
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 23.987 ms/op 27.816 ms/op 0.86
BeaconState.hashTreeRoot - 1 balances 109.98 us/op 115.97 us/op 0.95
BeaconState.hashTreeRoot - 32 balances 849.35 us/op 1.1443 ms/op 0.74
BeaconState.hashTreeRoot - 512 balances 8.8955 ms/op 10.560 ms/op 0.84
BeaconState.hashTreeRoot - 250000 balances 173.66 ms/op 173.84 ms/op 1.00
aggregationBits - 2048 els - zipIndexesInBitList 23.066 us/op 19.081 us/op 1.21
byteArrayEquals 32 62.511 ns/op 64.485 ns/op 0.97
Buffer.compare 32 36.113 ns/op 39.294 ns/op 0.92
byteArrayEquals 1024 1.6918 us/op 1.7520 us/op 0.97
Buffer.compare 1024 42.917 ns/op 47.440 ns/op 0.90
byteArrayEquals 16384 26.825 us/op 27.801 us/op 0.96
Buffer.compare 16384 197.13 ns/op 230.18 ns/op 0.86
byteArrayEquals 123687377 200.20 ms/op 210.53 ms/op 0.95
Buffer.compare 123687377 6.0249 ms/op 3.8619 ms/op 1.56
byteArrayEquals 32 - diff last byte 59.794 ns/op 61.922 ns/op 0.97
Buffer.compare 32 - diff last byte 36.204 ns/op 39.059 ns/op 0.93
byteArrayEquals 1024 - diff last byte 1.7304 us/op 1.5624 us/op 1.11
Buffer.compare 1024 - diff last byte 42.162 ns/op 43.371 ns/op 0.97
byteArrayEquals 16384 - diff last byte 26.581 us/op 27.469 us/op 0.97
Buffer.compare 16384 - diff last byte 210.11 ns/op 223.42 ns/op 0.94
byteArrayEquals 123687377 - diff last byte 198.42 ms/op 205.52 ms/op 0.97
Buffer.compare 123687377 - diff last byte 5.8530 ms/op 3.7580 ms/op 1.56
byteArrayEquals 32 - random bytes 4.6720 ns/op 4.4240 ns/op 1.06
Buffer.compare 32 - random bytes 39.258 ns/op 39.873 ns/op 0.98
byteArrayEquals 1024 - random bytes 4.7030 ns/op 4.3960 ns/op 1.07
Buffer.compare 1024 - random bytes 39.045 ns/op 38.350 ns/op 1.02
byteArrayEquals 16384 - random bytes 4.5730 ns/op 4.3910 ns/op 1.04
Buffer.compare 16384 - random bytes 39.391 ns/op 38.984 ns/op 1.01
byteArrayEquals 123687377 - random bytes 8.1900 ns/op 7.8100 ns/op 1.05
Buffer.compare 123687377 - random bytes 41.540 ns/op 41.340 ns/op 1.00
regular array get 100000 times 42.414 us/op 40.488 us/op 1.05
wrappedArray get 100000 times 43.277 us/op 40.603 us/op 1.07
arrayWithProxy get 100000 times 10.810 ms/op 10.487 ms/op 1.03
ssz.Root.equals 57.774 ns/op 53.621 ns/op 1.08
byteArrayEquals 57.259 ns/op 52.561 ns/op 1.09
Buffer.compare 10.194 ns/op 9.9050 ns/op 1.03
shuffle list - 16384 els 6.1644 ms/op 5.8390 ms/op 1.06
shuffle list - 250000 els 90.272 ms/op 84.936 ms/op 1.06
processSlot - 1 slots 12.128 us/op 15.349 us/op 0.79
processSlot - 32 slots 2.3083 ms/op 2.9212 ms/op 0.79
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 44.768 ms/op 49.851 ms/op 0.90
getCommitteeAssignments - req 1 vs - 250000 vc 2.3303 ms/op 2.3226 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 3.4722 ms/op 3.4564 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 3.8602 ms/op 3.7696 ms/op 1.02
findModifiedValidators - 10000 modified validators 282.77 ms/op 280.47 ms/op 1.01
findModifiedValidators - 1000 modified validators 155.44 ms/op 172.60 ms/op 0.90
findModifiedValidators - 100 modified validators 148.11 ms/op 152.54 ms/op 0.97
findModifiedValidators - 10 modified validators 175.88 ms/op 152.03 ms/op 1.16
findModifiedValidators - 1 modified validators 164.68 ms/op 150.52 ms/op 1.09
findModifiedValidators - no difference 177.21 ms/op 172.64 ms/op 1.03
compare ViewDUs 4.6149 s/op 3.8097 s/op 1.21
compare each validator Uint8Array 1.3974 s/op 1.6436 s/op 0.85
compare ViewDU to Uint8Array 1.0933 s/op 1.1608 s/op 0.94
migrate state 1000000 validators, 24 modified, 0 new 780.67 ms/op 806.22 ms/op 0.97
migrate state 1000000 validators, 1700 modified, 1000 new 1.0144 s/op 966.62 ms/op 1.05
migrate state 1000000 validators, 3400 modified, 2000 new 1.2074 s/op 1.1223 s/op 1.08
migrate state 1500000 validators, 24 modified, 0 new 662.58 ms/op 686.90 ms/op 0.96
migrate state 1500000 validators, 1700 modified, 1000 new 815.92 ms/op 902.06 ms/op 0.90
migrate state 1500000 validators, 3400 modified, 2000 new 1.2643 s/op 1.0471 s/op 1.21
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.1500 ns/op 4.4000 ns/op 1.40
state getBlockRootAtSlot - 250000 vs - 7PWei 691.28 ns/op 545.74 ns/op 1.27
computeProposers - vc 250000 7.0465 ms/op 5.6909 ms/op 1.24
computeEpochShuffling - vc 250000 89.486 ms/op 82.104 ms/op 1.09
getNextSyncCommittee - vc 250000 107.36 ms/op 96.955 ms/op 1.11
computeSigningRoot for AttestationData 28.698 us/op 19.818 us/op 1.45
hash AttestationData serialized data then Buffer.toString(base64) 1.2600 us/op 1.1969 us/op 1.05
toHexString serialized data 772.80 ns/op 823.44 ns/op 0.94
Buffer.toString(base64) 179.51 ns/op 163.35 ns/op 1.10

by benchmarkbot/action

@nazarhussain nazarhussain marked this pull request as draft April 5, 2024 13:03
@nazarhussain
Copy link
Contributor Author

Decided to split this PR to multiple, so temporally converting to draft.

@nazarhussain nazarhussain marked this pull request as ready for review April 8, 2024 08:16
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silly comment but can we add JS to the Web3 portion of the file and function names, or perhaps go back to Eth1Provider naming so its a bit more clear what is happening in here. Makes sense after opening it but looking through the file structure in vscode it would be a bit less clear unless someone understood what it was. While i love file names to match the class they export we dont do that anywhere else in the codebase so we should probably match that and call the file web3JsPluginProvider or web3JsProvider or something similar. Up to you how you want to handle that though.

packages/cli/test/utils/simulation/utils/syncing.ts Outdated Show resolved Hide resolved
Comment on lines +27 to +32
const blob = new Uint8Array(FIELD_ELEMENTS_PER_BLOB * BYTES_PER_FIELD_ELEMENT);
const dv = new DataView(blob.buffer, blob.byteOffset, blob.byteLength);
for (let i = 0; i < FIELD_ELEMENTS_PER_BLOB; i++) {
dv.setUint32(i * BYTES_PER_FIELD_ELEMENT, i);
}
return blob;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just use return Uint8Array.from(crypto.randomBytes(FIELD_ELEMENTS_PER_BLOB * BYTES_PER_FIELD_ELEMENT)) here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe because that will be bigendian and we want the blob to be little-endian.

@@ -18,9 +18,9 @@ export function createExecutionHeadAssertion({
return AssertionMatch.None;
},
async capture({node}) {
const blockNumber = await node.execution.provider?.getBlockNumber();
const blockNumber = await node.execution.provider?.eth.getBlockNumber();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't notice the eth namespace in Web3Plugins.ts. Is that the default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That comes as default with the web3.js

return blobSideCars.response.data.length;
},

assert: async ({store}) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you said there is no reason to assert the blobs are correct but part of me thinks that seems incorrect. How do you know they are not getting included in the wrong block or that the commitments match?

Copy link
Member

@matthewkeil matthewkeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I still tend to believe we need to check that the blobs are the correct ones in the assertion. Other than than that im pretty sure this is all correct but have to rely on the CI run itself and others to double check as well.

@matthewkeil
Copy link
Member

Had a partial review from before you broke out the other PR’s from this one. Some of those comments are still relevant though

@nazarhussain
Copy link
Contributor Author

LGTM. I still tend to believe we need to check that the blobs are the correct ones in the assertion. Other than than that im pretty sure this is all correct but have to rely on the CI run itself and others to double check as well.

I want to cover few more scenarios as well, will do in separate PR. This PR is just converting existing assertions from old tests to sim tests, so that's fine.

@nazarhussain
Copy link
Contributor Author

@matthewkeil I am merging this PR and open small PRs to address remaining comments.

@nazarhussain nazarhussain merged commit 6bd9af9 into unstable Apr 9, 2024
20 checks passed
@nazarhussain nazarhussain deleted the nh/sim-4844 branch April 9, 2024 14:00
nflaig pushed a commit that referenced this pull request Apr 11, 2024
* Remove old 4844 sim test

* Add support for 4844 tests in sim tests

* Fix file path

* Fix merge conflict issue
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.18.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants