Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add support for web3.js to simulation tests #6641

Merged
merged 7 commits into from
Apr 8, 2024
Merged

Conversation

nazarhussain
Copy link
Contributor

Motivation

Add the support for web3.js to simulation tests.

Description

  • We need to test some scenarios in future which requires to interact with the EL nodes
  • Having custom logic for all interactions will be redundant and error prune
  • Using proven web3.js as ETH RPC client will solve a lot of issues.

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner April 5, 2024 13:47
@nazarhussain nazarhussain self-assigned this Apr 5, 2024
Copy link

codecov bot commented Apr 5, 2024

Codecov Report

Merging #6641 (af9ca28) into unstable (65927c6) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff            @@
##           unstable    #6641   +/-   ##
=========================================
  Coverage     61.69%   61.69%           
=========================================
  Files           556      556           
  Lines         58820    58820           
  Branches       1887     1887           
=========================================
  Hits          36287    36287           
  Misses        22492    22492           
  Partials         41       41           

@nflaig nflaig changed the title feat: add support for web3.js to simulation tests test: add support for web3.js to simulation tests Apr 5, 2024
Copy link
Contributor

github-actions bot commented Apr 5, 2024

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: 943397c Previous: 65927c6 Ratio
Map set x1000 58.002 ns/op 17.652 ns/op 3.29
altair processAttestation - 250000 vs - 7PWei normalcase 4.9451 ms/op 1.5979 ms/op 3.09
altair processEth1Data - 250000 vs - 7PWei normalcase 838.35 us/op 267.90 us/op 3.13
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 20.103 us/op 3.5450 us/op 5.67
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 102.34 us/op 30.893 us/op 3.31
Full benchmark results
Benchmark suite Current: 943397c Previous: 65927c6 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 856.21 us/op 627.21 us/op 1.37
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 99.560 us/op 52.724 us/op 1.89
BLS verify - blst-native 1.4030 ms/op 1.0981 ms/op 1.28
BLS verifyMultipleSignatures 3 - blst-native 2.9143 ms/op 2.3436 ms/op 1.24
BLS verifyMultipleSignatures 8 - blst-native 6.6049 ms/op 5.1807 ms/op 1.27
BLS verifyMultipleSignatures 32 - blst-native 23.318 ms/op 18.565 ms/op 1.26
BLS verifyMultipleSignatures 64 - blst-native 45.435 ms/op 36.253 ms/op 1.25
BLS verifyMultipleSignatures 128 - blst-native 89.190 ms/op 71.694 ms/op 1.24
BLS deserializing 10000 signatures 921.06 ms/op 804.89 ms/op 1.14
BLS deserializing 100000 signatures 9.1848 s/op 8.1518 s/op 1.13
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.4108 ms/op 1.1271 ms/op 1.25
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.5568 ms/op 1.4424 ms/op 1.08
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4680 ms/op 2.2853 ms/op 1.08
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.5328 ms/op 3.3356 ms/op 1.06
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.7752 ms/op 6.5747 ms/op 0.88
BLS aggregatePubkeys 32 - blst-native 26.231 us/op 23.924 us/op 1.10
BLS aggregatePubkeys 128 - blst-native 104.39 us/op 85.914 us/op 1.22
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 63.387 ms/op 47.677 ms/op 1.33
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 67.656 ms/op 49.961 ms/op 1.35
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 47.431 ms/op 27.696 ms/op 1.71
getSlashingsAndExits - default max 244.49 us/op 161.55 us/op 1.51
getSlashingsAndExits - 2k 501.67 us/op 443.96 us/op 1.13
proposeBlockBody type=full, size=empty 6.9020 ms/op 4.5914 ms/op 1.50
isKnown best case - 1 super set check 419.00 ns/op 398.00 ns/op 1.05
isKnown normal case - 2 super set checks 368.00 ns/op 421.00 ns/op 0.87
isKnown worse case - 16 super set checks 370.00 ns/op 405.00 ns/op 0.91
InMemoryCheckpointStateCache - add get delete 6.8310 us/op 4.6020 us/op 1.48
validate api signedAggregateAndProof - struct 2.7962 ms/op 2.4607 ms/op 1.14
validate gossip signedAggregateAndProof - struct 2.8970 ms/op 2.4515 ms/op 1.18
validate gossip attestation - vc 640000 1.4128 ms/op 1.1988 ms/op 1.18
batch validate gossip attestation - vc 640000 - chunk 32 180.04 us/op 158.30 us/op 1.14
batch validate gossip attestation - vc 640000 - chunk 64 164.64 us/op 141.68 us/op 1.16
batch validate gossip attestation - vc 640000 - chunk 128 148.42 us/op 127.15 us/op 1.17
batch validate gossip attestation - vc 640000 - chunk 256 144.61 us/op 115.90 us/op 1.25
pickEth1Vote - no votes 1.4864 ms/op 936.38 us/op 1.59
pickEth1Vote - max votes 12.727 ms/op 7.9667 ms/op 1.60
pickEth1Vote - Eth1Data hashTreeRoot value x2048 26.734 ms/op 14.376 ms/op 1.86
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 30.127 ms/op 18.842 ms/op 1.60
pickEth1Vote - Eth1Data fastSerialize value x2048 863.87 us/op 481.14 us/op 1.80
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.9661 ms/op 4.5691 ms/op 1.74
bytes32 toHexString 708.00 ns/op 459.00 ns/op 1.54
bytes32 Buffer.toString(hex) 314.00 ns/op 307.00 ns/op 1.02
bytes32 Buffer.toString(hex) from Uint8Array 604.00 ns/op 436.00 ns/op 1.39
bytes32 Buffer.toString(hex) + 0x 396.00 ns/op 323.00 ns/op 1.23
Object access 1 prop 0.23400 ns/op 0.20400 ns/op 1.15
Map access 1 prop 0.16600 ns/op 0.19400 ns/op 0.86
Object get x1000 8.3470 ns/op 5.5740 ns/op 1.50
Map get x1000 0.91000 ns/op 0.76300 ns/op 1.19
Object set x1000 68.139 ns/op 25.135 ns/op 2.71
Map set x1000 58.002 ns/op 17.652 ns/op 3.29
Return object 10000 times 0.31480 ns/op 0.23530 ns/op 1.34
Throw Error 10000 times 4.4792 us/op 2.8088 us/op 1.59
fastMsgIdFn sha256 / 200 bytes 3.9730 us/op 2.0080 us/op 1.98
fastMsgIdFn h32 xxhash / 200 bytes 399.00 ns/op 303.00 ns/op 1.32
fastMsgIdFn h64 xxhash / 200 bytes 379.00 ns/op 365.00 ns/op 1.04
fastMsgIdFn sha256 / 1000 bytes 12.208 us/op 6.3400 us/op 1.93
fastMsgIdFn h32 xxhash / 1000 bytes 520.00 ns/op 433.00 ns/op 1.20
fastMsgIdFn h64 xxhash / 1000 bytes 533.00 ns/op 432.00 ns/op 1.23
fastMsgIdFn sha256 / 10000 bytes 119.42 us/op 53.854 us/op 2.22
fastMsgIdFn h32 xxhash / 10000 bytes 2.1780 us/op 1.8870 us/op 1.15
fastMsgIdFn h64 xxhash / 10000 bytes 1.5170 us/op 1.3090 us/op 1.16
send data - 1000 256B messages 22.460 ms/op 12.103 ms/op 1.86
send data - 1000 512B messages 32.572 ms/op 15.396 ms/op 2.12
send data - 1000 1024B messages 45.246 ms/op 26.786 ms/op 1.69
send data - 1000 1200B messages 27.778 ms/op 29.708 ms/op 0.94
send data - 1000 2048B messages 52.076 ms/op 36.774 ms/op 1.42
send data - 1000 4096B messages 48.299 ms/op 32.875 ms/op 1.47
send data - 1000 16384B messages 138.69 ms/op 83.741 ms/op 1.66
send data - 1000 65536B messages 535.22 ms/op 381.21 ms/op 1.40
enrSubnets - fastDeserialize 64 bits 1.8270 us/op 1.0530 us/op 1.74
enrSubnets - ssz BitVector 64 bits 608.00 ns/op 421.00 ns/op 1.44
enrSubnets - fastDeserialize 4 bits 263.00 ns/op 198.00 ns/op 1.33
enrSubnets - ssz BitVector 4 bits 613.00 ns/op 418.00 ns/op 1.47
prioritizePeers score -10:0 att 32-0.1 sync 2-0 122.74 us/op 64.341 us/op 1.91
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 152.05 us/op 81.577 us/op 1.86
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 221.67 us/op 110.38 us/op 2.01
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 378.81 us/op 182.43 us/op 2.08
prioritizePeers score 0:0 att 64-1 sync 4-1 422.02 us/op 201.20 us/op 2.10
array of 16000 items push then shift 1.8425 us/op 1.2241 us/op 1.51
LinkedList of 16000 items push then shift 11.294 ns/op 6.6740 ns/op 1.69
array of 16000 items push then pop 128.68 ns/op 63.924 ns/op 2.01
LinkedList of 16000 items push then pop 10.716 ns/op 5.5940 ns/op 1.92
array of 24000 items push then shift 2.6910 us/op 1.8556 us/op 1.45
LinkedList of 24000 items push then shift 11.107 ns/op 6.3310 ns/op 1.75
array of 24000 items push then pop 180.58 ns/op 91.359 ns/op 1.98
LinkedList of 24000 items push then pop 10.369 ns/op 5.4800 ns/op 1.89
intersect bitArray bitLen 8 6.4060 ns/op 4.4120 ns/op 1.45
intersect array and set length 8 76.606 ns/op 45.416 ns/op 1.69
intersect bitArray bitLen 128 37.571 ns/op 27.831 ns/op 1.35
intersect array and set length 128 927.54 ns/op 640.17 ns/op 1.45
bitArray.getTrueBitIndexes() bitLen 128 1.7330 us/op 1.2540 us/op 1.38
bitArray.getTrueBitIndexes() bitLen 248 3.2750 us/op 2.0270 us/op 1.62
bitArray.getTrueBitIndexes() bitLen 512 6.4050 us/op 3.8350 us/op 1.67
Buffer.concat 32 items 1.4660 us/op 804.00 ns/op 1.82
Uint8Array.set 32 items 2.2690 us/op 1.7020 us/op 1.33
Set add up to 64 items then delete first 5.1823 us/op 1.7324 us/op 2.99
OrderedSet add up to 64 items then delete first 6.9681 us/op 2.6818 us/op 2.60
Set add up to 64 items then delete last 5.6605 us/op 1.9877 us/op 2.85
OrderedSet add up to 64 items then delete last 7.1099 us/op 2.9536 us/op 2.41
Set add up to 64 items then delete middle 5.5234 us/op 1.9893 us/op 2.78
OrderedSet add up to 64 items then delete middle 8.9807 us/op 4.1535 us/op 2.16
Set add up to 128 items then delete first 11.742 us/op 3.9718 us/op 2.96
OrderedSet add up to 128 items then delete first 16.332 us/op 6.1361 us/op 2.66
Set add up to 128 items then delete last 11.338 us/op 3.8273 us/op 2.96
OrderedSet add up to 128 items then delete last 14.930 us/op 5.6706 us/op 2.63
Set add up to 128 items then delete middle 11.260 us/op 3.7961 us/op 2.97
OrderedSet add up to 128 items then delete middle 24.039 us/op 10.674 us/op 2.25
Set add up to 256 items then delete first 22.983 us/op 7.6738 us/op 3.00
OrderedSet add up to 256 items then delete first 31.497 us/op 12.209 us/op 2.58
Set add up to 256 items then delete last 21.343 us/op 7.4449 us/op 2.87
OrderedSet add up to 256 items then delete last 27.757 us/op 11.296 us/op 2.46
Set add up to 256 items then delete middle 21.911 us/op 7.4236 us/op 2.95
OrderedSet add up to 256 items then delete middle 54.929 us/op 30.767 us/op 1.79
transfer serialized Status (84 B) 2.1050 us/op 1.4070 us/op 1.50
copy serialized Status (84 B) 1.5630 us/op 1.1300 us/op 1.38
transfer serialized SignedVoluntaryExit (112 B) 2.2450 us/op 1.5360 us/op 1.46
copy serialized SignedVoluntaryExit (112 B) 1.5980 us/op 1.1990 us/op 1.33
transfer serialized ProposerSlashing (416 B) 2.6260 us/op 2.0300 us/op 1.29
copy serialized ProposerSlashing (416 B) 2.4460 us/op 1.9290 us/op 1.27
transfer serialized Attestation (485 B) 2.4450 us/op 2.4910 us/op 0.98
copy serialized Attestation (485 B) 2.2090 us/op 1.9890 us/op 1.11
transfer serialized AttesterSlashing (33232 B) 2.9040 us/op 2.6960 us/op 1.08
copy serialized AttesterSlashing (33232 B) 9.5230 us/op 4.2570 us/op 2.24
transfer serialized Small SignedBeaconBlock (128000 B) 3.5400 us/op 2.6450 us/op 1.34
copy serialized Small SignedBeaconBlock (128000 B) 28.129 us/op 9.0720 us/op 3.10
transfer serialized Avg SignedBeaconBlock (200000 B) 4.5310 us/op 2.5440 us/op 1.78
copy serialized Avg SignedBeaconBlock (200000 B) 49.778 us/op 12.661 us/op 3.93
transfer serialized BlobsSidecar (524380 B) 5.4050 us/op 3.0400 us/op 1.78
copy serialized BlobsSidecar (524380 B) 155.49 us/op 70.538 us/op 2.20
transfer serialized Big SignedBeaconBlock (1000000 B) 5.1260 us/op 3.3050 us/op 1.55
copy serialized Big SignedBeaconBlock (1000000 B) 265.13 us/op 133.64 us/op 1.98
pass gossip attestations to forkchoice per slot 5.2221 ms/op 2.6196 ms/op 1.99
forkChoice updateHead vc 100000 bc 64 eq 0 789.50 us/op 415.70 us/op 1.90
forkChoice updateHead vc 600000 bc 64 eq 0 6.3023 ms/op 2.7159 ms/op 2.32
forkChoice updateHead vc 1000000 bc 64 eq 0 9.0477 ms/op 4.2181 ms/op 2.14
forkChoice updateHead vc 600000 bc 320 eq 0 5.0273 ms/op 2.5639 ms/op 1.96
forkChoice updateHead vc 600000 bc 1200 eq 0 5.3707 ms/op 2.5952 ms/op 2.07
forkChoice updateHead vc 600000 bc 7200 eq 0 6.3032 ms/op 3.2708 ms/op 1.93
forkChoice updateHead vc 600000 bc 64 eq 1000 13.081 ms/op 9.9033 ms/op 1.32
forkChoice updateHead vc 600000 bc 64 eq 10000 13.651 ms/op 9.7852 ms/op 1.40
forkChoice updateHead vc 600000 bc 64 eq 300000 26.537 ms/op 11.889 ms/op 2.23
computeDeltas 500000 validators 300 proto nodes 8.0828 ms/op 3.0280 ms/op 2.67
computeDeltas 500000 validators 1200 proto nodes 8.0669 ms/op 2.8487 ms/op 2.83
computeDeltas 500000 validators 7200 proto nodes 7.6076 ms/op 2.8616 ms/op 2.66
computeDeltas 750000 validators 300 proto nodes 11.309 ms/op 4.3352 ms/op 2.61
computeDeltas 750000 validators 1200 proto nodes 11.995 ms/op 4.3885 ms/op 2.73
computeDeltas 750000 validators 7200 proto nodes 12.014 ms/op 4.3698 ms/op 2.75
computeDeltas 1400000 validators 300 proto nodes 23.066 ms/op 9.1265 ms/op 2.53
computeDeltas 1400000 validators 1200 proto nodes 22.702 ms/op 8.8408 ms/op 2.57
computeDeltas 1400000 validators 7200 proto nodes 23.860 ms/op 8.9838 ms/op 2.66
computeDeltas 2100000 validators 300 proto nodes 35.775 ms/op 13.738 ms/op 2.60
computeDeltas 2100000 validators 1200 proto nodes 33.441 ms/op 13.104 ms/op 2.55
computeDeltas 2100000 validators 7200 proto nodes 34.061 ms/op 13.057 ms/op 2.61
altair processAttestation - 250000 vs - 7PWei normalcase 4.9451 ms/op 1.5979 ms/op 3.09
altair processAttestation - 250000 vs - 7PWei worstcase 6.4303 ms/op 2.3178 ms/op 2.77
altair processAttestation - setStatus - 1/6 committees join 264.40 us/op 102.57 us/op 2.58
altair processAttestation - setStatus - 1/3 committees join 432.68 us/op 201.31 us/op 2.15
altair processAttestation - setStatus - 1/2 committees join 544.62 us/op 284.35 us/op 1.92
altair processAttestation - setStatus - 2/3 committees join 706.24 us/op 376.00 us/op 1.88
altair processAttestation - setStatus - 4/5 committees join 994.22 us/op 501.34 us/op 1.98
altair processAttestation - setStatus - 100% committees join 1.2310 ms/op 557.44 us/op 2.21
altair processBlock - 250000 vs - 7PWei normalcase 9.9851 ms/op 6.8625 ms/op 1.46
altair processBlock - 250000 vs - 7PWei normalcase hashState 52.862 ms/op 28.261 ms/op 1.87
altair processBlock - 250000 vs - 7PWei worstcase 49.166 ms/op 33.239 ms/op 1.48
altair processBlock - 250000 vs - 7PWei worstcase hashState 129.14 ms/op 82.467 ms/op 1.57
phase0 processBlock - 250000 vs - 7PWei normalcase 3.5858 ms/op 2.2847 ms/op 1.57
phase0 processBlock - 250000 vs - 7PWei worstcase 40.394 ms/op 23.524 ms/op 1.72
altair processEth1Data - 250000 vs - 7PWei normalcase 838.35 us/op 267.90 us/op 3.13
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 20.103 us/op 3.5450 us/op 5.67
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 102.34 us/op 30.893 us/op 3.31
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 34.215 us/op 14.036 us/op 2.44
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 13.214 us/op 11.690 us/op 1.13
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 236.75 us/op 162.20 us/op 1.46
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.8511 ms/op 1.0509 ms/op 1.76
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.2907 ms/op 1.4481 ms/op 1.58
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.1463 ms/op 1.3716 ms/op 1.56
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 5.0008 ms/op 2.0847 ms/op 2.40
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 3.1969 ms/op 1.5561 ms/op 2.05
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 7.4512 ms/op 3.7570 ms/op 1.98
Tree 40 250000 create 693.91 ms/op 272.35 ms/op 2.55
Tree 40 250000 get(125000) 230.24 ns/op 112.79 ns/op 2.04
Tree 40 250000 set(125000) 1.7260 us/op 745.81 ns/op 2.31
Tree 40 250000 toArray() 27.489 ms/op 9.6558 ms/op 2.85
Tree 40 250000 iterate all - toArray() + loop 29.218 ms/op 9.7725 ms/op 2.99
Tree 40 250000 iterate all - get(i) 85.965 ms/op 38.406 ms/op 2.24
MutableVector 250000 create 21.959 ms/op 11.826 ms/op 1.86
MutableVector 250000 get(125000) 7.5080 ns/op 5.3960 ns/op 1.39
MutableVector 250000 set(125000) 415.34 ns/op 198.22 ns/op 2.10
MutableVector 250000 toArray() 4.4695 ms/op 2.4298 ms/op 1.84
MutableVector 250000 iterate all - toArray() + loop 4.7450 ms/op 2.5369 ms/op 1.87
MutableVector 250000 iterate all - get(i) 1.7343 ms/op 1.2656 ms/op 1.37
Array 250000 create 4.5089 ms/op 2.0940 ms/op 2.15
Array 250000 clone - spread 1.7392 ms/op 1.2352 ms/op 1.41
Array 250000 get(125000) 2.4430 ns/op 1.0810 ns/op 2.26
Array 250000 set(125000) 6.1120 ns/op 1.2850 ns/op 4.76
Array 250000 iterate all - loop 200.22 us/op 153.35 us/op 1.31
effectiveBalanceIncrements clone Uint8Array 300000 72.737 us/op 15.277 us/op 4.76
effectiveBalanceIncrements clone MutableVector 300000 1.5610 us/op 460.00 ns/op 3.39
effectiveBalanceIncrements rw all Uint8Array 300000 225.64 us/op 183.50 us/op 1.23
effectiveBalanceIncrements rw all MutableVector 300000 157.44 ms/op 67.841 ms/op 2.32
phase0 afterProcessEpoch - 250000 vs - 7PWei 128.23 ms/op 72.473 ms/op 1.77
phase0 beforeProcessEpoch - 250000 vs - 7PWei 64.076 ms/op 37.120 ms/op 1.73
altair processEpoch - mainnet_e81889 629.74 ms/op 412.66 ms/op 1.53
mainnet_e81889 - altair beforeProcessEpoch 113.49 ms/op 64.647 ms/op 1.76
mainnet_e81889 - altair processJustificationAndFinalization 38.624 us/op 6.7260 us/op 5.74
mainnet_e81889 - altair processInactivityUpdates 10.442 ms/op 4.6855 ms/op 2.23
mainnet_e81889 - altair processRewardsAndPenalties 73.865 ms/op 73.016 ms/op 1.01
mainnet_e81889 - altair processRegistryUpdates 4.4040 us/op 2.0870 us/op 2.11
mainnet_e81889 - altair processSlashings 1.1330 us/op 645.00 ns/op 1.76
mainnet_e81889 - altair processEth1DataReset 1.5760 us/op 692.00 ns/op 2.28
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.7498 ms/op 1.6411 ms/op 1.07
mainnet_e81889 - altair processSlashingsReset 6.9070 us/op 2.4710 us/op 2.80
mainnet_e81889 - altair processRandaoMixesReset 17.048 us/op 2.7560 us/op 6.19
mainnet_e81889 - altair processHistoricalRootsUpdate 1.7530 us/op 638.00 ns/op 2.75
mainnet_e81889 - altair processParticipationFlagUpdates 5.4060 us/op 1.4870 us/op 3.64
mainnet_e81889 - altair processSyncCommitteeUpdates 1.2540 us/op 657.00 ns/op 1.91
mainnet_e81889 - altair afterProcessEpoch 141.97 ms/op 80.077 ms/op 1.77
capella processEpoch - mainnet_e217614 2.1616 s/op 1.6515 s/op 1.31
mainnet_e217614 - capella beforeProcessEpoch 489.22 ms/op 389.68 ms/op 1.26
mainnet_e217614 - capella processJustificationAndFinalization 14.675 us/op 10.729 us/op 1.37
mainnet_e217614 - capella processInactivityUpdates 18.439 ms/op 15.139 ms/op 1.22
mainnet_e217614 - capella processRewardsAndPenalties 499.83 ms/op 450.04 ms/op 1.11
mainnet_e217614 - capella processRegistryUpdates 21.437 us/op 15.221 us/op 1.41
mainnet_e217614 - capella processSlashings 460.00 ns/op 675.00 ns/op 0.68
mainnet_e217614 - capella processEth1DataReset 464.00 ns/op 551.00 ns/op 0.84
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.4059 ms/op 4.3070 ms/op 1.02
mainnet_e217614 - capella processSlashingsReset 3.5280 us/op 1.4520 us/op 2.43
mainnet_e217614 - capella processRandaoMixesReset 4.7630 us/op 1.8480 us/op 2.58
mainnet_e217614 - capella processHistoricalRootsUpdate 547.00 ns/op 475.00 ns/op 1.15
mainnet_e217614 - capella processParticipationFlagUpdates 1.4500 us/op 768.00 ns/op 1.89
mainnet_e217614 - capella afterProcessEpoch 308.12 ms/op 234.59 ms/op 1.31
phase0 processEpoch - mainnet_e58758 429.32 ms/op 404.99 ms/op 1.06
mainnet_e58758 - phase0 beforeProcessEpoch 137.72 ms/op 99.742 ms/op 1.38
mainnet_e58758 - phase0 processJustificationAndFinalization 16.092 us/op 8.6810 us/op 1.85
mainnet_e58758 - phase0 processRewardsAndPenalties 62.985 ms/op 63.211 ms/op 1.00
mainnet_e58758 - phase0 processRegistryUpdates 10.324 us/op 10.079 us/op 1.02
mainnet_e58758 - phase0 processSlashings 513.00 ns/op 704.00 ns/op 0.73
mainnet_e58758 - phase0 processEth1DataReset 907.00 ns/op 598.00 ns/op 1.52
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0967 ms/op 1.3495 ms/op 0.81
mainnet_e58758 - phase0 processSlashingsReset 3.1000 us/op 2.6880 us/op 1.15
mainnet_e58758 - phase0 processRandaoMixesReset 5.3440 us/op 2.9380 us/op 1.82
mainnet_e58758 - phase0 processHistoricalRootsUpdate 602.00 ns/op 606.00 ns/op 0.99
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.2200 us/op 2.9170 us/op 1.45
mainnet_e58758 - phase0 afterProcessEpoch 105.83 ms/op 66.995 ms/op 1.58
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.4228 ms/op 938.87 us/op 1.52
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5306 ms/op 1.0607 ms/op 1.44
altair processInactivityUpdates - 250000 normalcase 25.081 ms/op 24.855 ms/op 1.01
altair processInactivityUpdates - 250000 worstcase 27.271 ms/op 24.400 ms/op 1.12
phase0 processRegistryUpdates - 250000 normalcase 10.025 us/op 6.5080 us/op 1.54
phase0 processRegistryUpdates - 250000 badcase_full_deposits 387.08 us/op 213.89 us/op 1.81
phase0 processRegistryUpdates - 250000 worstcase 0.5 135.22 ms/op 120.99 ms/op 1.12
altair processRewardsAndPenalties - 250000 normalcase 57.032 ms/op 70.843 ms/op 0.81
altair processRewardsAndPenalties - 250000 worstcase 65.156 ms/op 67.671 ms/op 0.96
phase0 getAttestationDeltas - 250000 normalcase 11.868 ms/op 5.5798 ms/op 2.13
phase0 getAttestationDeltas - 250000 worstcase 11.716 ms/op 5.6775 ms/op 2.06
phase0 processSlashings - 250000 worstcase 120.77 us/op 75.182 us/op 1.61
altair processSyncCommitteeUpdates - 250000 174.63 ms/op 109.88 ms/op 1.59
BeaconState.hashTreeRoot - No change 563.00 ns/op 361.00 ns/op 1.56
BeaconState.hashTreeRoot - 1 full validator 139.93 us/op 151.63 us/op 0.92
BeaconState.hashTreeRoot - 32 full validator 1.6008 ms/op 1.5924 ms/op 1.01
BeaconState.hashTreeRoot - 512 full validator 15.048 ms/op 16.425 ms/op 0.92
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 200.87 us/op 153.51 us/op 1.31
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.5993 ms/op 1.7795 ms/op 1.46
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 34.264 ms/op 27.816 ms/op 1.23
BeaconState.hashTreeRoot - 1 balances 148.78 us/op 115.97 us/op 1.28
BeaconState.hashTreeRoot - 32 balances 1.6449 ms/op 1.1443 ms/op 1.44
BeaconState.hashTreeRoot - 512 balances 11.967 ms/op 10.560 ms/op 1.13
BeaconState.hashTreeRoot - 250000 balances 214.21 ms/op 173.84 ms/op 1.23
aggregationBits - 2048 els - zipIndexesInBitList 42.216 us/op 19.081 us/op 2.21
byteArrayEquals 32 85.430 ns/op 64.485 ns/op 1.32
Buffer.compare 32 62.535 ns/op 39.294 ns/op 1.59
byteArrayEquals 1024 2.1991 us/op 1.7520 us/op 1.26
Buffer.compare 1024 74.976 ns/op 47.440 ns/op 1.58
byteArrayEquals 16384 35.680 us/op 27.801 us/op 1.28
Buffer.compare 16384 276.66 ns/op 230.18 ns/op 1.20
byteArrayEquals 123687377 271.56 ms/op 210.53 ms/op 1.29
Buffer.compare 123687377 10.204 ms/op 3.8619 ms/op 2.64
byteArrayEquals 32 - diff last byte 85.877 ns/op 61.922 ns/op 1.39
Buffer.compare 32 - diff last byte 68.957 ns/op 39.059 ns/op 1.77
byteArrayEquals 1024 - diff last byte 2.3783 us/op 1.5624 us/op 1.52
Buffer.compare 1024 - diff last byte 78.822 ns/op 43.371 ns/op 1.82
byteArrayEquals 16384 - diff last byte 35.130 us/op 27.469 us/op 1.28
Buffer.compare 16384 - diff last byte 277.44 ns/op 223.42 ns/op 1.24
byteArrayEquals 123687377 - diff last byte 284.09 ms/op 205.52 ms/op 1.38
Buffer.compare 123687377 - diff last byte 11.230 ms/op 3.7580 ms/op 2.99
byteArrayEquals 32 - random bytes 8.2520 ns/op 4.4240 ns/op 1.87
Buffer.compare 32 - random bytes 67.346 ns/op 39.873 ns/op 1.69
byteArrayEquals 1024 - random bytes 7.0420 ns/op 4.3960 ns/op 1.60
Buffer.compare 1024 - random bytes 66.324 ns/op 38.350 ns/op 1.73
byteArrayEquals 16384 - random bytes 6.8200 ns/op 4.3910 ns/op 1.55
Buffer.compare 16384 - random bytes 67.637 ns/op 38.984 ns/op 1.73
byteArrayEquals 123687377 - random bytes 9.8100 ns/op 7.8100 ns/op 1.26
Buffer.compare 123687377 - random bytes 77.080 ns/op 41.340 ns/op 1.86
regular array get 100000 times 50.153 us/op 40.488 us/op 1.24
wrappedArray get 100000 times 48.865 us/op 40.603 us/op 1.20
arrayWithProxy get 100000 times 14.943 ms/op 10.487 ms/op 1.42
ssz.Root.equals 59.754 ns/op 53.621 ns/op 1.11
byteArrayEquals 57.407 ns/op 52.561 ns/op 1.09
Buffer.compare 12.519 ns/op 9.9050 ns/op 1.26
shuffle list - 16384 els 8.8072 ms/op 5.8390 ms/op 1.51
shuffle list - 250000 els 130.28 ms/op 84.936 ms/op 1.53
processSlot - 1 slots 20.707 us/op 15.349 us/op 1.35
processSlot - 32 slots 4.0900 ms/op 2.9212 ms/op 1.40
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 64.658 ms/op 49.851 ms/op 1.30
getCommitteeAssignments - req 1 vs - 250000 vc 3.0289 ms/op 2.3226 ms/op 1.30
getCommitteeAssignments - req 100 vs - 250000 vc 4.3918 ms/op 3.4564 ms/op 1.27
getCommitteeAssignments - req 1000 vs - 250000 vc 4.8404 ms/op 3.7696 ms/op 1.28
findModifiedValidators - 10000 modified validators 430.54 ms/op 280.47 ms/op 1.54
findModifiedValidators - 1000 modified validators 243.90 ms/op 172.60 ms/op 1.41
findModifiedValidators - 100 modified validators 204.03 ms/op 152.54 ms/op 1.34
findModifiedValidators - 10 modified validators 232.21 ms/op 152.03 ms/op 1.53
findModifiedValidators - 1 modified validators 244.55 ms/op 150.52 ms/op 1.62
findModifiedValidators - no difference 220.96 ms/op 172.64 ms/op 1.28
compare ViewDUs 5.0068 s/op 3.8097 s/op 1.31
compare each validator Uint8Array 1.7749 s/op 1.6436 s/op 1.08
compare ViewDU to Uint8Array 1.4058 s/op 1.1608 s/op 1.21
migrate state 1000000 validators, 24 modified, 0 new 729.00 ms/op 806.22 ms/op 0.90
migrate state 1000000 validators, 1700 modified, 1000 new 1.1021 s/op 966.62 ms/op 1.14
migrate state 1000000 validators, 3400 modified, 2000 new 1.2502 s/op 1.1223 s/op 1.11
migrate state 1500000 validators, 24 modified, 0 new 731.60 ms/op 686.90 ms/op 1.07
migrate state 1500000 validators, 1700 modified, 1000 new 1.1017 s/op 902.06 ms/op 1.22
migrate state 1500000 validators, 3400 modified, 2000 new 1.6326 s/op 1.0471 s/op 1.56
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.2000 ns/op 4.4000 ns/op 1.41
state getBlockRootAtSlot - 250000 vs - 7PWei 816.19 ns/op 545.74 ns/op 1.50
computeProposers - vc 250000 10.809 ms/op 5.6909 ms/op 1.90
computeEpochShuffling - vc 250000 141.97 ms/op 82.104 ms/op 1.73
getNextSyncCommittee - vc 250000 173.22 ms/op 96.955 ms/op 1.79
computeSigningRoot for AttestationData 36.059 us/op 19.818 us/op 1.82
hash AttestationData serialized data then Buffer.toString(base64) 2.6490 us/op 1.1969 us/op 2.21
toHexString serialized data 1.7210 us/op 823.44 ns/op 2.09
Buffer.toString(base64) 310.12 ns/op 163.35 ns/op 1.90

by benchmarkbot/action

Copy link
Member

@matthewkeil matthewkeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice addition! Just a couple comments on naming.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know the simulation rig has file names in PascalCase but not sure that is the right way to go. Can we name this file web3JsPlugins.ts

Copy link
Contributor Author

@nazarhussain nazarhussain Apr 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know the simulation rig has file names in PascalCase but not sure that is the right way to go. Can we name this file web3JsPlugins.ts

Let's do the renaming of all files for consistent pattern in different PR.

packages/cli/test/utils/simulation/Web3Plugins.ts Outdated Show resolved Hide resolved
packages/cli/test/utils/simulation/Web3Plugins.ts Outdated Show resolved Hide resolved
nazarhussain and others added 2 commits April 5, 2024 17:24
Co-authored-by: Matthew Keil <me@matthewkeil.com>
Copy link
Member

@matthewkeil matthewkeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!! 🚀

@nazarhussain nazarhussain enabled auto-merge (squash) April 8, 2024 07:53
@nazarhussain nazarhussain merged commit b1f4e7e into unstable Apr 8, 2024
17 of 20 checks passed
@nazarhussain nazarhussain deleted the nh/sim-web3 branch April 8, 2024 07:56
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.18.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants