Add PoV Tracking to Benchmarking Pipeline

[shawntabrizi]

This PR introduces the PoV Tracking / Proof Recorder into the Runtime Benchmarking Pipeline.

In summary, when a parachain presents a new block for the relay chain to validate, it must also provide a minimal PoV block which contains all the necessary storage items which can be used to prove the validity of the state transition.

This means if you are trying to transfer a balance between two accounts on a parachain, you need to provide some subsection of the full state trie to the relay chain which contains those two account balances.

Due to the base-16 Patricia-Merkle trie used in Substrate, this can end up being a lot of extra data. We need to not only confirm that a block of extrinsics would execute in time, but also that the PoV block needed for all those extrinsics would fit within the limits of the relay chain.

When benchmarking for PoV block size, you need to ensure to follow the path of any extrinsic logic which touches the most storage items, especially those that are large, as this is likely to create the largest PoV block. Furthermore, you must populate the storage to have all of those storage items as full as they can be. Always benchmark the worst case scenario.

Initial tests from this PR show:

Pallet: "pallet_balances", Extrinsic: "transfer",
extrinsic_time,storage_root_time,reads,repeat_reads,writes,repeat_writes,proof_size
65000,13000,1,18,1,4,1806

Pallet: "pallet_balances", Extrinsic: "transfer_keep_alive",
extrinsic_time,storage_root_time,reads,repeat_reads,writes,repeat_writes,proof_size
48000,4000,1,12,1,3,1571

So a balance transfer on an empty chain is around 1.6 - 1.8 kB.

Another benchmark which populates more data into the state:

Pallet: "pallet_collective", Extrinsic: "close_approved",
b,m,p,extrinsic_time,storage_root_time,reads,repeat_reads,writes,repeat_writes,proof_size
1,100,100,126000,3000,5,10,3,2,12554
103,100,100,148000,4000,5,10,3,2,12801
205,100,100,144000,4000,5,10,3,2,12761
307,100,100,113000,2000,5,10,3,2,12863
409,100,100,121000,3000,5,10,3,2,12965
511,100,100,121000,3000,5,10,3,2,12935
613,100,100,119000,3000,5,10,3,2,13103
715,100,100,124000,3000,5,10,3,2,13073
817,100,100,125000,4000,5,10,3,2,13713
919,100,100,125000,3000,5,10,3,2,13551
1021,100,100,126000,3000,5,10,3,2,13709
1024,4,100,83000,3000,5,10,3,2,7368
1024,13,100,88000,3000,5,10,3,2,7944
1024,22,100,214000,3000,5,10,3,2,8520
1024,31,100,106000,17000,5,10,3,2,9096
1024,40,100,96000,3000,5,10,3,2,9672
1024,49,100,99000,3000,5,10,3,2,10248
1024,58,100,100000,3000,5,10,3,2,10824
1024,67,100,106000,3000,5,10,3,2,11402
1024,76,100,111000,3000,5,10,3,2,11978
1024,85,100,113000,3000,5,10,3,2,12554
1024,94,100,115000,3000,5,10,3,2,13130
1024,100,1,88000,3000,5,10,3,2,8941
1024,100,10,108000,3000,5,10,3,2,9835
1024,100,19,97000,3000,5,10,3,2,10453
1024,100,28,110000,3000,5,10,3,2,10945
1024,100,37,104000,3000,5,10,3,2,11299
1024,100,46,113000,4000,5,10,3,2,11653
1024,100,55,115000,3000,5,10,3,2,12281
1024,100,64,117000,3000,5,10,3,2,12296
1024,100,73,153000,3000,5,10,3,2,12782
1024,100,82,115000,3000,5,10,3,2,12872
1024,100,91,119000,3000,5,10,3,2,13226
1024,100,100,122000,3000,5,10,3,2,13514

Here we see the PoV size of a closed_approve where we change the number of:

• b: bytes in the proposal
• m: members in the collective
• p: number of proposals in the system

Here we can see the proof size increases as the values of these things increases. Up to nearly 14 kB.

More analysis will be done to verify all of this benchmarking process is sane, and we will introduce processes in future PRs that will fill up the state for worst case PoV benchmarking.

[shawntabrizi]

bot merge

[ghost]

Error: Missing process info; check that the PR belongs to a project column.

Merge cannot succeed as it is. Check out the criteria for merge.

[athei]

So we can benchmark PoV impact but we don't have any way to track and limit it within FRAME like for weight, right?