allow try-runtime and TestExternalities to report PoV size
#10372
Conversation
kianenigma
added
A0-please_review
github-actions
bot
added
A3-in_progress
kianenigma
changed the title
| let proof = proving_ext.backend.extract_proof(); | ||
|
|
||
| // ensure that all changes are propagated, and the recorded is clean. | ||
| proving_ext.backend.clear_recorder(); |
There was a problem hiding this comment.
as per @cheme's previous feedback, the clear recorded is not really needed here. Will double check and remove.
kianenigma
added
A0-please_review
| @@ -348,4 +393,32 @@ mod tests { | |||
| ext.commit_all().unwrap(); | |||
| assert!(ext.backend.eq(&backend), "Both backend should be equal."); | |||
| } | |||
|
|
|||
| #[test] | |||
| fn execute_and_generate_proof_works() { | |||
There was a problem hiding this comment.
I tried to make this test such that I would make two exts with the same initial data, read 1 key from one and 3 keys from the other one, and expect the proof of the latter to be bigger than the former, but this failed. Any idea why @cheme?
There was a problem hiding this comment.
If values are small, they can be encoded in a same node (inline encoding) and the proof will be the same length.
Using values of length 32 or more is better for testing this kind of things.
There was a problem hiding this comment.
Sounds like a bad test.
Convert the proof back to a db and check that it contains the expected values.
There was a problem hiding this comment.
the backing memory db contains the same data trie nodes of proof, and thei hash, not sure what is there to check. If you want me to improve the test, doing in on the proof itself is enough.
I am looking into checking the proof itself, but it sounds a bit like an overkill. It is not the scope of this test to make sure the proof recorded is doing its job well, just that something is recorded.
Inserting large values (32+ bytes) does help indeed.
There was a problem hiding this comment.
looking a bit into example code, I presume making sure that the proof (as memory-db or as-is) contains the root is a reasonable check to do, aka using create_proof_check_backend.
| assert!(to_delete.len() > 0); | ||
|
|
||
| for d in to_delete { | ||
| use std::os::unix::fs::MetadataExt; |
There was a problem hiding this comment.
What happens with this test on non-unix systems? :P
There was a problem hiding this comment.
Not my problem, these tests don't run on CI either, they are only for me, and I run unix :D
| @@ -348,4 +393,32 @@ mod tests { | |||
| ext.commit_all().unwrap(); | |||
| assert!(ext.backend.eq(&backend), "Both backend should be equal."); | |||
| } | |||
|
|
|||
| #[test] | |||
| fn execute_and_generate_proof_works() { | |||
There was a problem hiding this comment.
Sounds like a bad test.
Convert the proof back to a db and check that it contains the expected values.
| @@ -1019,4 +1023,47 @@ mod tests { | |||
| panic!("Hey, I'm an error"); | |||
| }); | |||
| } | |||
|
|
|||
| #[test] | |||
| fn execute_and_generate_proof_works() { | |||
There was a problem hiding this comment.
Maybe a different crate would make more sense, but I am ok with this one.
kianenigma
changed the title
TestExternalities to report PoV size
|
@bkchr @kianenigma what is the "reasonable" PoV size range in your view? |
It depend on how much storage is being read. I don't think there's any reasonable range. |
| log::info!( | ||
| target: LOG_TARGET, | ||
| "proof: {} / {} nodes", | ||
| HexDisplay::from(&proof_nodes.iter().flatten().cloned().collect::<Vec<_>>()), |
There was a problem hiding this comment.
Isn't that to much to be displayed?
There was a problem hiding this comment.
Display for HexDisplay trims stuff relatively nicely.
| ); | ||
|
|
||
| let mut child_kv = vec![]; | ||
| for prefixed_top_key in child_bearing_top_keys { |
There was a problem hiding this comment.
I got really confused, why don't you just call the child storage roots? :D
There was a problem hiding this comment.
renamed this instance. Generally speaking, the jargon here is super confusing. Kinda hard to specify what node is in what tree. I though child_bearing_top_keys is pretty accurate actually.
|
|
||
| info!(target: LOG_TARGET, "injecting a total of {} top keys", top_kv.len()); | ||
| for (k, v) in top_kv { | ||
| // skip writing the child root data. |
There was a problem hiding this comment.
because it is computed (re)automatically.
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
…to kiz-pov-test-ext
|
bot merge |
|
Waiting for commit status. |
|
bot merge |
…ech#10372) * allow try-runtime and test-externalities to report proof size * self review * fix test * Fix humanized dispaly of bytes * Fix some test * Fix some review grumbles * last of the review comments * fmt * remove unused import * move test * fix import * Update primitives/state-machine/src/testing.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * last touches * fix Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
…ech#10372) * allow try-runtime and test-externalities to report proof size * self review * fix test * Fix humanized dispaly of bytes * Fix some test * Fix some review grumbles * last of the review comments * fmt * remove unused import * move test * fix import * Update primitives/state-machine/src/testing.rs Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * last touches * fix Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
This PR extracts a few features from #10073, cleans them up, and exposes them so that both a normal
TestExternalityandtry-runtimecan use them.Namely, we now allow:
RemoteExternalitiesto fetch the child-tree as well.TestExternalitiesto report its PoV size as well. An example test is provided of how this can be examined. Most often you are interested in zstd compressed version of the compact proof.try-runtimesubcommands, so theoretically you can thoroughly test a new runtime, and its migrations, and ensure that their PoV size is sane, using a combination ofon-runtime-upgradeandfollow-chain.Next to paritytech/cumulus#793. this will be useful for parachain teams and making sure they perform migrations safely.
My only grumble here is that each team needs to integrate try-runtime into their own CLI. Although this is super easy and they just need to wrap one
Commandimplementation offrame_try_runtime_cli, I would have been happier if I could make a standalone tool for this that teams can just work with, only using a.wasmruntime. In my next life, perhaps.skip check-dependent-polkadot
Examples:
Running Polkadot Migrations using Try-runtime
Generic Test: Measuring the PoV of a full runtime upgrade block
This can be either a test in your top-level runtime, or it can simply be a standalone binary that pulls in the correct dependencies.