Custom Benchmark Errors and Override

[shawntabrizi]

This PR allows the benchmarking pipeline to return a custom BenchmarkError error type.

With this new error type, we can allow a user to define a "default" weight to output if a benchmark would fail, or is otherwise unimplemented:

override_benchmark {
	let b in 1 .. 1000;
	let caller = account::<T::AccountId>("caller", 0, 0);
}: {
	Err(BenchmarkError::Override(
		BenchmarkResult {
			extrinsic_time: 1_234_567_890,
			reads: 1337,
			writes: 420,
			..Default::default()
		}
	))?;
}

Any data set here will then be swallowed by the benchmarking pipeline, and be used in the final output.

polkadot companion: paritytech/polkadot#3600

[bkchr]

Why do you skip this here?

[shawntabrizi]

What is the proper fix for?

error[E0277]: the trait bound `&'static str: WrapperTypeDecode` is not satisfied
   --> frame/benchmarking/src/utils.rs:128:7
    |
128 |     Stop(&'static str),
    |          ^ the trait `WrapperTypeDecode` is not implemented for `&'static str`
    | 
   ::: /Users/shawntabrizi/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/parity-scale-codec-2.2.0/src/codec.rs:284:18
    |
284 |     fn decode<I: Input>(input: &mut I) -> Result<Self, Error>;
    |                  ----- required by this bound in `utils::_::_parity_scale_codec::Decode::decode`
    |
    = note: required because of the requirements on the impl of `Decode` for `&'static str`

[bkchr]

Use sp_runtime::RuntimeString

[shawntabrizi]

Runtime string came with a whole host of other issues, again related to Arbitrary Error -> Static String, then Static String -> Runtime String, since it rust can't figure out to do two jumps.

Instead I removed Encode, Decode cause it just wasnt needed

[gui1117]

what is the usecase for this ?

[kianenigma]

what is the usecase for this ?

For XCM, we want to be able to specify a benchmark for all commands and orders, but based on runtime configurations, return custom values (example: return u64::max_value() to signal "don't send this XCM to me since it is not supported"). I think there are similar scenarios as well in substrate (see elections-phragmen)

[apopiak]

LGTM.
Just a little unsure the override should be an error.
(but probably too bikesheddy to be worth delaying the PR for)

[KiChjang]

Why not implement From<Weight> for BenchmarkResult?

[KiChjang]

Looks okay overall; doesn't quite matter if the comment is addressed or not.

[shawntabrizi]

bot merge

[ghost]

Waiting for commit status.