contracts: Add automated weights for wasm instructions

[athei]

This PR is the last piece of having a completely automatic weight determination for contracts that is able to generate secure weights. After adding weights for extrinsic and contract callable host functions this adds the weights for each supported instruction.

It also adds some additional static checks for contracts that are necessary to price all instructions securely:

• Number of globals
• Number of parameters a function can have
• Number of entries in the immediate value of BrTable

The resulting Schedule as benchmarked by the bot for the substrate runtime looks as follows:

Schedule (click to open)

Schedule {
    version: 0,
    enable_println: false,
    limits: Limits {
        event_topics: 4,
        stack_height: 512,
        globals: 256,
        parameters: 128,
        memory_pages: 16,
        table_size: 4096,
        br_table_size: 256,
        subject_len: 32,
        code_size: 524288,
    },
    instruction_weights: InstructionWeights {
        i64const: "1.6 ns",
        i64load: "157.3 ns",
        i64store: "226.3 ns",
        select: "6.1 ns",
        r#if: "7.9 ns",
        br: "3.0 ns",
        br_if: "6.3 ns",
        br_table: "14.6 ns",
        br_table_per_entry: "103 ps",
        call: "95.1 ns",
        call_indirect: "199.5 ns",
        call_indirect_per_param: "2.1 ns",
        local_get: "1.8 ns",
        local_set: "2.1 ns",
        local_tee: "2.1 ns",
        global_get: "6.6 ns",
        global_set: "10.7 ns",
        memory_current: "2.3 ns",
        memory_grow: "143.5 µs",
        i64clz: "2.3 ns",
        i64ctz: "2.3 ns",
        i64popcnt: "2.9 ns",
        i64eqz: "2.4 ns",
        i64extendsi32: "2.2 ns",
        i64extendui32: "2.2 ns",
        i32wrapi64: "2.3 ns",
        i64eq: "2.6 ns",
        i64ne: "2.5 ns",
        i64lts: "2.5 ns",
        i64ltu: "2.6 ns",
        i64gts: "2.5 ns",
        i64gtu: "2.5 ns",
        i64les: "2.5 ns",
        i64leu: "2.6 ns",
        i64ges: "2.6 ns",
        i64geu: "2.5 ns",
        i64add: "2.5 ns",
        i64sub: "2.6 ns",
        i64mul: "2.5 ns",
        i64divs: "8.2 ns",
        i64divu: "7.2 ns",
        i64rems: "8.1 ns",
        i64remu: "7.3 ns",
        i64and: "2.5 ns",
        i64or: "2.5 ns",
        i64xor: "2.6 ns",
        i64shl: "2.5 ns",
        i64shrs: "2.6 ns",
        i64shru: "2.6 ns",
        i64rotl: "2.5 ns",
        i64rotr: "2.6 ns",
    },
    host_fn_weights: HostFnWeights {
        caller: "3.7 µs",
        address: "3.7 µs",
        gas_left: "3.6 µs",
        balance: "8.1 µs",
        value_transferred: "3.6 µs",
        minimum_balance: "3.6 µs",
        tombstone_deposit: "3.6 µs",
        rent_allowance: "8.5 µs",
        block_number: "3.6 µs",
        now: "3.7 µs",
        weight_to_fee: "6.1 µs",
        gas: "1.9 µs",
        input: "7.8 µs",
        input_per_byte: "266 ps",
        r#return: "6.1 µs",
        return_per_byte: "676 ps",
        terminate: "678.6 µs",
        restore_to: "606.7 µs",
        restore_to_per_delta: "162.4 µs",
        random: "9.4 µs",
        deposit_event: "13.4 µs",
        deposit_event_per_topic: "132.4 µs",
        deposit_event_per_byte: "2.3 ns",
        set_rent_allowance: "10.0 µs",
        set_storage: "266.9 µs",
        set_storage_per_byte: "2.0 ns",
        clear_storage: "176.0 µs",
        get_storage: "35.7 µs",
        get_storage_per_byte: "1.5 ns",
        transfer: "185.9 µs",
        call: "130.2 µs",
        call_transfer_surcharge: "175.0 µs",
        call_per_input_byte: "505 ps",
        call_per_output_byte: "731 ps",
        instantiate: "495.1 µs",
        instantiate_per_input_byte: "1.5 ns",
        instantiate_per_output_byte: "737 ps",
        hash_sha2_256: "3.3 µs",
        hash_sha2_256_per_byte: "4.1 ns",
        hash_keccak_256: "3.3 µs",
        hash_keccak_256_per_byte: "3.3 ns",
        hash_blake2_256: "3.1 µs",
        hash_blake2_256_per_byte: "1.5 ns",
        hash_blake2_128: "3.1 µs",
        hash_blake2_128_per_byte: "1.5 ns",
    },
}

The strategy here is to include every wasm instruction on its own and not manually curate them into categories. Otherwise a manual intervention would be necessary whenever those categories don't fit the current wasm execution engine we are benchmarking those on. Our strategy minimizes assumptions and therefore maximizes the amount of automation. Creating a schedule for compiled execution (this schedule is for wasmi executed contracts) is possible with a simple re-benchmark without writing any new code.

However, there are some assumptions that we needed to make. They are explained here:

substrate/frame/contracts/src/schedule.rs

Lines 112 to 134 in b3b6099

	/// Describes the weight for all categories of supported wasm instructions.
	///
	/// There there is one field for each wasm instruction that describes the weight to
	/// execute one instruction of that name. There are a few execptions:
	///
	/// 1. If there is a i64 and a i32 variant of an instruction we use the weight
	/// of the former for both.
	/// 2. The following instructions are free of charge because they merely structure the
	/// wasm module and cannot be spammed without making the module invalid (and rejected).
	/// End, Unrechable, Return, Else
	/// 3. The following instructions cannot be benchmarked because they are removed by any
	/// real world execution engine as a preprocessing step and therefore don't yield a
	/// meaningful benchmark result. However, in contrast to the instructions mentioned
	/// in 2. they can be spammed. We price them with the same weight as the cheapest
	/// instruction.
	/// 4. We price both i64.const and drop as InstructionWeights.i64const / 2. The reason
	/// for that is that we cannot benchmark either of them on its own but we need their
	/// individual values to derive (by substraction) the weight of all other instructions
	/// that use them as supporting instructions. Supporting means mainly pushing arguments
	/// and dropping return values in order to maintain a valid module.
	#[cfg_attr(feature = "std", derive(Serialize, Deserialize, WeightDebug))]
	#[derive(Clone, Encode, Decode, PartialEq, Eq)]
	pub struct InstructionWeights<T: Trait> {

As always: The easiest way to review this might be commit by commit.

@jacogr This will also add changes to the Schedule data structure that therefore needs an update in its js mirror.

[athei]

/bench pallet pallet_contracts

[parity-benchapp]

Error running benchmark: at-seal-instr-benchmark

stdout

TypeError: Cannot read property 'title' of undefined

[athei]

/bench runtime pallet pallet_contracts

[parity-benchapp]

Starting benchmark for branch: at-seal-instr-benchmark (vs master)

Comment will be updated.

[ascjones]

All in all LGTM, pending a review from someone with more wasm expertise.

One observation: where the instruction costs are essentially equivalent (within a certain error threshold) could we equalize them e.g. all the instr_i64* are virtually identical ~24_650_000 + 7_400_000, the minor differences being from the benchmarking run.

[athei]

One observation: where the instruction costs are essentially equivalent (within a certain error threshold) could we equalize them e.g. all the instr_i64* are virtually identical ~24_650_000 + 7_400_000, the minor differences being from the benchmarking run.

This is expected and most other contract platforms define manually curated categories of instructions in order to coalesce them. However, my goal here is to not make unnecessary assumptions about the execution engine. We just take the results at face value because changing the execution engine could break those category assumptions. We want that to be a highly automated process. So this category stuff won't fly here.

What you are proposing is a post process step to equalize weights that are closely together, right? I don't see the benefit of that. And frankly I don't want to introduce yet another tuning knob (what should the threshold be?).

[NikVolf]

Sorry for being too late

Looks great, other than few nits

[athei]

bot merge

[ghost]

Trying merge.