Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

new pallet: whitelist pallet #10159

Merged
merged 24 commits into from
Mar 1, 2022

Conversation

gui1117
Copy link
Contributor

@gui1117 gui1117 commented Nov 2, 2021

should be ready, but I need to a final review myself

@gui1117 gui1117 changed the title whitelist pallet new pallet: whitelist pallet Nov 2, 2021
@github-actions github-actions bot added the A3-in_progress Pull request is in progress. No review needed at this stage. label Nov 2, 2021
@xlc
Copy link
Contributor

xlc commented Nov 2, 2021

We have already implemented exactly this in our pallet open-web3-stack/open-runtime-module-library#603

@gui1117 gui1117 force-pushed the gui-shawntabrizi-preimage-pallet branch from ddc71ca to dd84eda Compare November 24, 2021 00:44
@gui1117 gui1117 changed the base branch from shawntabrizi-preimage-pallet to gav-schedule-by-hash November 24, 2021 00:44
@gui1117 gui1117 force-pushed the gui-shawntabrizi-preimage-pallet branch 3 times, most recently from 0eb5166 to 81d8e9e Compare December 1, 2021 07:05
Base automatically changed from gav-schedule-by-hash to master December 11, 2021 14:55
@gui1117 gui1117 force-pushed the gui-shawntabrizi-preimage-pallet branch from 0f0530a to 4fe4ca3 Compare December 13, 2021 07:00
@gui1117 gui1117 force-pushed the gui-shawntabrizi-preimage-pallet branch from 4fe4ca3 to 4826e1e Compare December 13, 2021 07:04
@gui1117 gui1117 marked this pull request as ready for review December 13, 2021 07:19
@gui1117 gui1117 added A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D5-nicetohaveaudit ⚠️ PR contains trivial changes to logic that should be properly reviewed. D9-needsaudit 👮 PR contains changes to fund-managing logic that should be properly reviewed and externally audited and removed A3-in_progress Pull request is in progress. No review needed at this stage. D5-nicetohaveaudit ⚠️ PR contains trivial changes to logic that should be properly reviewed. labels Dec 13, 2021
@viniul viniul added D1-audited 👍 PR contains changes to fund-managing logic that has been properly reviewed and externally audited and removed D9-needsaudit 👮 PR contains changes to fund-managing logic that should be properly reviewed and externally audited labels Jan 17, 2022
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@ggwpez
Copy link
Member

ggwpez commented Feb 25, 2022

/bench runtime pallet pallet_whitelist

@parity-benchapp
Copy link

parity-benchapp bot commented Feb 25, 2022

Benchmark Runtime Pallet for branch "gui-shawntabrizi-preimage-pallet" with command cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
Pallet: "pallet_whitelist", Extrinsic: "whitelist_call", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Whitelist WhitelistedCall (r:1 w:1)
Storage: Preimage StatusFor (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    16.12
              µs

Reads = 2
Writes = 2

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    16.12
              µs

Reads = 2
Writes = 2

Pallet: "pallet_whitelist", Extrinsic: "remove_whitelisted_call", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Whitelist WhitelistedCall (r:1 w:1)
Storage: Preimage StatusFor (r:1 w:1)
Storage: Preimage PreimageFor (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    17.79
              µs

Reads = 2
Writes = 3

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    17.79
              µs

Reads = 2
Writes = 3

Pallet: "pallet_whitelist", Extrinsic: "dispatch_whitelisted_call", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Whitelist WhitelistedCall (r:1 w:1)
Storage: Preimage PreimageFor (r:1 w:1)
Storage: Preimage StatusFor (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=     6696
              µs

Reads = 3
Writes = 3

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=     6696
              µs

Reads = 3
Writes = 3

Pallet: "pallet_whitelist", Extrinsic: "dispatch_whitelisted_call_with_preimage", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Whitelist WhitelistedCall (r:1 w:1)
Storage: Preimage StatusFor (r:1 w:1)
Storage: Preimage PreimageFor (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    20.48
    + n    0.001
              µs

Reads = 2 + (0 * n)
Writes = 3 + (0 * n)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    n   mean µs  sigma µs       %
    1     19.83     0.183    0.9%
  200     20.73     0.112    0.5%
  399     21.19     0.068    0.3%
  598     21.35      0.14    0.6%
  797     21.89     0.128    0.5%
  996     21.95     0.166    0.7%
 1195     22.15      0.11    0.4%
 1394      22.7     0.126    0.5%
 1593     22.85     0.131    0.5%
 1792     23.05     0.171    0.7%
 1991     23.56     0.076    0.3%
 2190     24.05     0.171    0.7%
 2389     24.25     0.208    0.8%
 2588     24.71     0.161    0.6%
 2787     24.76     0.169    0.6%
 2986        25     0.161    0.6%
 3185     25.35     0.233    0.9%
 3384     25.63     0.192    0.7%
 3583     25.95     0.121    0.4%
 3782     26.12     0.164    0.6%
 3981     26.53     0.161    0.6%
 4180     26.69     0.069    0.2%
 4379      27.2     0.308    1.1%
 4578     27.39     0.183    0.6%
 4777     27.74     0.193    0.6%
 4976     27.99     0.065    0.2%
 5175      28.5     0.139    0.4%
 5374     28.84     0.135    0.4%
 5573     28.71     0.285    0.9%
 5772     29.27     0.284    0.9%
 5971     29.39     0.123    0.4%
 6170     29.96     0.244    0.8%
 6369     30.05     0.102    0.3%
 6568     30.23     0.185    0.6%
 6767     30.93     0.114    0.3%
 6966     31.29     0.124    0.3%
 7165     31.68     0.227    0.7%
 7364     31.71     0.159    0.5%
 7563     31.88     0.136    0.4%
 7762     32.25     0.123    0.3%
 7961     32.72     0.216    0.6%
 8160     32.86     0.158    0.4%
 8359     33.43     0.218    0.6%
 8558     33.44     0.094    0.2%
 8757     34.01     0.217    0.6%
 8956     34.31      0.36    1.0%
 9155     34.46     0.254    0.7%
 9354     34.84     0.232    0.6%
 9553      35.1      0.18    0.5%
 9752     35.11      0.13    0.3%
 9951     35.41     0.233    0.6%

Quality and confidence:
param     error
n             0

Model:
Time ~=    20.45
    + n    0.002
              µs

Reads = 2 + (0 * n)
Writes = 3 + (0 * n)


…--manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@ggwpez
Copy link
Member

ggwpez commented Feb 25, 2022

/bench runtime pallet pallet_whitelist

@parity-benchapp
Copy link

parity-benchapp bot commented Feb 25, 2022

Benchmark Runtime Pallet for branch "gui-shawntabrizi-preimage-pallet" with command cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results
Pallet: "pallet_whitelist", Extrinsic: "whitelist_call", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Whitelist WhitelistedCall (r:1 w:1)
Storage: Preimage StatusFor (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    16.25
              µs

Reads = 2
Writes = 2

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    16.25
              µs

Reads = 2
Writes = 2

Pallet: "pallet_whitelist", Extrinsic: "remove_whitelisted_call", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Whitelist WhitelistedCall (r:1 w:1)
Storage: Preimage StatusFor (r:1 w:1)
Storage: Preimage PreimageFor (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    18.34
              µs

Reads = 2
Writes = 3

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    18.34
              µs

Reads = 2
Writes = 3

Pallet: "pallet_whitelist", Extrinsic: "dispatch_whitelisted_call", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Whitelist WhitelistedCall (r:1 w:1)
Storage: Preimage PreimageFor (r:1 w:1)
Storage: Preimage StatusFor (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=     6618
              µs

Reads = 3
Writes = 3

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=     6618
              µs

Reads = 3
Writes = 3

Pallet: "pallet_whitelist", Extrinsic: "dispatch_whitelisted_call_with_preimage", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Whitelist WhitelistedCall (r:1 w:1)
Storage: Preimage StatusFor (r:1 w:1)
Storage: Preimage PreimageFor (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    20.62
    + n    0.001
              µs

Reads = 2 + (0 * n)
Writes = 3 + (0 * n)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    n   mean µs  sigma µs       %
    1     19.89     0.072    0.3%
  200     21.02     0.127    0.6%
  399     21.45     0.139    0.6%
  598     21.65     0.238    1.0%
  797     22.15     0.147    0.6%
  996      22.1     0.102    0.4%
 1195     22.42     0.159    0.7%
 1394     22.88     0.085    0.3%
 1593     22.82     0.167    0.7%
 1792     23.27     0.085    0.3%
 1991     23.67     0.137    0.5%
 2190     23.82     0.106    0.4%
 2389     23.98     0.113    0.4%
 2588     24.84     0.176    0.7%
 2787     24.93     0.111    0.4%
 2986      25.1     0.148    0.5%
 3185     25.41     0.263    1.0%
 3384     25.61     0.151    0.5%
 3583     26.19     0.099    0.3%
 3782     26.25     0.099    0.3%
 3981      26.6     0.232    0.8%
 4180     27.17     0.367    1.3%
 4379     27.04     0.169    0.6%
 4578     27.22     0.211    0.7%
 4777     27.77     0.199    0.7%
 4976     27.97     0.158    0.5%
 5175     29.69     0.847    2.8%
 5374      28.8     0.124    0.4%
 5573     28.95     0.209    0.7%
 5772     29.12     0.215    0.7%
 5971     29.54     0.093    0.3%
 6170     29.91     0.114    0.3%
 6369     30.49     0.173    0.5%
 6568     30.64     0.181    0.5%
 6767     30.68     0.138    0.4%
 6966     31.16      0.12    0.3%
 7165     31.54     0.214    0.6%
 7364     31.62     0.219    0.6%
 7563     32.31      0.23    0.7%
 7762     32.36     0.206    0.6%
 7961     32.85     0.175    0.5%
 8160     33.07     0.208    0.6%
 8359     32.97     0.185    0.5%
 8558      33.4     0.183    0.5%
 8757     34.11     0.223    0.6%
 8956     33.89      0.24    0.7%
 9155     34.88     0.285    0.8%
 9354      34.8     0.133    0.3%
 9553     34.89     0.171    0.4%
 9752     35.36     0.299    0.8%
 9951     35.55     0.128    0.3%

Quality and confidence:
param     error
n             0

Model:
Time ~=    20.61
    + n    0.002
              µs

Reads = 2 + (0 * n)
Writes = 3 + (0 * n)


Parity Bot and others added 2 commits February 25, 2022 15:05
…--manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
.ok_or(Error::<T>::UnavailablePreImage)?;

let call = <T as Config>::Call::decode_all_with_depth_limit(
sp_api::MAX_EXTRINSIC_DEPTH,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For completeness we may need to add a bench which tests it with nested calls.
Currently we only bench the length of an encoded call, but if nesting is more expensive than length, it would be missed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think nesting makes a difference here. The depth limit here is to prevent against stackoverflow and other kinds of call decoding bombs. But the actual computational overhead here should be the same with anything being encoded or decoded really.

And then the weight of the call can help us determine cheaply and ahead of time that the call itself is okay.

@shawntabrizi
Copy link
Member

bot merge

@paritytech-processbot paritytech-processbot bot merged commit b4c188b into master Mar 1, 2022
@paritytech-processbot paritytech-processbot bot deleted the gui-shawntabrizi-preimage-pallet branch March 1, 2022 13:31
grishasobol pushed a commit to gear-tech/substrate that referenced this pull request Mar 28, 2022
* pallet whitelist

* refactor a bit

* fmt

* address audit

* improve tests

* return Ok + refund

* add test for dispatching failing

* add dispatch_whitelisted_call_with_preimage

* fmt

* better name

* Consume all data on decode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add error docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove phantom data

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use rust 2021

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update crate features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Bump Preimage max size

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>
grishasobol pushed a commit to gear-tech/substrate that referenced this pull request Mar 28, 2022
* pallet whitelist

* refactor a bit

* fmt

* address audit

* improve tests

* return Ok + refund

* add test for dispatching failing

* add dispatch_whitelisted_call_with_preimage

* fmt

* better name

* Consume all data on decode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add error docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove phantom data

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use rust 2021

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update crate features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Bump Preimage max size

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>
grishasobol pushed a commit to gear-tech/substrate that referenced this pull request Mar 28, 2022
* pallet whitelist

* refactor a bit

* fmt

* address audit

* improve tests

* return Ok + refund

* add test for dispatching failing

* add dispatch_whitelisted_call_with_preimage

* fmt

* better name

* Consume all data on decode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add error docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove phantom data

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use rust 2021

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update crate features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Bump Preimage max size

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>
driemworks added a commit to ideal-lab5/substrate-offchain-ipfs that referenced this pull request May 9, 2022
* Make some UI test expectations crisper (#10791)

* Make some UI test expectations crisper

* Update frame/support/test/tests/construct_runtime_ui/undefined_genesis_config_part.rs

* Update UI test expectations

* Missing newlines

* More whitespace issues

* contracts: `is_contract(address)` and `caller_is_origin()` are added to API (#10789)

* is_contract() and caller_is_origin() added to Ext API

* is_contract() exposed in wasm runtime.rs

* + test for is_contract()

* + seal_is_contract benchmark

* caller_is_origin() exposed to wasm/runtime.rs and covered by a test

* + seal_caller_is_origin benchmark

* Update frame/contracts/src/exec.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Update frame/contracts/src/exec.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Update frame/contracts/src/exec.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Update frame/contracts/src/wasm/runtime.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Update frame/contracts/src/wasm/runtime.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Update frame/contracts/src/wasm/runtime.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Update frame/contracts/src/exec.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* identation fix for benchmark macroses; test cosmetic improvement

* benchmark fix

* + is_contract() wasm test

* + caller_is_origin() wasm test

* Apply suggestions from code review

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* is_contract() to borrow param instead of taking ownership

* phrasing improved

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* fixed wasm tests according to @athei feedback

* dead code warnings suppressed by unstable-interface attributes

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>

* maybe_total_supply in pallet-assets (#10799)

* maybe_total_supply in pallet-assets

* Update frame/assets/src/functions.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Referenda and Conviction Voting pallets (#10195)

* Initial draft of new referendum state machine.

* Docs

* Fixes

* Fixes

* Add conviction-voting pallet

* Basic build

* Building

* Some TODOs

* Tests building

* Add missing file

* Basic lifecycle test

* Add couple of tests

* Another test

* More tests

* Fixes

* Fixes

* Formatting

* Fixes

* Tests

* Fixes

* Fixes

* More tests

* Formatting

* First few benchmarks

* First few benchmarks

* Defered queue servicing

* More testing

* Benchmarks

* Fiddly benchmark

* Final nudge benchmarks

* Formatting

* Formatting

* Finished up benchmarks

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_referenda --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/referenda/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Events finished

* Missing file

* No GenesisConfig for Referenda

* Formatting

* Docs

* Docs

* Docs

* Per-class conviction voting

* New test & mock utils

* More tests

* Tests

* Tests finished 🎉

* Benchmarking stuff

* Fixes

* Test harness

* Test harness

* Benchmarks for Conviction=Voting

* Benchmarking pipeline complete

* Docs

* Formatting

* Remove unneeded warning

* Fix UI tests

* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_conviction_voting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/conviction-voting/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Docs

* Update frame/conviction-voting/src/vote.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* update sp-runtime version

* MEL Fixes for Referenda and Conviction Voting (#10725)

* free maxencodedlen

* more maxencodedlen

* more MEL

* more mel

* disable storage info

* More Referenda Patches (#10760)

* basic fixes

* fix benchmarking

* fix license

* prevent panic in curve math

* fmt

* bump crate versions

* Update mock.rs

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* More efficient WASM instance memory decommit on macos (#10801)

* More efficient WASM instance memory decommit on macos

* Apply suggestions from code review

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Updated error message

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* make inner field of IdentityFields pub (#10773)

* Corrected description of clear_attribute extrinsic (#10729)

* block validators after X sessions, update mock runtime for test

* sp-maybe-compressed-blob: reduce boilerplate code (#10814)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* `seal_delegate_call` api function (support for library contracts)  (#10617)

* seal_call_code implementation

- tests
- benchmark

* Addressing @xgreenx's comments

* Fix test-linux-stable-int

* Rename seal_call_code to seal_delegate_call

* Pass value unchanged into lib contract

* Address @athei's comments

- whitespace .wat issues
- wrong/missing .wat comments
- redundant .wat calls/declarations

- change order of functions (seal_delegate_call right after seal_call)
  in decls, tests, benchmark
- fix comments, move doc comments to enum variants
- remove unnecessary empty lines

- rename runtime cost DelegateCall to DelegateCallBase
- do not set CallFlags::ALLOW_REENTRY for delegate_call

* Do not pass CallFlags::ALLOWS_REENTRY for delegate_call

* Update comment for seal_delegate_call and CallFlags

* Addressing @athei's comments (minor)

* Allow reentry for a new frame after delegate_call (revert)

* Same seal_caller and seal_value_transferred for lib contract

- test
- refactor frame args due to review
- logic for seal_caller (please review)

* Put caller on frame for delegate_call, minor fixes

* Update comment for delegate_call

* Addressing @athei's comments

* Update weights generated by benchmark

* Improve comments

* Address @HCastano's comments

* Update weights, thanks @joao-paulo-parity

* Improve InvalidCallFlags error comment

* Add Storage Info to Various Pallets (#10810)

* atomic swap

* bounties

* bounties fmt

* gilt

* indices

* nicks

* randomness-collective-flip

* recovery

* reuse maxapprovals

* Update tests.rs

* Update frame/randomness-collective-flip/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* use the correct bound

* complete recovery

* use `bounded_vec` macro

* Update tests.rs

* transaction payment

* uniques

* mmr

* example offchain worker

* beefy-mmr

* Update frame/recovery/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use BoundedVec instead of a type-parameterized BoundedString

* cargo fmt

* Update frame/atomic-swap/src/lib.rs

* use config const

* Update lib.rs

* update mel_bound

* fmt

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* fixed regression in inline docs (#10819)

* try-runtime-cli: Add to docs (#10822)

* try-runtime-cli: Add to docs of pre/post hooks

* Add note about --ws-max-out-buffer-capacity 1000

* Bump serde from 1.0.132 to 1.0.136 (#10816)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.132 to 1.0.136.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.132...v1.0.136)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* uodate tests, ipfs tests failing

* --dev implies --tmp (#10828)

* Add a new host function for reporting fatal errors; make WASM backtraces readable when printing out errors (#10741)

* Add a new host function for reporting fatal errors

* Fix one of the wasmtime executor tests

* Have `#[runtime_interface(wasm_only)]` actually mean WASM-only, and not no_std-only

* Print out errors through `Display` instead of `Debug`

* Switch one more trait to require `Error` for its error instead of only `Debug`

* Align to review comments

* specify ipfs version, update tests

* Add db params for export-state CLI command (#10830)

* update tests, cleanup

* Allow `SetBalance` to handle error when trying to kill acount with reference counter. (#10826)

* bug found

* fix logic

* a little simpler

* add test

* UncheckedExtrinsic: Harden decode and clarify `EXTRINSIC_FORMAT_VERSION` (#10829)

* UncheckedExtrinsic: Harden decode and clarify `EXTRINSIC_FORMAT_VERSION`

* Apply suggestions from code review

* fix typo

* pallet-staking: Add extrinsic `force_apply_min_commission` (#10786)

* pallet-staking: Add extrinsic `force_apply_min_commission`

* Add benchmarks

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Bound iteration by  max_validator_count

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Only apply to 1 validator

* Update doc comments

* Uncomment tests

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Accept signed origins

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Remove contains_key check

* Add test for try_mutate_exists

* Impove try_mutate_exists docs

* Delete redundant try_mutate_exists tests;

* Delete residual from removed test

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Return an error when the stash does not exist

* Update try_mutate_exist doc wording

* Update frame/staking/src/pallet/mod.rs

* Apply suggestions from code review

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Upgradable contracts using `set_code` function (#10690)

* poc logic

* set_code_hash impl, tests, benchmark

* Address @xgreenx's comments

* Move func defs closer to set_storage

* Check if code exists

- increment/decrement codehash refcount

* Document error for non-existing code hash

* Revert unrelated change

* Changes due to @athei's review

* Fix error handling

- comment errors: ReturnCodes
- update mock ext implementation
- return Error::CodeNotFound when no code for such hash

* Emit ContractCodeUpdated when setting new code_hash

* Address @athei's comments

* Move related defs to the bottom

* Minor comment update

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Improve docs

* Improve docs

* Update frame/contracts/src/wasm/runtime.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Refactor set_code_hash test

* Minor change to benchmark

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Minor change to benchmark

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Minor comment refactor

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Address @HCastano's comments

* Update seal_set_code_hash comment

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Move set_code_hash after delegate_call

* Move function to the bottom

* Moved and changed banchmark, added verify block

* Bring back previous benchmark

* Remove skip_meta for seal_set_code_hash

* Bring back skip_meta for seal_set_storage_per_new_kb

* Apply weights

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Separate wasmi and wasmer sandbox implementations into their own modules (#10563)

* Moves wasmi specific `ImportResolver` and `MemoryTransfer` impls to submodule

* Splits context store environmental, moves impl `Externals` to wasmi backend

* Adds wasmer sandbox backend stub module

* Move sandbox impl code to backend specific modules

* Moves wasmi stuff

* Fixes value conversion

* Makes it all compile

* Remove `with_context_store`

* Moves `WasmerBackend` to the impl

* Reformat the source

* Moves wasmer MemoryWrapper

* Reformats the source

* Fixes mutability

* Moves backend impls to a submodule

* Fix visibility

* Reformat the source

* Feature gate wasmer backend module

* Moves wasmi memory allocation to backend module

* Rename WasmerBackend to Backend

* Refactor dispatch result decoding, get rid of Wasmi types in common sandbox code

* Reformat the source

* Remove redundant prefixes in backend functions

* Remove wasmer-sandbox from default features

* Post-review changes

* Add conversion soundness proof

* Remove redundant prefix

* Removes now redundant clone_inner

* Add `Error::SandboxBackend`, refactor invoke result

* Fix comments

* Rename `Error::SandboxBackend` to `Sandbox`

* Simplifies logic in `wasmer_backend::invoke`

* Fixes memory management

* Show Network ID when creating and inspecting (#10838)

Signed-off-by: Antonio Yang <yanganto@gmail.com>

* Reduce overhead of generating network event metrics (#10839)

* Fix reentrancy of FrozenBalance::died hook (#10473)

* assets: execute `died` hook outside of mutate

Signed-off-by: Oliver Tale-Yazdi <oliver@tasty.limo>

* assets: extend tests for `died` hook

Signed-off-by: Oliver Tale-Yazdi <oliver@tasty.limo>

* assets: update doc of FrozenBalance::died

Signed-off-by: Oliver Tale-Yazdi <oliver@tasty.limo>

* assets: review fixes

- fix cases where `died` should not have been called
- use `Option<DeadConsequence>` instead of `DeadConsequence`

Signed-off-by: Oliver Tale-Yazdi <oliver@tasty.limo>

* assets: update comment in mock.rs

Signed-off-by: Oliver Tale-Yazdi <oliver@tasty.limo>

* assets: return `Remove` in dead_account

The return value is ignored in the only case that it is produced
by a call, but having it this way makes it more understandable.

Signed-off-by: Oliver Tale-Yazdi <oliver@tasty.limo>

* allow trailing comma (#10841)

* build-script-utils: allow reading the git commit hash from env var (#10845)

* build-script-utils: allow reading the git commit hash from env var

* build-script-utils: make the env var name substrate specific

* Don't create DB leaves when syncing historic blocks (#10844)

* Don't create DB leaves when syncing historic blocks

* Changed leaves check and added test

* fmt

* cleanup, use unsigned tx when appropriate

* add weights

* fix broken link (#10846)

* subkey: Support `--version` cli command (#10853)

* subkey: Support `--version` cli command

* FMT :facepalm:

* Remove `u32_trait` (#10850)

* Remove `u32_trait`

This trait only existed because there wasn't any const generic support at time of creation. However,
we now have support for it :)

* FMT

* wasm-builder: Support latest nightly (#10837)

* wasm-builder: Support latest nightly

With latest nightly, aka rust version 1.60+ namespaced features are added. This changes the handling
of optional dependencies. We currently have features that enable optional dependencies when `std` is
enabled. This was before no problem, but now the wasm-builder detects them as enabled. To support
the transition period until 1.60 is released as stable, this pr adds an heuristic to not enable these
optional crates in the wasm build when they are enabled in the `std` feature. This heuristic fails
when someones enables these optional dependencies from the outside as well as via the `std` feature,
however we hope that no one is doing this at the moment. When namespaced features are enabled, these
dependencies needs to be enabled using `dep:dependency-name` to solve this properly.

https://doc.rust-lang.org/cargo/reference/unstable.html#namespaced-features

* Remove accidentally added features

* update tests, remove unneeded params

* min authorities check before removal

* contracts: Fix `seal_call` weights (#10796)

* Fix call weights

* Fix instantiate benchmark

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Remove stale and superflous comments

* `decrement_refcount` should be infallible

* Don't hardcode increment_refcount, decrement_refcount

* Rename CopyIn/CopyOut

* Fix warning in tests

Co-authored-by: Parity Bot <admin@parity.io>

* Enable the `parking_lot` feature for `tokio` (#10855)

* [ci] fix publish-rustdocs (#10858)

* Fix beefy mock ecdsa keys (#10854)

Compressed ECDSA keys requires to have 0x02 or 0x03 as their first byte
in order to allow public key recovery.

Nevertheless the test was working because of the `unwrap_or_default()`
at the end of the conversion routine (i.e. the invalid keys were
converted to an empty vector).

* Mark sync_state_genSyncSpec JSON-RPC as safe (#10832)

* Mark sync_state_genSyncSpec JSON-RPC as safe

* Note that parameter is unused

* Ideally I'd wait for compilation to be finished before pushing, but it's really taking a long time

* Remove deny_unsafe parameter

* Remove unused dependency

* Reduce CPU overhead of gossip (#10859)

* sp-core-hashing: use the `digest::Digest` trait to handle the hashing function uniformly (#10835)

* sp-core-hashing: use sha3 instead of tiny-keccak

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* use blake2 instead of blake2-rfc

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* improve som hashing of sp-core and sp-api-proc-macro

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Some nits

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* cargo fmt

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Use fully qualified sytnax for `retain_mut` (#10865)

* add llvm (#10864)

* refactor election score (#10834)

* refactor election score

* Test for ord

* remove reference

* vec -> slice

* change iter to iter_by_significance

* improve doc

* fix typo

* add explanation about [u128; 3]

* consolidate threshold and epsilon

* random fixes

* rename

* remove Into

* make iter_by_sig private

* remove vec

* Fix tests

* Measure per byte and not kb for certain benchmarks (#10863)

* Tidy Democracy (#10867)

* add test

* Assorted refactorings

* complete test

* saturating math

* final check

* use `default`

Co-authored-by: Gav Wood <gavin@parity.io>

* cleanup storage maps, cleanup unused imports

* sc-executor: Improve logging (#10869)

Improves the logging by switching to `tracing` for a better log output. Besides that, it also adds a
trace for the function being executed.

* fix test issues

* Track allowed requests for state/warp sync (#10843)

* Track allowed requests for state/warp sync

* Added missing allowed_requests resets

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fmt

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Slots: Ensure that a race betwen finalized and best number is taken care off (#10876)

* Remove old lock file (#10879)

* Further reduce the CPU overhead of networking metrics (#10875)

* Simplify `num_connected_peers`

* Track requested peer counts

* Revert "Track requested peer counts"

This reverts commit 9f1c8704353df6afc17ed7e9f4ab8d8e29466ae4.

* Remove `substrate_sub_libp2p_peerset_num_requested` metric

* Remove two unused functions that I forgot to get rid of in previous commit

* Introduce `BoundedVec::iter_mut` (#10884)

* Introduce iters into BoundedVec

* Fix

* Remove unneeded funcs

* Update frame/support/src/storage/bounded_vec.rs

* Update frame/support/src/storage/bounded_vec.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Slots: Log total proposing duration as milliseconds (#10886)

Parachains have currently a total proposing time of 500ms, so it this
currently always prints `0`. While actually the value is not `0` ;)

* staking: Clarify reward calc docs (#10890)

* tracing: Adds `init_for_tests` (#10893)

This function is useful for tests. It will enable `TRACE` logging and also uses the libtest aware writer.

* make submissions pub (#10899)

* contracts: Allow stack height metering to be disabled (#10877)

* Allow stack height metering to be disabled

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Parity Bot <admin@parity.io>

* Rename Uniques Error::Unknown to something more sensible (#10895)

* Rename Uniques Error::Unknown to something more sensible

* Typos

* Typos

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* state-machine: Move all functionality from trie backend to the essence (#10904)

* state-machine: Move all functionality from trie backend to the essence

This is required for some future changes of me and it also makes more sense to have all the
functionality inside the essence. Besides that it changes the child root cache to directly
cache the hash.

* Update primitives/state-machine/src/trie_backend_essence.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* FMT

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* staking: Remove `SessionInterface` supertrait (#10901)

* consensus-slots: cleanup SlotDuration config (#10878)

* consensus-slots: cleanup the SlotDuration config

* fix tests

* address review comments

* add contracts pallet + rpc to runtime

* Replace libsecp256k1 with k256 in FRAME related code (#10883)

* Replace libsecp256k1 with k256 in beefy-mmr

* Port of FRAME `contracts` benchmarking from `libsecp256k1` to `k256`

* Newtype to allow `Pcg32` rng usage with `k256` in contracts benchmarks

* Use `sp-io::crypto` to generate dummy keys in `contracts` bechmarks

* More compact code

* Cargo fmt

* Build `sp-keystore` only for dev profile

* Move public key generation back to the `map`

* Clean obsolete BABE's weight data (#10748)

* Clean obsolete BABE weight data
* Take out test assertion from check closure
* Optimize metadata access using `HeaderMetadata` trait
* Apply suggestions from code review
* Introduce finalize and import pre-commit synchronous actions
* Do not hold locks between internal methods calls
* Remove unused generic bound
* Apply suggestions from code review
* Register BABE's pre-commit actions on `block_import` instead of `start_babe`
* PreCommit actions should be `Fn` instead of `FnMut`
* More robust safenet in case of malformed finality notifications

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Factor DB weights out into their own files (#10908)

* Factor DB weights out into their own files

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Review fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix CI

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Weights in own mod

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Get rid of unnecessary use of `async-std` in non-test code (#10891)

* No longer generate specs with consensus_engine field (#10346)

* No longer generate specs with consensus_engine field

* #[allow(unused)]

* Upgrading parity-scale-codec to v3 (#10825)

* Upgraded dependencies

* Adapting code to scale v3

* Empty commit to trigger CI

* Triggering CI

* Fixing UI test

* Remove superfluous dev-dep added by #9228

* Cryout for CI

* sc-cli: Fix bugs after switching to clap3 (#10920)

* sc-cli: Fix bugs after switching to clap3

Before switching to clap3 we support cli options like `--reserved-nodes A B` and after you needed to
pass `--reserved-nodes` cli option multiple times `--reserved-nodes A --reserved-nodes B`. This is
fixed by setting `multiple_occurrences(true)` option. This also done for all the other `Vec` cli
options in `sc-cli`. Besides that `--sync` wasn't supporting case insensitive parsing of the value.
This is now also supported. For both regressions a test is added. Besides that the pr removes all
the `rename_all = PascalCase` attributes, because they are not needed. All other `ArgEnum`s were
checked and all are already using `ignore_case(true)`.

* Bring back `PascalCase`, because otherwise it falls back to `kebab-case`...

* Storage benchmarking (#10897)

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP: DB benchmarking

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* WIP

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Simplify code

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove old files

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove old files

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Minimize changes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add license

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove dependencies

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Extend template

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Linter

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Linter

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Beauty fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove default

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add feature

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove seed

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* CI wakeup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Review fixes

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Adding doc

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Adding doc

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Improve template

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Do not expose columns

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix ColumnId

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Nicer template prints

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix json path

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Simplify `bench_write` logic

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Invert tx before the second commit

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* add transfer assets extrinsic

* add chain ext

* Minor improvements to `bounded_vec` and `defensive`.  (#10873)

* Fix a few things in bounded_vec

* add test for try_extend

* Update frame/support/src/storage/bounded_vec.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* some review comments

* use swap

* remove clone

* use pop instead of truncate

* remove warn

* review comments

* Update frame/support/src/storage/bounded_vec.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fix rustdoc

* fix links

* undo link

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Replace libsecp256k1 with secp256k1 (#10798)

* Replace libsecp256k1 with secp256k1

* Wipe ecdsa secret key from memory on drop

* Some comments for a known issue

* Safer core crypto primitives `from_slice` constructor

Previous version panics if slice lenght is not the expected one.

* Unit test fix

* Enable use of global secp256k1 context

* Better comments for ecdsa `Pair` drop

* Replace `libsecp256k1` with `seco256k1` in `beefy-mmr`

Used to convert ecdsa public key to ETH address

* Replace `libsecp256k1` with `secp256k1` in FRAME `contracts`benchmarks

* Temporary rollback of `beefy-mmr` to libsecp256k1

Check for detected build issues

* Cargo fmt

* Rollback of FRAME `contracts` benchmarks to `libsecp256k1`

* Rollback for unrelated changes

* Typo fix

* Add comments for deprecated `ecdsa_verify` and `secp256k1_ecdsa_recover`

* Integrate try-runtime into substrate node template (#10909)

* [10892-integrate-try-runtime-into-node-template] - Integrated try-runtime into node template

* [10892-integrate-try-runtime-into-node-template] Added match arms for try-runtime in command.rs

* [10892-integrate-try-runtime-into-node-template] Added match arms for try-runtime in command.rs

* Added feature flag for try-runtime in node-template/node and enabled try-runtime for node-template/runtime

* Added missing type annotations for try-runtime SubCommand in node-template

* Added missing type annotations for try-runtime SubCommand in node-template

* Implemented frame_try_runtime::TryRuntime<Block> for the node-template Runtime

* doc corrections (#10936)

* #10576: generic utility to unsubscribe from broadcast upon drop of the rx-side. (#10708)

* #10576: refactor `sc-utils::notification` and `sc-client-api::notifications`, so that they use common subscribe/unsubscribe routines

* Add some docs. Reorganise `sc-utils::notification`

* `sc-clent-api::notifications` and `sc-utils::notification` — ensure the SubscriptionGuard is dropped before the Rx-channel

* `sc-utils::pubsub::SubscriptionGuard` make it a bit more ergonomic.

Let the `Rx` to be put inside of the `SubscriptionGuard`, so that the latter shall guarantee the order:
- first unsubscribe;
- then drop the `Rx`.

* Being less zealous with splitting the modules into little pieces

* rework pubsub: the concrete usage should only define a good registry type

* sc-client-api::notifications: make it comply with the reworked pubsub

* cargo fmt

* make sc-client-api tests work

* Address the review notes

* cargo fmt

* Describe the behaviour of pubsub registry

* Doc-comments for module `sc-utils::pubsub`

* Fix: it used to send notifications regardless of the filter setup during subscription

* `sc-client-api::StorageNotifications` the API does not have to require mut-self-reference.

As a result `sc-service::Client` does not have to wrap its `storage_notifications` into a Mutex.

* cargo fmt

* Several changes addressing the notes by @bckhr.

- Remove the `impl Default for StorageNotifications<Block>`;
- no need for groupping the `remove_from` and `listen_from` into a separate `helpers` module;
- remove unnecessary import `use registry::SubscribeOp`.

* Add a doc-comment to the `sc-client::notifications::SubscribeOp`

* As per @bkchr note on the unproven assertion: behave gracefully upon receiving a duplicate subscription-ID.

* sc-utils::pubsub: log when a registry yields an ID that does point to an existing sink

* `sc-utils::notifications`: payload materialized lazily

* Update Cargo.lock (after adding `log` as a dependency to the `sc-utils`)

* `sc-client-api::notifications`: introduce a struct (instead of a type def) for the notification message

* Get rid of `sc-utils::pubsub::Channel` trait (instead just use the `sc-utils::mpsc`)

* The SubsID is no more generic: the fact it is a `Copy` is known — no need to pass it by ref

* sc-utils::pubsub internals do not have to be generic over the channel type

* Rename Hub::dispatch into Hub::send

* That method was unnecessary (`SubscriberSink::render_notification`)

* cargo fmt

* No need for a separate UnsubscribeGuard type

* Ditch the type-def of SubsID in the sc-utils::pubsub, instead — just use the crate::id_sequence::SeqID

* Return the <Registry as Dispatch>::Ret when sending an item

* Make the `Hub<M, R>::lock_registry(...)` method more ergonomic

* cargo doc links

* cargo doc links

* Use a simpler name for the type

* cargo doc links

* Derive `Default` rather than implement it

* Derive `Default` rather than implement it

* Remove an unnecessary usage of type_name

* Define a more cautious order between sinks.remove->registry.unsubscribe and registry.subscribe->sinks.insert

* Hub: lock_registry_for_tests->map_registry_for_tests — a safer choice for a public API

* Replace Mutex over the shared Registry with a ReentrableMutex+RefCell

* sc-utils::pubsub: add tests for a panicking registry

* Add the missing copyright headers

* Arc<Vec<_>> -> Arc<[_]>

* Bring back MaxNominations as a metadata constant (#10947)

* chain ext: parse params and call transfer func

* Bump futures from 0.3.16 to 0.3.19 (#10930)

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.19.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.19)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Clean up extra_constant renaming. (#10935)

Co-authored-by: Xiankun Cheng <xiankuncheng@Xiankuns-MacBook-Pro-2.local>

* Bump smallvec from 1.7.0 to 1.8.0 (#10949)

Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.7.0 to 1.8.0.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.7.0...v1.8.0)

---
updated-dependencies:
- dependency-name: smallvec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* new pallet: whitelist pallet (#10159)

* pallet whitelist

* refactor a bit

* fmt

* address audit

* improve tests

* return Ok + refund

* add test for dispatching failing

* add dispatch_whitelisted_call_with_preimage

* fmt

* better name

* Consume all data on decode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add error docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove phantom data

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use rust 2021

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update crate features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Bump Preimage max size

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>

* Release primitive crates (#10937)

* cargo set-version --bump major -p pallet-contracts-primitives

* cargo set-version --bump major -p sp-core

* cargo set-version --bump major -p sp-runtime-interface

* cargo set-version --bump major -p sp-wasm-interface

* cargo set-version --bump major -p sp-runtime

* cargo set-version --bump major -p sp-storage

* cargo set-version --bump major -p sp-rpc

* cargo set-version --bump major -p sp-io

* cargo set-version --bump major -p sp-trie

* cargo set-version -p sp-state-machine  -- 0.12.0

* cargo set-version -p sp-externalities  -- 0.12.0

* cargo set-version -p sp-keystore -- 0.12.0

* cargo set-version --bump major -p sp-keyring

* cargo set-version --bump major -p sp-version

* cargo set-version --bump major -p sp-tracing

* cargo set-version --bump major -p sp-application-crypto

* cargo set-version --bump major -p sp-arithmetic

* cargo unleash version bump-major -p sp-runtime-interface-proc-macro

* Add codec max-encoded-len feature to sp-arithmetic

* cargo unleash version bump-major -p sp-core-hashing-proc-macro

* Update new whitelist pallet to new primitive versions (#10953)

* sp-trie: Switch to thiserror and some other small cleanups (#10954)

* sp-trie: Switch to thiserror and some other small cleanups

* Add some extra method for converting a compact proof to a memory db

* Only maintain at most 1 `UnlockChunk` per era (#10670)

* Only maintain at most 1 `UnlockChunk` per era

* Bound `unlocking`

* Run cargo +nightly-2021-10-29 fmt

* Make benchmarks stuff compile

* Update frame/staking/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove DerefMut; Implement neccesary methods directly

* Doc comments for new BoundedVec methods

* Fix benchmarks

* wip bonded_vec macro

* Correct rust doc

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update staking::Config impls

* Add MaxUnlockingChunks to more places

* Use defensive saturating add

* FMT

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* staking: Expose `MaxUnlockingChunks` in metadata as a const (#10958)

* Allow trailing commas for `bounded_vec!` (#10959)

* SimpleSlotWorker: Do not implement `SlotWorker` for all types implementing `SimpleSlotWorker` (#10934)

Because Rust currently doesn't support specialization, it prevents users from implementing
`SlotWorker` for their own types. This pr solves this by removing the generic implementation of
`SlotWorker` for `SimpleSlotWorker` and providing some wrapper type for that.

* Bump digest from 0.10.2 to 0.10.3 (#10960)

Bumps [digest](https://github.com/RustCrypto/traits) from 0.10.2 to 0.10.3.
- [Release notes](https://github.com/RustCrypto/traits/releases)
- [Commits](https://github.com/RustCrypto/traits/compare/digest-v0.10.2...digest-v0.10.3)

---
updated-dependencies:
- dependency-name: digest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump scale-info from 2.0.0 to 2.0.1 (#10965)

Bumps [scale-info](https://github.com/paritytech/scale-info) from 2.0.0 to 2.0.1.
- [Release notes](https://github.com/paritytech/scale-info/releases)
- [Changelog](https://github.com/paritytech/scale-info/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/scale-info/commits)

---
updated-dependencies:
- dependency-name: scale-info
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add pr-custom-review config (#10968)

* Revise how staking configurations are set (#10955)

* Revise how staking configurations are set

fixes #10938

* Fix and add additional tests

* Format

* Formatting

* Add doc

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/staking/src/tests.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Format

* Fix build

* Update weights.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>

* Bump libc from 0.2.112 to 0.2.119 (#10967)

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.112 to 0.2.119.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.112...0.2.119)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump parity-db from 0.3.5 to 0.3.8 (#10970)

Bumps [parity-db](https://github.com/paritytech/parity-db) from 0.3.5 to 0.3.8.
- [Release notes](https://github.com/paritytech/parity-db/releases)
- [Commits](https://github.com/paritytech/parity-db/commits/v0.3.8)

---
updated-dependencies:
- dependency-name: parity-db
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [EPM pallet]: remove `number of signed submissions` (#10945)

* [EPM pallet]: remove `number of signed submissions`

Closing #9229

* fix tests

* remove needless assert

* Update frame/election-provider-multi-phase/src/lib.rs

* cargo fmt

Signed-off-by: Niklas <niklasadolfsson1@gmail.com>

* fix grumbles

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* drop `num_signed_submissions` in WeightInfo too

* fix build

Co-authored-by: Parity Bot <admin@parity.io>

* Refactored block body database scheme (#10779)

* Refactored tx storage database scheme

* Bump parity-db

* fmt

* Fix handling invalid index size + test

* Removed superflous result

* Minor changes

* fmt

* Deprecate "paritydb-experimental" CLI in favour or "paritydb" (#10975)

* Deprecate paritydb-experimental

* Updated comment

* [contracts] Stabilize "seal0" `is_contract` and `caller_is_origin`  (#10971)

* stabilize `seal_is_contract`

* stabilize `seal_caller_is_origin`

* Trie version migration pallet  (#10073)

* starting

* Updated from other branch.

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right logic to avoid switch on reencode when default layout.

* Clean up some todos

* remove trie meta from root upstream

* update upstream and fix benches.

* split some long lines.

* UPdate trie crate to work with new design.

* Finish update to refactored upstream.

* update to latest triedb changes.

* Clean up.

* fix executor test.

* rust fmt from master.

* rust format.

* rustfmt

* fix

* start host function driven versioning

* update state-machine part

* still need access to state version from runtime

* state hash in mem: wrong

* direction likely correct, but passing call to code exec for genesis
init seem awkward.

* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.

* stateversion from runtime version (core api >= 4).

* update trie, fix tests

* unused import

* clean some TODOs

* Require RuntimeVersionOf for executor

* use RuntimeVersionOf to resolve genesis state version.

* update runtime version test

* fix state-machine tests

* TODO

* Use runtime version from storage wasm with fast sync.

* rustfmt

* fmt

* fix test

* revert useless changes.

* clean some unused changes

* fmt

* removing useless trait function.

* remove remaining reference to state_hash

* fix some imports

* Follow chain state version management.

* trie update, fix and constant threshold for trie layouts.

* update deps

* Update to latest trie pr changes.

* fix benches

* Verify proof requires right layout.

* update trie_root

* Update trie deps to  latest

* Update to latest trie versioning

* Removing patch

* update lock

* extrinsic for sc-service-test using layout v0.

* Adding RuntimeVersionOf to CallExecutor works.

* fmt

* error when resolving version and no wasm in storage.

* use existing utils to instantiate runtime code.

* migration pallet

* Patch to delay runtime switch.

* Revert "Patch to delay runtime switch."

This reverts commit d35f273b7d67b1b85a9e72973cab13c5c156c1d3.

* fix test

* fix child migration calls.

* useless closure

* remove remaining state_hash variables.

* Fix and add more tests

* Remove outdated comment

* useless inner hash

* fmt

* remote tests

* finally ksm works

* batches are broken

* clean the benchmarks

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>

* Update frame/state-trie-migration/src/lib.rs

* brand new version

* fix build

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update primitives/storage/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* fmt and opt-in feature to apply state change.

* feature gate core version, use new test feature for node and test node

* Use a 'State' api version instead of Core one.

* fix merge of test function

* use blake macro.

* Fix state api (require declaring the api in runtime).

* Opt out feature, fix macro for io to select a given version
instead of latest.

* run test nodes on new state.

* fix

* new test structure

* new testing stuff from emeric

* Add commit_all, still not working

* Fix all tests

* add comment

* we have PoV tracking baby

* document stuff, but proof size is still wrong

* FUCK YEAH

* a big batch of review comments

* add more tests

* tweak test

* update config

* some remote-ext stuff

* delete some of the old stuff

* sync more files with master to minimize the diff

* Fix all tests

* make signed migration a bit more relaxed

* add witness check to signed submissions

* allow custom migration to also go above limit

* Fix these pesky tests

* ==== removal of the unsigned stuff ====

* Make all tests work again

* separate the tests from the logic so it can be reused easier

* fix overall build

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Slightly better termination

* some final tweaks

* Fix tests

* Restrict access to signed migrations

* address most of the review comments

* fix defensive

* New simplified code

* Fix weights

* fmt

* Update frame/state-trie-migration/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* make the tests correctly fail

* Fix build

* Fix build

* try and fix the benchmarks

* fix build

* Fix cargo file

* Fix runtime deposit

* make rustdoc happy

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_state_trie_migration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/state-trie-migration/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>

* BEEFY and GRANDPA protocol names should use full genesis hash (#10974)

std::fmt::Display shows formats as reduced hash (e.g. 0xb0a8…dafe)

Use hex::encode to format full hash.

Signed-off-by: acatangiu <adrian@parity.io>

* Move weight constants to own mod (#10980)

* Move block+ext weights to own mod

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Unused import

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* add block usage logs to system pallet (#10940)

* add block usage logs to system pallet

* add Debug

* use % instead of default Debug impl

* change formatting

* revert

* update runtime storage, update chain extension

* cleanup chain ext

* Fix the undeterministic storage proof recorded for the same execution (#10915)

* Add a test case for the determinism of recorded proof

* Replace HashMap with BTreeMap for the actual proof records

* cargo +nightly fmt --all

* Store the trie nodes in BTreeSet for StorageProof

* Nit

* Revert the BTreeMap changes and sort when converting to storage proof

* Remove PartialEq from StorageProof

* Remove unnecessary change

* Add `compare` method to StorageProof

* FMT

* Dummy change to trigger CI

* Use `BTreeSet` for StorageProof and keep using `Vec` for CompactProof

* Update comment on `iter_nodes`

* Revert `PartialEq` removal

* create iris ledger pallet

* ledger funcs in chain ext

* rename Iris -> IrisAssets, update readmes

* Minor Uniques pallet improvements and XCM v3 preparations (#10896)

* Introduce Helper to Uniques for benchmark stuff

* Fixes

* Formatting

* Featuregate the Helper, include ContainsPair

* Introduce & use EnsureOriginWithArg

* Benchmarking

* Docs

* More ContainsBoth helpers

* Formatting

* Formatting

* Fixes

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* election provider support: Update some test only types (#10983)

* Feedback from @XLC for Referenda Pallet (#10991)

* feedback from @xlc

* english

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make bags-list generic over node value and instantiable (#10997)

* make instantiable

* update

* cargo fmt

* Clean up

* bags-list: Make it generic over node value

* Respond to some feedback

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add back default impl for weight update worst case

* Update to Score in more places'

* Use VoteWeight, not u64 to reduce test diff

* FMT

* FullCodec implies Codec

* formatting

* Fixup bags list remote test

Co-authored-by: doordashcon <jesse.chejieh@gmail.com>
Co-authored-by: Doordashcon <90750465+Doordashcon@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* sc-finality-grandpa: use the #[from] attriute to remove boilerplate code (#11003)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* contracts: Add test to verify unique trie ids (#10914)

* Add test to verify unique trie ids

* Rename trie_seed to nonce

* Rename AccountCounter -> Nonce

* fmt

* sp-core: `full_crypto` doesn't imply `std` (#11006)

* sp-core: `full_crypto` doesn't imply `std`

This pr changes the feature set of `secp256k1` to not use `global-context` when only the
`full_crypto` is enabled. It will be slower when the `std` feature is not enabled as the context
always needs to be recreated, but that is fine.

* Update client/cli/src/arg_enums.rs

Co-authored-by: Davide Galassi <davxy@datawok.net>

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Remove unused `parent_hash` in `OverlayedChanges::into_storage_changes` (#11011)

Ref https://github.com/paritytech/substrate/pull/10922#issuecomment-1064258443

CC @cheme

* iris ledger pallet -> transfer not yet functioning

* sp-api: Don't be dirty (#11015)

Ensure that the sp api macros don't use functions without providing the full path to the function.
This hygiene ensures that we don't actually try to call a method of an imported trait for example.

* unlock and transfer works

* Update clap to the latest version (#11017)

* Update clap to the latest version

Besides that it also removes some `structopt` leftovers from some docs.

* Fix compile errors

* More fixes

* Move scripts used in CI to the new location (#11008)

Move scripts used in CI to the new location - **./scripts/ci/**

* Move github scripts

* Move more files

* Move ci scripts and fix dependencies

* Update docs/node-template-release.md

Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>

* Remove Cargo.lock

* Apply suggestions from code review

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Make more paths uniform

Co-authored-by: João Paulo Silva de Souza <77391175+joao-paulo-parity@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Move `sp-npos-elections-solution-type` to `frame-election-provider-support` (#11016)

* Move `sp-npos-elections-solution-type`
to `frame-election-provider-support`
First stab at it, will need to amend some more stuff

* Fixing tests

* Fixing tests

* Fixing cargo.toml for std configuration

* fmt

* Committing suggested changes
renaming, and re exporting macro.

* Removing unneeded imports

* shell.nix: Update to a newer nightly (#11028)

* update chain ext

* update naming, formatting, start tests

* Update lockfile (#11035)

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Additional `benchmark-storage` flags (#11004)

* Fix typos

* Enable overwriting handlebars template

* Optionally name json output or disable json altogether

* Don't write to json by default

* Include block id in handlebars output

* Include warmups for write benchmarks

* PR comments

* Drop unnecessary file extension

* Use more appropriate types

* Use more appropriate error message

* More use of more appropriate types

* Rework write benchmark warmups

* Run same benchmark for both read and write

* SharedData: Update locks to mention possible deadlocks (#11034)

* SharedData: Update locks to mention possible deadlocks

* Update `Cargo.lock`

* Stabilize `seal_delegate_call` (#11037)

* add tests

* re-enumerate chain ext func ids

* Bump names from 0.12.0 to 0.13.0 (#11047)

Bumps [names](https://github.com/fnichol/names) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/fnichol/names/releases)
- [Changelog](https://github.com/fnichol/names/blob/main/CHANGELOG.md)
- [Commits](https://github.com/fnichol/names/compare/v0.12.0...v0.13.0)

---
updated-dependencies:
- dependency-name: names
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* State migration rpc (#10981)

* setting flag

* flag in storage struct

* fix flagging to access and insert.

* added todo to fix

* also missing serialize meta to storage proof

* extract meta.

* Isolate old trie layout.

* failing test that requires storing in meta when old hash scheme is used.

* old hash compatibility

* Db migrate.

* runing tests with both states when interesting.

* fix chain spec test with serde default.

* export state (missing trie function).

* Pending using new branch, lacking genericity on layout resolution.

* extract and set global meta

* Update to branch 4

* fix iterator with root flag (no longer insert node).

* fix trie root hashing of root

* complete basic backend.

* Remove old_hash meta from proof that do not use inner_hashing.

* fix trie test for empty (force layout on empty deltas).

* Root update fix.

* debug on meta

* Use trie key iteration that do not include value in proofs.

* switch default test ext to use inner hash.

* small integration test, and fix tx cache mgmt in ext.
test  failing

* Proof scenario at state-machine level.

* trace for db upgrade

* try different param

* act more like iter_from.

* Bigger batches.

* Update trie dependency.

* drafting codec changes and refact

* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.

* alt hashing only for branch with value.

* fix trie tests

* Hash of value include the encoded size.

* removing fields(broken)

* fix trie_stream to also include value length in inner hash.

* triedbmut only using alt type if inner hashing.

* trie_stream to also only use alt hashing type when actually alt hashing.

* Refactor meta state, logic should work with change of trie treshold.

* Remove NoMeta variant.

* Remove state_hashed trigger specific functions.

* pending switching to using threshold, new storage root api does not
make much sense.

* refactoring to use state from backend (not possible payload changes).

* Applying from previous state

* Remove default from storage, genesis need a special build.

* rem empty space

* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.

* fix compilation

* Right …
ark0f pushed a commit to gear-tech/substrate that referenced this pull request Feb 27, 2023
* pallet whitelist

* refactor a bit

* fmt

* address audit

* improve tests

* return Ok + refund

* add test for dispatching failing

* add dispatch_whitelisted_call_with_preimage

* fmt

* better name

* Consume all data on decode

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add error docs

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove phantom data

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use rust 2021

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update crate features

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make compile

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Bump Preimage max size

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Parity Bot <admin@parity.io>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D1-audited 👍 PR contains changes to fund-managing logic that has been properly reviewed and externally audited
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants