Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Commit

Permalink
Decouple Staking and Election - Part 3: Signed Phase (#7910)
Browse files Browse the repository at this point in the history
* Base features and traits.

* pallet and unsigned phase

* add signed phase.

* remove comments

* Undo bad formattings.

* some formatting cleanup.

* Small self-cleanup.

* Add todo

* Make it all build

* self-review

* Some doc tests.

* Some changes from other PR

* Fix session test

* Update bin/node/runtime/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Fix name.

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* typos and verbiage

* no glob imports in signed.rs

* meaningful generic type parameters for SignedSubmission

* dedup feasibility check weight calculation

* simplify/optimize fn insert_submission

* tests: remove glob, cause to build without error

* use sp_std::vec::Vec

* maintain invariant within fn insert_submission

* fix accidentally ordering the list backward

* intentionally order the list in reverse

* get rid of unused import

* ensure signed submissions are cleared in early elect

* finalize the signed phase when appropriate

- ensure we don't leave storage lying around, even if elect called prematurely
- test that proposition
- disable the unsigned phase if a viable solution from the signed phase exists
- ensure signed phase finalization weight is accounted for

* resolve dispatch error todo

* update assumptions in submit benchmark

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* line length

* make a few more things pub

* restore missing import

* update ui test output

* update tests from master branch

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* remove duplicate definitions

* remove signed reward factor due to its attack potential

* Update frame/election-provider-multi-phase/src/signed.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* remove SignedRewardMax; no longer necessary

* compute the encoded size without actually encoding

* remove unused PostInfo

* pub use some stuff

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* ensure `pub use` things are in fact `pub`

* add event information: was another solution ejected to make room

* unconditionally run the unsigned phase even if signed was successful

* remove dead test code

* meaningful witness data name

* use errors instead of defensive `unwrap_or_default`

* get rid of a log message redundant with an event

* saturating math

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* import Saturating

* mv `fn submit` to end of call

* add log line

* Use a better data structure for SignedSubmissions instead of Vec (#8933)

* Remove: (#8748)

* `NetworkStatusSinks`
* `sc_service::SpawnTasksParams::network_status_sinks`

Also:
* `sc_service::build_network()` does not return `network_status_sinks`

* CI: fix simnet trigger (#8927)

* CI: chore

* CI: pin simnet version

* More sc-service config reexports (#8887)

* Reexport ExecutionStrategies and ExecutionStrategy

* Reexport more of the network

* Reexport the ExecutionStrategy as it's used within ExecutionStrategies

* Fix check runtime CI (#8930)

* Fix check_runtime.sh script

* contracts: Remove confusing "Related Modules" doc

* Bump parity-wasm and pwasm-utils to the newest versions everywhere (#8928)

* BROKEN: convert SignedSubmissions to BoundedBTreeSet

Eventually, once it works, this change should improve overall performance.
However, in the meantime, the trait bounds aren't playing nicely, and
this is turning into too much of a pain to handle right now as part of
/#7910. We can take care of it later.

* Simple `MaxBoundedLen` Implementations (#8793)

* implement max_values + storages info

* some formatting + doc

* sudo sanity check

* timestamp

* assets (not working)

* fix assets

* impl for proxy

* update balances

* rename StoragesInfo -> PalletStorageInfo

* merge both StorageInfoTrait and PalletStorageInfo

I think it is more future proof. In the future some storage could make
use of multiple prefix. Like one to store how much value has been
inserted, etc...

* Update frame/support/procedural/src/storage/parse.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Update frame/support/procedural/src/storage/storage_struct.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Fix max_size using hasher information

hasher now expose `max_len` which allows to computes their maximum len.
For hasher without concatenation, it is the size of the hash part,
for hasher with concatenation, it is the size of the hash part + max
encoded len of the key.

* fix tests

* fix ui tests

* Move `MaxBoundedLen` into its own crate (#8814)

* move MaxEncodedLen into its own crate

* remove MaxEncodedLen impl from frame-support

* add to assets and balances

* try more fixes

* fix compile

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* nits

* fix compile

* line width

* fix max-values-macro merge

* Add some derive, needed for test and other purpose

* use weak bounded vec in some cases

* Update lib.rs

* move max-encoded-len crate

* fix

* remove app crypto for now

* width

* Revert "remove app crypto for now"

This reverts commit 73623e9.

* unused variable

* more unused variables

* more fixes

* Add #[max_encoded_len_crate(...)] helper attribute

The purpose of this attribute is to reduce the surface area of
max_encoded_len changes. Crates deriving `MaxEncodedLen` do not
need to add it to `Cargo.toml`; they can instead just do

```rust
\#[derive(Encode, MaxEncodedLen)]
\#[max_encoded_len_crate(frame_support::max_encoded_len)]
struct Example;
```

* fix a ui test

* use #[max_encoded_len_crate(...)] helper in app_crypto

* remove max_encoded_len import where not necessary

* update lockfile

* fix ui test

* ui

* newline

* fix merge

* try fix ui again

* Update max-encoded-len/derive/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* extract generate_crate_access_2018

* Update lib.rs

* compiler isnt smart enough

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <peter.r.goodspeedniklaus@gmail.com>

* remove duplicate Issued/Burned events (#8935)

* weather -> whether (#8938)

* make remote ext use batch ws-client (#8916)

* make remote ext use batch ws-client

* Add debug log for key length

* better assertions

* new sanity_checl

* try and make it work with batch

* update test

* remove exctra uri

* add missing at

* remove unused rpc stuff

* improve

Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>

* Make `Schedule` fields public to allow for customization (#8924)

* Make `Schedule` fields public for customization

* Fix doc typo

Co-authored-by: Andrew Jones <ascjones@gmail.com>

Co-authored-by: Andrew Jones <ascjones@gmail.com>

* Session key should be settable at genesis even for non-endowed accounts (#8942)

* Session key should be settable at genesis even for non-endowed accounts

* Docs

* Migrate pallet-scored-pool to pallet attribute macro (#8825)

* Migrate pallet-scored-pool to pallet attribute macro.

* Remove dummy event.

* Apply review suggestions.

* Bump retain_mut from 0.1.2 to 0.1.3 (#8951)

Bumps [retain_mut](https://github.com/upsuper/retain_mut) from 0.1.2 to 0.1.3.
- [Release notes](https://github.com/upsuper/retain_mut/releases)
- [Commits](upsuper/retain_mut@v0.1.2...v0.1.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Use correct CreateInherentDataProviders impl for manual seal (#8852)

* use correct CreateInherentDataProviders impl for manual seal

* add babe inherent provider

* move client into factory fn

* Refactor code a little bit (#8932)

* Optimize `next_storage_key` (#8956)

* Optimize `next_storage_key`

- Do not rely on recursion
- Use an iterator over the overlay to not always call the same method

* Fix bug

* Add deserialize for TransactionValidityError in std. (#8961)

* Add deserialize for TransactionValidityError in std.

* Fix derives

* Bump getrandom from 0.2.2 to 0.2.3 (#8952)

Bumps [getrandom](https://github.com/rust-random/getrandom) from 0.2.2 to 0.2.3.
- [Release notes](https://github.com/rust-random/getrandom/releases)
- [Changelog](https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md)
- [Commits](rust-random/getrandom@v0.2.2...v0.2.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Allow usage of path in construct_runtime! (#8801)

* Allow usage of path in construct_runtime!

* Fix whitespace

* Fix whitespace

* Make expand_runtime_metadata accept slice instead of Iterator

* Include Call and Event in construct_runtime for testing

* Migrate impl_outer_event to proc macro

* Fix integrity_test_works

* Update UI test expectations

* Factor in module path while generating enum variant or fn names

* Use ParseStream::lookahead for more helpful error messages

* Remove generating outer_event_metadata

* Ensure pallets with different paths but same last path segment can coexist

* Remove unnecessary generated function

* Migrate decl_outer_config to proc macro

* Add default_filter test for expand_outer_origin

* Allow crate, self and super keywords to appear in pallet path

* Add UI test for specifying empty pallet paths in construct_runtime

* Reduce cargo doc warnings (#8947)

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update wasmtime to 0.27 (#8913)

* Update wasmtime to 0.27

A couple of notes:

- Now we are fair about unsafeness of runtime creation via an compiled artifact.
  This change was prompted by the change in wasmtime which made
  `deserialize` rightfully unsafe. Now `CodeSupplyMode` was hidden and
  the `create_runtime` now takes the blob again and there is now a new
  fn for creating a runtime with a compiled artifact.
- This is a big change for wasmtime. They switched to the modern backend
  for code generation. While this can bring performance improvements, it
  can also introduce some problems. In fact, 0.27 fixed a serious issue
  that could lead to sandbox escape. Hence we need a proper burn in.

This would require a change to PVF validation host as well.

* Filter regalloc logging

* Spellling corrections (no code changes) (#8971)

* Spelling corrections

* As this might break let's do as a separate PR

* Dependabot use correct label (#8973)

* Inject hashed prefix for remote-ext (#8960)

* Inject for remote-ext

* Update utils/frame/remote-externalities/src/lib.rs

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* Update utils/frame/remote-externalities/src/lib.rs

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* Use `SpawnTaskHandle`s for spawning tasks in the tx pool  (#8958)

* Remove futures-diagnose

* Use `SpawnTaskHandle`s for spawning tasks in the tx pool

* Box the spawner

* Fix tests

* Use the testing task executor

* Do not spend time on verifying the signatures before calling Runtime (#8980)

* Revert "Use `SpawnTaskHandle`s for spawning tasks in the tx pool  (#8958)" (#8983)

This reverts commit bfef07c.

* Uniques: An economically-secure basic-featured NFT pallet (#8813)

* Uniques: An economically-secure basic-featured NFT pallet

* force_transfer

* freeze/thaw

* team management

* approvals

* Fixes

* force_asset_status

* class_metadata

* instance metadata

* Fixes

* use nmap

* Fixes

* class metadata has information field

* Intiial mock/tests and a fix

* Remove impl_non_fungibles

* Docs

* Update frame/uniques/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update frame/uniques/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update frame/uniques/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update frame/uniques/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Reserve, don't transfer.

* Fixes

* Tests

* Tests

* refresh_deposit

* Tests and proper handling of metdata destruction

* test burn

* Tests

* Update impl_fungibles.rs

* Initial benchmarking

* benchmark

* Fixes

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_uniques --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/uniques/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Attributes

* Attribute metadata

* Fixes

* Update frame/uniques/README.md

* Docs

* Docs

* Docs

* Simple metadata

* Use BoundedVec

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_uniques --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/uniques/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Update frame/uniques/src/lib.rs

Co-authored-by: Lohann Paterno Coutinho Ferreira <developer@lohann.dev>

* Update frame/uniques/src/lib.rs

Co-authored-by: Lohann Paterno Coutinho Ferreira <developer@lohann.dev>

* Update frame/uniques/src/lib.rs

Co-authored-by: Lohann Paterno Coutinho Ferreira <developer@lohann.dev>

* Update frame/uniques/src/lib.rs

Co-authored-by: Lohann Paterno Coutinho Ferreira <developer@lohann.dev>

* Update frame/uniques/src/lib.rs

Co-authored-by: Lohann Paterno Coutinho Ferreira <developer@lohann.dev>

* Fixes

* Update frame/uniques/README.md

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update frame/uniques/README.md

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update frame/uniques/README.md

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Docs

* Bump

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Lohann Paterno Coutinho Ferreira <developer@lohann.dev>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update WeakBoundedVec's remove and swap_remove (#8985)

Co-authored-by: Boiethios <felix-dev@daudre-vignier.fr>

* Convert another instance of Into impl to From in the macros (#8986)

* Convert another instance of Into impl to From in the macros

* Convert another location

* also fix bounded vec (#8987)

* fix most compiler errors

Mostly the work so far has been in tracking down where precisely
to insert appropriate trait bounds, and updating `fn insert_submission`.
However, there's still a compiler error remaining:

```
error[E0275]: overflow evaluating the requirement `Compact<_>: Decode`
  |
  = help: consider adding a `#![recursion_limit="256"]` attribute to your crate (`pallet_election_provider_multi_phase`)
  = note: required because of the requirements on the impl of `Decode` for `Compact<_>`
  = note: 126 redundant requirements hidden
  = note: required because of the requirements on the impl of `Decode` for `Compact<_>`
```

Next up: figure out how we ended up with that recursive bound, and fix it.

* extract type SignedSubmissionsOf<T>

Weirdly, we still encounter the recursive trait definition error here,
despite removing the trait bounds. Something weird is happening.

* impl Decode bounds on BoundedBTreeMap/Set on T, not predecessor

Otherwise, Rust gets confused and decides that the trait bound is
infinitely recursive.

For that matter, it _still_ gets confused somehow and decides
that the trait bound is infinitely recursive, but at least
this should somewhat simplify the matter.

* fix recursive trait bound problem

* minor fixes

* more little fixes

* correct semantics for try_insert

* more fixes

* derive Ord for SolutionType

* tests compile

* fix most tests, rm unnecessary one

* Transactionpool: Make `ready_at` return earlier (#8995)

`ready_at` returns when we have processed the requested block. However,
on startup we already have processed the best block and there
are no transactions in the pool on startup anyway. So, we can set `updated_at`
to the best block on startup.

Besides that `ready_at` now returns early when there are no ready nor
any future transactions in the pool.

* Discard notifications if we have failed to parse handshake (#8806)

* Migrate pallet-democracy to pallet attribute macro (#8824)

* Migrate pallet-democracy to pallet attribute macro.

* Metadata fix.

* Trigger CI.

* Add ecdsa::Pair::verify_prehashed() (#8996)

* Add ecdsa::Pair::verify_prehashed()

* turn verify_prehashed() into an associated function

* add Signature::recover_prehashed()

* Non-fungible token traits (#8993)

* Non-fungible token traits

* Docs

* Fixes

* Implement non-fungible trait for Uniques

* Update frame/uniques/src/impl_nonfungibles.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update frame/uniques/src/impl_nonfungibles.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Removes unused import (#9007)

* Add Call Filter That Prevents Nested `batch_all` (#9009)

* add filter preventing nested `batch_all`

* more tests

* fix test

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_utility --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/utility/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Parity Bot <admin@parity.io>

* Transaction pool: Ensure that we prune transactions properly (#8963)

* Transaction pool: Ensure that we prune transactions properly

There was a bug in the transaction pool that we didn't pruned
transactions properly because we called `prune_known`, instead of `prune`.

This bug was introduced by:
#4629

This is required to have stale extrinsics being removed properly, so
that they don't fill up the tx pool.

* Fix compilation

* Fix benches

* ...

* Storage chain: Runtime module (#8624)

* Transaction storage runtime module

* WIP: Tests

* Tests, benchmarks  and docs

* Made check_proof mandatory

* Typo

* Renamed a crate

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Added weight for on_finalize

* Fixed counter mutations

* Reorganized tests

* Fixed build

* Update for the new inherent API

* Reworked for the new inherents API

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Store transactions in a Vec

* Added FeeDestination

* Get rid of constants

* Fixed node runtime build

* Fixed benches

* Update frame/transaction-storage/src/lib.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* more useful error message (#9014)

* Named reserve (#7778)

* add NamedReservableCurrency

* move currency related trait and types into a new file

* implement NamedReservableCurrency

* remove empty reserves

* Update frame/support/src/traits.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* fix build

* bump year

* add MaxReserves

* repatriate_reserved_named should put reserved fund into named reserved

* add tests

* add some docs

* fix warning

* Update lib.rs

* fix test

* fix test

* fix

* fix

* triggier CI

* Move NamedReservableCurrency.

* Use strongly bounded vec for reserves.

* Fix test.

* remove duplicated file

* trigger CI

* Make `ReserveIdentifier` assosicated type

* add helpers

* make ReserveIdentifier assosicated type

* fix

* update

* trigger CI

* Apply suggestions from code review

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* trigger CI

* Apply suggestions from code review

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Gavin Wood <i@gavwood.com>
Co-authored-by: Shaun Wang <spxwang@gmail.com>

* update ss58 type to u16 (#8955)

* Fixed build (#9021)

* Bump parity-db (#9024)

* consensus: handle justification sync for blocks authored locally (#8698)

* consensus: add trait to control justification sync process

* network: implement JustificationSyncLink for NetworkService

* slots: handle justification sync in slot worker

* babe: fix slot worker instantiation

* aura: fix slot worker instantiation

* pow: handle justification sync in miner

* babe: fix tests

* aura: fix tests

* node: fix compilation

* node-template: fix compilation

* consensus: rename justification sync link parameter

* aura: fix test compilation

* consensus: slots: move JustificationSyncLink out of on_slot

* arithmetic: fix PerThing pow (#9030)

* arithmetic: add failing test for pow

* arithmetic: fix PerThing::pow

* Revert back to previous optimisations

Co-authored-by: Gav Wood <gavin@parity.io>

* Compact proof utilities in sp_trie. (#8574)

* validation extension in sp_io

* need paths

* arc impl

* missing host function in executor

* io to pkdot

* decode function.

* encode primitive.

* trailing tab

* multiple patch

* fix child trie logic

* restore master versionning

* bench compact proof size

* trie-db 22.3 is needed

* line width

* split line

* fixes for bench (additional root may not be needed as original issue was
with empty proof).

* revert compact from block size calculation.

* New error type for compression.

* Adding test (incomplete (failing)).
Also lacking real proof checking (no good primitives in sp-trie crate).

* There is currently no proof recording utility in sp_trie, removing
test.

* small test of child root in proof without a child proof.

* remove empty test.

* remove non compact proof size

* Missing revert.

* proof method to encode decode.

* Don't inlucde nominaotrs that back no one in the snapshot. (#9017)

* fix all_in_one test which had a logic error

* use sp_std, not std

* Periodically call `Peerset::alloc_slots` on all sets (#9025)

* Periodically call alloc_slots on all slots

* Add test

* contracts: Add new `seal_call` that offers new features (#8909)

* Add new `seal_call` that offers new features

* Fix doc typo

Co-authored-by: Michael Müller <michi@parity.io>

* Fix doc typos

Co-authored-by: Michael Müller <michi@parity.io>

* Fix comment on assert

* Update CHANGELOG.md

Co-authored-by: Michael Müller <michi@parity.io>

* fix unreserve_all_named (#9042)

* Delete legacy runtime metadata macros (#9043)

* `rpc-http-threads` cli arg (#8890)

* Add optional `rpc-http-threads` cli arg

* Update `http::ServerBuilder`threads

* allow inserting equal items into bounded map/set

* refactor: only load one solution at a time

This increases the database read load, because we read one solution
at a time. On the other hand, it substantially decreases the overall
memory load, because we _only_ read one solution at a time instead
of reading all of them.

* Emit `Bonded` event when rebonding (#9040)

* Emit `Bonded` event when rebonding

* fix borrow checker

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Parity Bot <admin@parity.io>

* fix tests

* Revert "Merge remote-tracking branch 'origin/master' into prgn-election-provider-multi-phase-bounded-btree-set-signed-submissions"

This reverts commit de92b1e, reversing
changes made to dae31f2.

* only derive debug when std

* write after check

* SignedSubmissions doesn't ever modify storage until .put()

This makes a true check-before-write pattern possible.

* REVERT ME: demo that Drop impl doesn't work

* Revert "REVERT ME: demo that Drop impl doesn't work"

This reverts commit 3317a4b.

* doc note about decode_len

* rename get_submission, take_submission for clarity

* add test which fails for current incorrect behavior

* inline fn insert_submission

This fixes a tricky check-before-write error, ensuring that we
really only ever modify anything if we have in fact succeeded.

Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: MOZGIII <mike-n@narod.ru>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Sebastian Müller <sebastian@web3.foundation>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Shaun Wang <spxwang@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Seun Lanlege <seun@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Lohann Paterno Coutinho Ferreira <developer@lohann.dev>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Boiethios <Boiethios@users.noreply.github.com>
Co-authored-by: Boiethios <felix-dev@daudre-vignier.fr>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Andreas Doerr <adoerr@users.noreply.github.com>
Co-authored-by: Dmitry Kashitsyn <korvin@deeptown.org>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Gavin Wood <i@gavwood.com>
Co-authored-by: Jakub Pánik <jakub.panik@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Michael Müller <michi@parity.io>
Co-authored-by: tgmichel <telmo@purestake.com>

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* remove duplicate weight definitions injected by benchmark bot

* check deletion overlay before getting

* clarify non-conflict between delete, insert overlays

* drain can be used wrong so is private

* update take_submission docs

* more drain improvements

* more take_submission docs

* debug assertion helps prove expectation is valid

* doc on changing SignedMaxSubmissions

* take_submission inner doc on system properties

* Apply suggestions from code review

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* get SolutionOrSnapshotSize out of the loop

Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com>

* doc which items comprise `SignedSubmissions`

* add doc about index as unique identifier

* Add debug assertions to prove drain worked properly

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* replace take_submission with swap_out_submission

* use a match to demonstrate all cases from signed_submissions.insert

* refactor signed_submissions.insert return type

* prettify test assertion

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* improve docs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* add tests that finalize_signed_phase is idempotent

* add some debug assertions to guard against misuse of storage

* log internal logic errors instead of panicing

* don't store the reward with each signed submission

The signed reward base can be treated as a constant. It can in principle
change, but even if it's updated in the middle of an election, it's
appropriate to use the current value for the winner.

* emit Rewarded, Slashed events as appropriate

Makes it easier to see who won/lost with signed submissions.

* update docs

* use a custom enum to be explicit about the outcome of insertion

* remove outdated docs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Peter Goodspeed-Niklaus <peter.r.goodspeedniklaus@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Roman Proskuryakov <humbug@deeptown.org>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: MOZGIII <mike-n@narod.ru>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Sebastian Müller <sebastian@web3.foundation>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Shaun Wang <spxwang@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Seun Lanlege <seun@parity.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Lohann Paterno Coutinho Ferreira <developer@lohann.dev>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Boiethios <Boiethios@users.noreply.github.com>
Co-authored-by: Boiethios <felix-dev@daudre-vignier.fr>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Andreas Doerr <adoerr@users.noreply.github.com>
Co-authored-by: Dmitry Kashitsyn <korvin@deeptown.org>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Gavin Wood <i@gavwood.com>
Co-authored-by: Jakub Pánik <jakub.panik@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Michael Müller <michi@parity.io>
Co-authored-by: tgmichel <telmo@purestake.com>
  • Loading branch information
Show file tree
Hide file tree
Showing 11 changed files with 1,592 additions and 104 deletions.
21 changes: 18 additions & 3 deletions bin/node/runtime/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ use pallet_session::{historical as pallet_session_historical};
use sp_inherents::{InherentData, CheckInherentsResult};
use static_assertions::const_assert;
use pallet_contracts::weights::WeightInfo;
use pallet_election_provider_multi_phase::FallbackStrategy;

#[cfg(any(feature = "std", test))]
pub use sp_runtime::BuildStorage;
Expand Down Expand Up @@ -516,9 +517,14 @@ parameter_types! {
pub const SignedPhase: u32 = EPOCH_DURATION_IN_BLOCKS / 4;
pub const UnsignedPhase: u32 = EPOCH_DURATION_IN_BLOCKS / 4;

// fallback: no need to do on-chain phragmen initially.
pub const Fallback: pallet_election_provider_multi_phase::FallbackStrategy =
pallet_election_provider_multi_phase::FallbackStrategy::Nothing;
// signed config
pub const SignedMaxSubmissions: u32 = 10;
pub const SignedRewardBase: Balance = 1 * DOLLARS;
pub const SignedDepositBase: Balance = 1 * DOLLARS;
pub const SignedDepositByte: Balance = 1 * CENTS;

// fallback: no on-chain fallback.
pub const Fallback: FallbackStrategy = FallbackStrategy::Nothing;

pub SolutionImprovementThreshold: Perbill = Perbill::from_rational(1u32, 10_000);

Expand Down Expand Up @@ -559,6 +565,14 @@ impl pallet_election_provider_multi_phase::Config for Runtime {
type MinerMaxWeight = MinerMaxWeight;
type MinerMaxLength = MinerMaxLength;
type MinerTxPriority = MultiPhaseUnsignedPriority;
type SignedMaxSubmissions = SignedMaxSubmissions;
type SignedRewardBase = SignedRewardBase;
type SignedDepositBase = SignedDepositBase;
type SignedDepositByte = SignedDepositByte;
type SignedDepositWeight = ();
type SignedMaxWeight = MinerMaxWeight;
type SlashHandler = (); // burn slashes
type RewardHandler = (); // nothing to do upon rewards
type DataProvider = Staking;
type OnChainAccuracy = Perbill;
type CompactSolution = NposCompactSolution16;
Expand Down Expand Up @@ -1556,6 +1570,7 @@ impl_runtime_apis! {
add_benchmark!(params, batches, pallet_uniques, Uniques);
add_benchmark!(params, batches, pallet_utility, Utility);
add_benchmark!(params, batches, pallet_vesting, Vesting);
add_benchmark!(params, batches, pallet_election_provider_multi_phase, ElectionProviderMultiPhase);

if batches.is_empty() { return Err("Benchmark not found for this pallet.".into()) }
Ok(batches)
Expand Down
105 changes: 93 additions & 12 deletions frame/election-provider-multi-phase/src/benchmarking.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

use super::*;
use crate::{Pallet as MultiPhase, unsigned::IndexAssignmentOf};
use frame_benchmarking::impl_benchmark_test_suite;
use frame_benchmarking::{account, impl_benchmark_test_suite};
use frame_support::{assert_ok, traits::OnInitialize};
use frame_system::RawOrigin;
use rand::{prelude::SliceRandom, rngs::SmallRng, SeedableRng};
Expand Down Expand Up @@ -57,7 +57,7 @@ fn solution_with_size<T: Config>(
let targets: Vec<T::AccountId> =
(0..size.targets).map(|i| frame_benchmarking::account("Targets", i, SEED)).collect();

let mut rng = SmallRng::seed_from_u64(SEED as u64);
let mut rng = SmallRng::seed_from_u64(SEED.into());

// decide who are the winners.
let winners = targets
Expand Down Expand Up @@ -176,6 +176,39 @@ frame_benchmarking::benchmarks! {
assert!(<MultiPhase<T>>::current_phase().is_unsigned());
}

finalize_signed_phase_accept_solution {
let receiver = account("receiver", 0, SEED);
let initial_balance = T::Currency::minimum_balance() * 10u32.into();
T::Currency::make_free_balance_be(&receiver, initial_balance);
let ready: ReadySolution<T::AccountId> = Default::default();
let deposit: BalanceOf<T> = 10u32.into();
let reward: BalanceOf<T> = 20u32.into();

assert_ok!(T::Currency::reserve(&receiver, deposit));
assert_eq!(T::Currency::free_balance(&receiver), initial_balance - 10u32.into());
}: {
<MultiPhase<T>>::finalize_signed_phase_accept_solution(ready, &receiver, deposit, reward)
} verify {
assert_eq!(T::Currency::free_balance(&receiver), initial_balance + 20u32.into());
assert_eq!(T::Currency::reserved_balance(&receiver), 0u32.into());
}

finalize_signed_phase_reject_solution {
let receiver = account("receiver", 0, SEED);
let initial_balance = T::Currency::minimum_balance().max(One::one()) * 10u32.into();
let deposit: BalanceOf<T> = 10u32.into();
T::Currency::make_free_balance_be(&receiver, initial_balance);
assert_ok!(T::Currency::reserve(&receiver, deposit));

assert_eq!(T::Currency::free_balance(&receiver), initial_balance - 10u32.into());
assert_eq!(T::Currency::reserved_balance(&receiver), 10u32.into());
}: {
<MultiPhase<T>>::finalize_signed_phase_reject_solution(&receiver, deposit)
} verify {
assert_eq!(T::Currency::free_balance(&receiver), initial_balance - 10u32.into());
assert_eq!(T::Currency::reserved_balance(&receiver), 0u32.into());
}

on_initialize_open_unsigned_without_snapshot {
// need to assume signed phase was open before
<MultiPhase<T>>::on_initialize_open_signed().unwrap();
Expand Down Expand Up @@ -227,16 +260,51 @@ frame_benchmarking::benchmarks! {
assert!(<MultiPhase<T>>::snapshot().is_some());
}

submit {
let c in 1 .. (T::SignedMaxSubmissions::get() - 1);

// the solution will be worse than all of them meaning the score need to be checked against
// ~ log2(c)
let solution = RawSolution {
score: [(10_000_000u128 - 1).into(), 0, 0],
..Default::default()
};

MultiPhase::<T>::on_initialize_open_signed().expect("should be ok to start signed phase");
<Round<T>>::put(1);

let mut signed_submissions = SignedSubmissions::<T>::get();
for i in 0..c {
let solution = RawSolution {
score: [(10_000_000 + i).into(), 0, 0],
..Default::default()
};
let signed_submission = SignedSubmission { solution, ..Default::default() };
signed_submissions.insert(signed_submission);
}
signed_submissions.put();

let caller = frame_benchmarking::whitelisted_caller();
T::Currency::make_free_balance_be(&caller, T::Currency::minimum_balance() * 10u32.into());

}: _(RawOrigin::Signed(caller), solution, c)
verify {
assert!(<MultiPhase<T>>::signed_submissions().len() as u32 == c + 1);
}

submit_unsigned {
// number of votes in snapshot.
let v in (T::BenchmarkingConfig::VOTERS[0]) .. T::BenchmarkingConfig::VOTERS[1];
// number of targets in snapshot.
let t in (T::BenchmarkingConfig::TARGETS[0]) .. T::BenchmarkingConfig::TARGETS[1];
// number of assignments, i.e. compact.len(). This means the active nominators, thus must be
// a subset of `v` component.
let a in (T::BenchmarkingConfig::ACTIVE_VOTERS[0]) .. T::BenchmarkingConfig::ACTIVE_VOTERS[1];
let a in
(T::BenchmarkingConfig::ACTIVE_VOTERS[0]) .. T::BenchmarkingConfig::ACTIVE_VOTERS[1];
// number of desired targets. Must be a subset of `t` component.
let d in (T::BenchmarkingConfig::DESIRED_TARGETS[0]) .. T::BenchmarkingConfig::DESIRED_TARGETS[1];
let d in
(T::BenchmarkingConfig::DESIRED_TARGETS[0]) ..
T::BenchmarkingConfig::DESIRED_TARGETS[1];

let witness = SolutionOrSnapshotSize { voters: v, targets: t };
let raw_solution = solution_with_size::<T>(witness, a, d);
Expand All @@ -249,7 +317,8 @@ frame_benchmarking::benchmarks! {
let encoded_call = <Call<T>>::submit_unsigned(raw_solution.clone(), witness).encode();
}: {
assert_ok!(<MultiPhase<T>>::submit_unsigned(RawOrigin::None.into(), raw_solution, witness));
let _decoded_snap = <RoundSnapshot<T::AccountId> as Decode>::decode(&mut &*encoded_snapshot).unwrap();
let _decoded_snap = <RoundSnapshot<T::AccountId> as Decode>::decode(&mut &*encoded_snapshot)
.unwrap();
let _decoded_call = <Call<T> as Decode>::decode(&mut &*encoded_call).unwrap();
} verify {
assert!(<MultiPhase<T>>::queued_solution().is_some());
Expand All @@ -263,13 +332,17 @@ frame_benchmarking::benchmarks! {
let t in (T::BenchmarkingConfig::TARGETS[0]) .. T::BenchmarkingConfig::TARGETS[1];
// number of assignments, i.e. compact.len(). This means the active nominators, thus must be
// a subset of `v` component.
let a in (T::BenchmarkingConfig::ACTIVE_VOTERS[0]) .. T::BenchmarkingConfig::ACTIVE_VOTERS[1];
let a in
(T::BenchmarkingConfig::ACTIVE_VOTERS[0]) .. T::BenchmarkingConfig::ACTIVE_VOTERS[1];
// number of desired targets. Must be a subset of `t` component.
let d in (T::BenchmarkingConfig::DESIRED_TARGETS[0]) .. T::BenchmarkingConfig::DESIRED_TARGETS[1];
let d in
(T::BenchmarkingConfig::DESIRED_TARGETS[0]) ..
T::BenchmarkingConfig::DESIRED_TARGETS[1];
// Subtract this percentage from the actual encoded size
let f in 0 .. 95;

// Compute a random solution, then work backwards to get the lists of voters, targets, and assignments
// Compute a random solution, then work backwards to get the lists of voters, targets, and
// assignments
let witness = SolutionOrSnapshotSize { voters: v, targets: t };
let RawSolution { compact, .. } = solution_with_size::<T>(witness, a, d);
let RoundSnapshot { voters, targets } = MultiPhase::<T>::snapshot().unwrap();
Expand Down Expand Up @@ -313,7 +386,11 @@ frame_benchmarking::benchmarks! {
} verify {
let compact = CompactOf::<T>::try_from(index_assignments.as_slice()).unwrap();
let encoding = compact.encode();
log!(trace, "encoded size prediction = {}", encoded_size_of(index_assignments.as_slice()).unwrap());
log!(
trace,
"encoded size prediction = {}",
encoded_size_of(index_assignments.as_slice()).unwrap(),
);
log!(trace, "actual encoded size = {}", encoding.len());
assert!(encoding.len() <= desired_size);
}
Expand All @@ -326,9 +403,12 @@ frame_benchmarking::benchmarks! {
let t in (T::BenchmarkingConfig::TARGETS[0]) .. T::BenchmarkingConfig::TARGETS[1];
// number of assignments, i.e. compact.len(). This means the active nominators, thus must be
// a subset of `v` component.
let a in (T::BenchmarkingConfig::ACTIVE_VOTERS[0]) .. T::BenchmarkingConfig::ACTIVE_VOTERS[1];
let a in
(T::BenchmarkingConfig::ACTIVE_VOTERS[0]) .. T::BenchmarkingConfig::ACTIVE_VOTERS[1];
// number of desired targets. Must be a subset of `t` component.
let d in (T::BenchmarkingConfig::DESIRED_TARGETS[0]) .. T::BenchmarkingConfig::DESIRED_TARGETS[1];
let d in
(T::BenchmarkingConfig::DESIRED_TARGETS[0]) ..
T::BenchmarkingConfig::DESIRED_TARGETS[1];

let size = SolutionOrSnapshotSize { voters: v, targets: t };
let raw_solution = solution_with_size::<T>(size, a, d);
Expand All @@ -340,7 +420,8 @@ frame_benchmarking::benchmarks! {
let encoded_snapshot = <MultiPhase<T>>::snapshot().unwrap().encode();
}: {
assert_ok!(<MultiPhase<T>>::feasibility_check(raw_solution, ElectionCompute::Unsigned));
let _decoded_snap = <RoundSnapshot<T::AccountId> as Decode>::decode(&mut &*encoded_snapshot).unwrap();
let _decoded_snap = <RoundSnapshot<T::AccountId> as Decode>::decode(&mut &*encoded_snapshot)
.unwrap();
}
}

Expand Down
8 changes: 4 additions & 4 deletions frame/election-provider-multi-phase/src/helpers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,13 @@ pub fn generate_voter_cache<T: Config>(
cache
}

/// Create a function the returns the index a voter in the snapshot.
/// Create a function that returns the index of a voter in the snapshot.
///
/// The returning index type is the same as the one defined in `T::CompactSolution::Voter`.
///
/// ## Warning
///
/// The snapshot must be the same is the one used to create `cache`.
/// Note that this will represent the snapshot data from which the `cache` is generated.
pub fn voter_index_fn<T: Config>(
cache: &BTreeMap<T::AccountId, usize>,
) -> impl Fn(&T::AccountId) -> Option<CompactVoterIndexOf<T>> + '_ {
Expand All @@ -78,7 +78,7 @@ pub fn voter_index_fn_owned<T: Config>(
///
/// ## Warning
///
/// The snapshot must be the same is the one used to create `cache`.
/// Note that this will represent the snapshot data from which the `cache` is generated.
pub fn voter_index_fn_usize<T: Config>(
cache: &BTreeMap<T::AccountId, usize>,
) -> impl Fn(&T::AccountId) -> Option<usize> + '_ {
Expand All @@ -103,7 +103,7 @@ pub fn voter_index_fn_linear<T: Config>(
}
}

/// Create a function the returns the index to a target in the snapshot.
/// Create a function that returns the index of a target in the snapshot.
///
/// The returned index type is the same as the one defined in `T::CompactSolution::Target`.
///
Expand Down
Loading

0 comments on commit c44e5d6

Please sign in to comment.