Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

BABE slot and epoch event notifications #6563

Merged
merged 15 commits into from
Jul 30, 2020
Merged

BABE slot and epoch event notifications #6563

merged 15 commits into from
Jul 30, 2020

Conversation

sorpaas
Copy link
Member

@sorpaas sorpaas commented Jul 2, 2020

rel paritytech/polkadot#1327

This adds slot notification stream support for BABE, of type (slot_number, epoch_descriptor). The receiver can check whether two epoch_descriptor equals to know whether an epoch change has happened. The stream lives on BabeWorker, returned when calling start_babe. BabeWorker implements Future<Output=()> so it won't affect current interface, and users can call slot_notification_stream function on that struct to grab handles of event receivers, before passing BabeWorker to event loop.

@sorpaas sorpaas added A0-please_review Pull request needs code review. B3-apinoteworthy C1-low PR touches the given topic and has a low impact on builders. labels Jul 2, 2020
@sorpaas sorpaas requested a review from andresilva as a code owner July 2, 2020 20:47
@sorpaas sorpaas requested a review from rphmeier July 2, 2020 20:48
/// Worker for Babe which implements `Future<Output=()>`. This must be polled.
pub struct BabeWorker<B: BlockT> {
inner: Pin<Box<dyn futures::Future<Output=()>>>,
slot_notification_sinks: Arc<Mutex<Vec<TracingUnboundedSender<(u64, ViableEpochDescriptor<B::Hash, NumberFor<B>, Epoch>)>>>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to avoid unbounded channels? We are making an effort to remove all of them from the codebase and it would be best not to add more.

@sorpaas sorpaas added A3-in_progress Pull request is in progress. No review needed at this stage. and removed A0-please_review Pull request needs code review. labels Jul 2, 2020
@sorpaas sorpaas added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Jul 13, 2020
epoch_descriptor: &ViableEpochDescriptor<B::Hash, NumberFor<B>, Epoch>,
) {
self.slot_notification_sinks.lock()
.retain_mut(|sink| sink.try_send((slot_number, epoch_descriptor.clone())).is_ok());
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs never explicitly mentions it, but from my readings of the source code (https://docs.rs/futures-channel/0.2.0/src/futures_channel/mpsc/mod.rs.html#416-429), try_send will never block and returns error if the buffer is full.

Only skip sending the message and emit a warning, because it is recoverable.
@sorpaas
Copy link
Member Author

sorpaas commented Jul 23, 2020

companion-build fail right now due to paritytech/polkadot#1394.

@gavofyork
Copy link
Member

@andresilva @rphmeier good to go now?

Copy link
Contributor

@andresilva andresilva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes LGTM but these changes only allow sending events if we are running as a validator (although we don't need to be in the set, we just need to have the BABE worker running). Just want to make sure that's OK for the use case, for paritytech/polkadot#1327 I think it is.

}

struct BabeWorker<B: BlockT, C, E, I, SO> {
/// Worker for Babe which implements `Future<Output=()>`. This must be polled.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add a #[must_use] here?

client: Arc<C>,
block_import: Arc<Mutex<I>>,
env: E,
sync_oracle: SO,
force_authoring: bool,
keystore: KeyStorePtr,
epoch_changes: SharedEpochChanges<B, Epoch>,
slot_notification_sinks: Arc<Mutex<Vec<Sender<(u64, ViableEpochDescriptor<B::Hash, NumberFor<B>, Epoch>)>>>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we introduce a type alias for this? type SlotNotifications<B> = Arc<Mutex<Vec<Sender<(u64, ViableEpochDescriptor<B, NumberFor<B>, Epoch>)>>>> perhaps?

@sorpaas sorpaas merged commit bff302d into master Jul 30, 2020
@sorpaas sorpaas deleted the sp-stream branch July 30, 2020 15:07
dvdplm added a commit that referenced this pull request Aug 3, 2020
* na-sgx-hello-world: (158 commits)
  commit `Cargo.lock` change
  Improve Benchmark Writer: Remove Unused Components, Remove Multiply by Zero, Files Split by Pallet (#6785)
  seal: Fix and improve error reporting (#6773)
  client/network: Fix wrong metric help text (#6794)
  Bump elliptic from 6.5.2 to 6.5.3 in /.maintain/chaostest (#6791)
  Fix transaction payment runtime api (#6792)
  client/network: Expose DHT query duration to Prometheus (#6784)
  Don't close inbound notifications substreams immediately (#6781)
  Order delta before calculating the storage root (#6780)
  Allow blacklisting blocks from being finalized again after block revert (#6301)
  Fix link (#6775)
  Fix graceful shutdown skipped if future ends with error (#6769)
  Update Balances Pallet to use `WeightInfo` (#6610)
  BABE slot and epoch event notifications (#6563)
  Rename task name to stick to the default naming scheme (#6768)
  pallet-evm: add builtin support for the four basic Ethereum precompiles (#6743)
  Add "✅ Successfully mined block" log message (#6764)
  Ignore flaky test (#6767)
  Add a `DefaultQueue` type alias to remove the need to use `sp_api::TransactionFor` (#6761)
  support custom ss58addressformat in from_ss58check_with_version (#5526)
  ...
drewstone added a commit to hicommonwealth/substrate that referenced this pull request Sep 8, 2020
* Update stale docstring with 'EnsureOneOf' introduced. (#6501)

* Update stale docstring with 'EnsureOneOf' introduced.

* Apply review suggestions.

* Add auth-label-issues.yml (#6488)

* Remove /self from mandatory rpc reviews. (#6507)

* Change contract fees to MILLICENTS (#6509)

* Re-enter runtime after resetting overlay from runtime (#6513)

This still assumes that the client did not start any transactions
before calling into runtime. This is the case for benchmarking
as long as either NativeWhenPossible or AlwaysWasm exection
strategy is chosen. Using any other will result in a panic.

* Releasing rc4 – Rhinoceros (#6515)

Co-authored-by: Martin Pugh <pugh@s3kr.it>

* Fix an extra semi-colon yielding a wrong error (#6520)

* Fix an extra semi-colon yielding a wrong error

* Update client/cli/src/commands/run_cmd.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Refactor as_sub to make things clearer. (#6503)

* Refactor as_sub to make things clearer.

- `as_sub` becomes `as_alternative`
- `as_sub_limited` becomes `as_derivative`
- `as_alternative` and `as_derivative` generate a mutually exclusive set of accounts.

* Test fix

* Add test

* Fix test

* Remove `as_alternative`.

* Docs.

* [CI] Fix warning in polkadot-companion-label action (#6514)

* Remove @cecton from CODEOWNERS (#6524)

* Initial commit

Forked at: 397068219c7a43a743f3979947efec35a8e93536
Parent branch: origin/master

* Remove @cecton from CODEOWNERS

* Impl From<UncheckedExtrinsic> for OpaqueExtrinsic (#6522)

* Implement Contains for pallet-membership (#6518)

* implement Contains for pallet-membership

* bump version

* Increase the limit for the maximum size of the telemetry name (#6523)

* Increase the limit for the maximum size of the telemetry name

* Fix test

* Fix some broken benchmarks (#6528)

* Implement `()` for `Happened` (#6529)

* Allow `retract_tip` on `tip_new` (#6511)

* Allow `retract_tip` on `tip_new`

* initial migration code

* test migration

* make pub

* bump spec

* Update to libp2p v0.20.1 (#6465)

* Update to libp2p-0.20.0

* Update to `libp2p-0.20.1`.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Remove unecessary &mut in call argument (#6540)

* feat: Allocate ss58 address format to DataHighway (#6530)

* Fix tx-pool returning the same transaction multiple times (#6535)

* Fix tx-pool returning the same transaction multiple times

This fixes a bug that lead to returning the same transaction multiple
times when iterating the `ready` iterator. Internally the transaction
was kept in the `best` list and could be duplicated in that list be
re-inserting it again. This `best` list is using a `TransactionRef`
which internally uses a `insertion_id`. This `insertion_id` could lead
to the same transaction being inserted multiple times into the `best`
list.

* Update client/transaction-pool/src/testing/pool.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Remove the service, replacing it with a struct of individual chain components (#6352)

* WIP

* Making progress

* Almost ready

* Get service tests compiling

* Fix node screenshot

* Line widths

* Fix node cli tests

* Fix node cli warning

* ChainComponents -> ServiceComponents, fix tests

* make spawn_handle public

* Remove spawnnamed impl for taskmanager

* Move the keep alive stuff to the task manager

* Move the telemetry, base path, rpc keep_alive to the service builder

* Make the task manager keep alive an internal detail

* Rewrite the browser start_client future

* Remove run_node etc

* Revert my personal changes to browser-demo/build.sh

* use |config|

* Add a runtime_version function to SubstrateCli

* Reexport role and runtime version from sc cli

* Update Cargo.lock

* runtime_version -> native_runtime_version

* Pass chain spec to native_runtime_version for polkadot

* Fix line widths

* Traitify ServiceComponents Client

* Block production integration benchmark (#6468)

* proposer benchmark

* update cargo.lock

* Make the `OnRuntimeUpgrade` docs more clear (#6542)

* Support synching of blocks that are not `new_best` (#6508)

* Start

* Remove debug println

* Add tests

* grandpa: minor cleanups in communication module (#6371)

* grandpa: replace Result<(), ()> with Option<()>

* grandpa: replace &Option<T> with Option<&T>

* grandpa: cleanup local id and keystore usages

* grandpa: return bool on check_message_signature

* grandpa: fix erroneous log message on startup

* grandpa: fix test

* Update to make cargo-deny happy (#6547)

* Update to make cargo-deny happy

* Remove cargo deny from CI

* change (ci): run cargo deny only on tags and schedules

Co-authored-by: Denis P <denis.pisarev@parity.io>

* pallet-evm: customizable chain id (#6537)

* Fix runtime benchmarks CI (#6545)

* debug (ci): ci config [skip ci]

* debug (ci): fix runtime benchmarks

* fix identity benchmarks

* fix utility benchmarks

* Revert "debug (ci): ci config [skip ci]"

This reverts commit 081b175b5e95604520c79ea4e5822b84ea35ddaa.

* change (ci): touch ci config to run CI

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Gav Wood <gavin@parity.io>

* Fix mocking multiple http calls in the same function call (#6510)

* Fix mocking multiple http calls in the same function call

Fixes an issue where a function call would perform more than one http request and wait for each to complete before proceeding. The `RequestId` comes from the length of the `requests` collection in the `OffchainState` and if a request is completed before the next one starts it will be removed and the "next expected" will be off by one. This PR tries to fix that by using a request counter that tracks how many requests have been performed so that we can `remove()` items from the `expected_requests` at the right index.

I suspect that this is a sub-optimal soluton and perhaps requests and their mocks should live side by side in the same collection, e.g. in a tuple of `(PendingRequest, Option<ExpectedRequest>)`.

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Panic on overflow

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Use a Deque and push/pop expected requests

* fix test

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* .maintain/monitoring/alerting-rules: Adjust transaction queue size alert (#6426)

The transaction queue size alert has been firing with a constant 10
transactions in the queue. While maybe problematic those 10 transactions
don't need to be the same across scrape intervals.

Instead of alerting with a size above 10, alert based on two things:

1. Monotonically increasing queue size

2. Upper limit queue size reached

* Fix auto-label-issues.yml (#6536)

statements in github actions cannot use ", must use '

Co-authored-by: Gav Wood <gavin@parity.io>

* [CI] Add Github Action to notify devops of PRs labelled with A1-needsburnin (#6525)

* add burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* Update .github/workflows/burnin-label-notification.yml

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Gav Wood <gavin@parity.io>

* Restrict `Protected` to some heap types. (#6471)

* Restrict `Protected` to some heap types.

* Comment abut Protected usage.

* Remove Protected from crypto, use secrecy crate for existing uses.

* use a parse function

* fix error convert

* Rename and move secretY string function.

* std result

* Remove `ServiceBuilderCommand` and implement the chain ops as standalone functions instead. (#6543)

* :)

* Slight tidy

* Remove ServiceBuilderCommand

* Remove whitespace

* Keep task manager alive for check_block/import_blocks

* Pass task_manager to run_until_exit

* Make task_manager in run_until_exit and make subcommands async

* Change the async_run fn to return a future and task manager

* async_run should take a result fn

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix spaces in export_raw_state

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Drop the tokio runtime before the task_manager (#6548)

* Initial commit

Forked at: 4919c808cb75618d95762944aa6f5664c1aa3b59
Parent branch: origin/master

* Drop the tokio runtime before the task_manager

The tokio runtime must be dropped before the task_manager. Otherwise the objects the task_manager
keep alive are dropped before the tasks are finished.

* Update SubstrateCli to return String (#6550)

* Update SubstrateCli to return String

* Add default implementation for executable_name()

* Use display instead of PathBuf

* Get file_name in default impl of executable_name

* Remove String::from and use .into()

* Use default impl for executable_name()

* Use .as_str() and remove useless .to_string()

* Update only sp-io when running companion build

* Remove unneeded update of sp-io in CI

Co-authored-by: Cecile Tonglet <cecile@parity.io>

* Allow specify schedule dispatch origin (#6387)

* allow specify schedule dispatch origin

* fix tests

* use caller origin for scheduled

* fix tests

* line width

* check origin for cancel

* line width

* fix some issues for benchmarking

* fix doc test

* another way to constraint origin

* fix build issues

* fix cancel

* line width

* fix benchmarks

* bump version

* enable runtime upgrade

* add migration code and test

* Update frame/scheduler/src/lib.rs

Co-authored-by: Gavin Wood <github@gavwood.com>

* expose migration method

* add notes

* bump version

* remove on_runtime_upgrade

* fix test

Co-authored-by: Gavin Wood <github@gavwood.com>

* Move `create_inherents` into the block-builder (#6553)

* Move `create_inherents` into the block-builder

This moves the `create_inherents` call into the block-builder. This has
the advantage that `create_inherents` will be able to reuse the same
context that will be used when applying the extrinsics and we also save
one call to `on_initialize`. To make sure that `create_inherents` does
not modify any state, we execute it in a transaction that is
rolled-back after doing the runtime call.

* Feedback and build fix

* Update primitives/runtime/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Update client/block-builder/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* client/network: Remove unused Result returned by NetworkWorker (#6552)

* New testing helpers (#6555)

* Initial commit

Forked at: 8ef1ac0ee13d2a72cc1c391d4624dfaaafe641e8
Parent branch: origin/master

* Add send_transaction to RpcHandlers

* Extension trait for RpcHandlers

* Revert "Add send_transaction to RpcHandlers"

This reverts commit 03c89e13d404bae3f3123387dd50f026061bca82.

* Add an extension trait for BlockchainEvents

* Update test-utils/client/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update test-utils/client/src/lib.rs

* fix

* deps fix

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove polkadot companion detection from branch name  (#6565)

* Initial commit

Forked at: facc6741400a21fa3b5a3eedfe444f30ab5df8b0
Parent branch: origin/master

* Remove polkadot companion detection from branch name

Even though it was nice it was also error prone as there were no indication whatsoever on the PR
that a polkadot companion branch exists.

* primitives: use generic Header in testing runtime (#6561)

* primitives: use generic Header in testing runtime

* frame: remove unused imports

* Remove warning

Co-authored-by: Bastian Köcher <git@kchr.de>

* Remove polkadot companion detection from branch name  (#6568)

* Initial commit

Forked at: 7e3cf944cf87a22036cc26f03bd859855b7ed3f1
Parent branch: origin/master

* Remove polkadot companion detection from branch name

Even though it was nice it was also error prone as there were no indication whatsoever on the PR
that a polkadot companion branch exists.

* Make the encoded-Call Vec<u8> explicitly so in metadata (#6566)

* ci: fix merge on polkadot companion job (#6574)

* pallet-evm: return Ok(()) when EVM execution fails (#6493)

* pallet-evm: return Ok(()) when EVM execution fails

* Bump spec version

* Init test module

* Add fail_call_return_ok test

* Fix tests and use full match pattern

Co-authored-by: Gav Wood <gavin@parity.io>

* babe: report equivocations (#6362)

* slots: create primitives crate for consensus slots

* offences: add method to check if an offence is unknown

* babe: initial equivocation reporting implementation

* babe: organize imports

* babe: working equivocation reporting

* babe: add slot number to equivocation proof

* session: move duplicate traits to session primitives

* babe: move equivocation stuff to its own file

* offences: fix test

* session: don't have primitives depend on frame_support

* babe: use opaque type for key owner proof

* babe: cleanup client equivocation reporting

* babe: cleanup equivocation code in pallet

* babe: allow sending signed equivocation reports

* node: fix compilation

* fix test compilation

* babe: return bool on check_equivocation_proof

* babe: add test for equivocation reporting

* babe: add more tests

* babe: add test for validate unsigned

* babe: take slot number in generate_key_ownership_proof API

* babe: add benchmark for equivocation proof checking

* session: add benchmark for membership proof checking

* offences: fix babe benchmark

* babe: add weights based on benchmark results

* babe: adjust weights after benchmarking on reference hardware

* babe: reorder checks in check_and_report_equivocation

* Benchmarks Writer CLI (#6567)

* initial mockup

* add and wipe

* track writes

* start to add to pipeline

* return all reads/writes

* Log reads and writes from bench db

* causes panic

* Allow multiple commits

* commit before ending benchmark

* doesn't work???

* fix

* Update lib.rs

* switch to struct for `BenchmarkResults`

* add to output

* fix test

* line width

* @kianenigma review

* Add Whitelist to DB Tracking in Benchmarks Pipeline (#6405)

* hardcoded whitelist

* Add whitelist to pipeline

* Remove whitelist pipeline from CLI, add to runtime

* clean-up unused db initialized whitelist

* Add regression analysis to DB Tracking (#6475)

* Add selector

* add tests

* debug formatter for easy formula

* initial idea

* use all benchmarks

* broken

* working without trait

* Make work for multiple pallets

* Fix merge issues

* writer appends to file

* implement () for balances weight trait

* update name of trait

* Weights to WeightInfo

* auto trait writer

* Heap pages are configurable

* clean out runtime changes

* more clean up

* Fix string generation

* Update comments

* Update bin/node/runtime/src/lib.rs

Co-authored-by: arkpar <arkady.paronyan@gmail.com>

* Don't require module name in inherents (#6576)

* Start

* Cleanup `construct_runtime!`

* Add tests

* Fix after merge

* Update the docs

* Derive `RuntimeDebug` for `Runtime` (#6581)

* Fix UI tests (#6589)

* Use async/await in build_network_future (#6533)

* Use async/await in build_network_future

* Address concerns

* Fix test

* decl_module! macro: use 'frame_system' instead of `system` as default ident (#6500)

* Use frame_system as default ident.

* Remove unused 'frame_system' to 'system' renaming.

* Fix construct_runtime_ui tests.

* Rename system to frame_system in sudo/utility pallet test.

* Bump runtime impl_version.

* Update formatting.

* Upgrade `kvdb-*`, `trie-db` and `memory-db` (#6584)

* Upgrade `kvdb-*`, `trie-db` and `memory-db`

The updates of `trie-db` and `memory-db` are important, as they fix the
non-deterministic build of Polkadot/Substrate.

* Change `trie-db` version

* Update test-utils/runtime/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update primitives/trie/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update `Cargo.lock` and `trie-bench`

* Fix UI tests

* Switch to fixed version of memory-db

Co-authored-by: Andronik Ordian <write@reusable.software>

* client/network: Rename DebugInfoBehaviour to PeerInfoBehaviour (#6556)

Information retrieved via `DebugInfoBehaviour` is not only used for
debugging purposes, e.g. disconnecting from nodes not responding to
pings, using external addresses retrieved via indentify, ...

In order for the name to reflect the usage of the module this commit
renames it.

* Add log rotation (#6564)

* Use flexi_logger; Add log rotation

* Add default rotation; Add FlexiLogger error

* Fix compilation error

* Remove logging to stdout if it's not a tty

* Fix formatting

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove needless debug statement

* Default to unlimited size for log rotation

* Add more comments about log-age option

* Remove unused variable

* Fix typo in comment

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Send Status message on all newly-opened legacy substreams (#6593)

* Send Status message on all newly-opened legacy substreams

* Fix tests

* *: Update to libp2p v0.21.1 (#6559)

* *Cargo.toml: Update versions

* client/network/src/discovery: Adjust to Kademlia  API changes

* client/network: Adjust to one_shot.rs changes

* client/network/discovery: Log address list on trace level

* client/network/discovery: Ignore RoutablePeer and PendingRoutablePeer

* Commit Cargo.lock

* Finish update

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Improve transaction submission (#6599)

* Improve transaction submission

Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.

Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.

* Make sure we don't submit the same transaction twice from the network concurrently

* Remove added listener call

* Feedback

* Ignore banned on resubmit

* Add `WeightInfo` to all pallets with benchmarks. (#6575)

* Start adding weight info

* More weightinfo

* finish weight info

* more fixes

* inital update of node runtime

* fix the rest of the compilation

* update balances

* add docs

* fix balances tests

* Fix more tests

* Fix compile

* Fix pallet-evm tests

* Fix build when with-kvdb-rocksdb is disabled. (#6546)

* Make evm errors public (#6598)

* Improved send_transaction helper to return an error in case of error  (#6592)

* Rename `CheckEra` to `CheckMortality` (#6619)

* Add an authority_discovery_known_authorities_count metric (#6614)

* Clean exit when no space left on device (#6339)

Fixes #6305

* seal: Rework contracts API (#6573)

* Transition getter functions to not use scratch buffer

* Remove scratch buffer from ext_get_storage

* Remove scratch buffer from ext_call

* Remove scratch buffer from ext_instantiate

* Add ext_input and remove scratch buffer

* Rework error handling (changes RPC exposed data)

* ext_return passes a flags field instead of a return code
	* Flags is only for seal and not for the caller
	* flags: u32 replaced status_code: u8 in RPC exposed type
* API functions use a unified error type (ReturnCode)
* ext_transfer now traps on error to be consistent with call and instantiate

* Remove the no longer used `Dispatched` event

* Updated inline documentation

* Prevent skipping of copying the output for getter API

* Return gas_consumed from the RPC contracts call interface

* Updated COMPLEXTITY.md

* Rename ext_gas_price to ext_weight_to_fee

* Align comments with spaces

* Removed no longer used `ExecError`

* Remove possible panic in `from_typed_value`

* Use a struct as associated data for SpecialTrap::Return

* Fix nits in COMPLEXITY.md

* Renamed SpecialTrap to TrapReason

* Fix test

* Finish renaming special_trap -> trap_reason

* Remove no longer used get_runtime_storage

* fixup! Remove no longer used get_runtime_storage

* Removed tabs for comment aligment

* Simplify a few chain components creation APIs related to the service (#6611)

* Simplify a few chain components creation APIs related to the service

* Fix basic-authorship doc tests

* Remove DefaultQueue

* Update client/service/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Move ExecutionExtensions comment around

* Remove unused BlakeTwo256

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Make `init_logging` more easily usable (#6620)

Instead of requiring the `LogRotationOpt`, it now requires an
`Option<LogRotationOpt>`. This makes it much more easy to use the
interface when someone isn't interested on the `LogRotationOpt`'s

* add ios to cfg gates in client/service (#6622)

* done

* sysinfo should not be not android

* add back sysinfo dep to satisfy web wasm ci

* Node template love (#6560)

* Node template love

Better description in node/Cargo.toml
Example dispatchables should have non-zero weights
Changes to README to bring in line with current DevHub Node Template

* Update README.md

cleaned up, reworked text, ready for round 2

* Add DbWeight

* Update README.md

* Consistent Playground buttons

* "setup" -> "set up"

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Import `Get` trait to fix build

Co-authored-by: Addie Wagenknecht <addie@nortd.com>
Co-authored-by: Ricardo Rius <ricardo@parity.io>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* change ss58 format for SubstraTEE from 44 to 13 and add ChainX (#6602)

* Deduplicate some dependencies (#6628)

* Revert "Add log rotation (#6564)" (#6627)

This reverts commit 802a0d0b0ade796a3b2d4663212518315923fe8a.

* seal: Prevent contracts from going below subsistence (#6623)

* seal: Do not allow transfers to bring total balance below subsistence deposit

This also reworks the rent system to take the total balance into account
when evaluating whether the account is above the subsistence deposit.

* Fix nits from review

* Fix typo

* Do not enforce subsistence when called from EOA

* Rename CallOrigin to TransactorKind

* Add debug asserts to check the invariants of a plain account transactor

* Fix typo

Co-authored-by: Sergei Shulepov <sergei@parity.io>

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Remove warning about networking taking a lot of time (#6630)

* Remove warning about networking taking a lot of time

* Fix warnings

* Add FromStr for AccountId32 (#6621)

* Add FromStr for AccountId32

* Review grumbles

* Clarify some host interfaces (#6626)

* Remove duplicate comments (#6638)

* Make signature batching use specialized methods (#6616)

It was a mistake to use the `*_verify` methods for signature batching.
This pr move the signature batching into their own functions. This is
required, because otherwise transaction signature verification infers
with other signature verifications.

This pr also temporarily disables signature batching. The functionality
stays, but we need to make sure that all nodes have the new runtime
interface, before we can bring back signature batching.

* Check that `serde` is present in `sc-chain-spec-derive` (#6636)

We should check that `serde` is present when using
`sc-chains-spec-derive`, because we require this for the generated
struct. This also switches the generated `derive` instruction to import
`Serialize`/`Deserialize` by path to not require the user to import
these derive macros.

* ci: add deployment of polkadot prometheus rules (#6646)

* Implement apply_state flag and allow fetching return data and used gas  (#6590)

* pallet-evm: return Ok(()) when EVM execution fails

* Bump spec version

* Implement apply_state flag and allow fetching return data and used gas

* Update evm version

* Remove legacy Clogged event (#6652)

* Fix two minor warnings (#6650)

* Unused import in no_std builds
* Global attribute in non-root of a crate

* allow to specify schedule time as a relative value (#6578)

* allow to specify schedule time as a relative value

* bump version

* line width

* fix benchmarks

* rename and updates

* Move the legacy protocol handshake to the legacy substream (#5938)

* Move the legacy protocol handshake to the legacy substream

* Fix tests

* Remove line that wasn't supposed to be committed

* Remove hack

* Rework how it's done

* Some little changes

* update_chain wasn't doing its thing

* Fix service tests not calling update_chain

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* [WIP]

* Revert "[WIP]"

This reverts commit 2b892e6a7637c0b1297e6ecdbb919321c9098ff5.

* Update client/network/src/protocol.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix received message not being handshake

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

Co-authored-by: Max Inden <mail@max-inden.de>

* [CI] Fix check_polkadot_companion_status.sh (#6631)

* check changes_requested and approved separately

* handle checking more than 1 CHANGES_REQUESTED review

* sc-offchain: upgrade hyper-rustls to 0.18 (#6663)

* sc-offchain: upgrade hyper-rustls to 0.18

* Cargo.lock: revert rand downgrade due to rust-lang/cargo/issues/5702

* Support Component-less Runtime Benchmarks (#6645)

* Update benchmarking macro for no components

* Handle output when error

* skip when empty

* Update analysis for zero components

* add back trace logs

* Apply suggestions from code review

* remove mean value, and use median value

* Add note

* Use standard for loop

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove polkadot companion detection from url anywhere  (#6570)

* The telemetry should not be initialized if no telemetry has been provided (#6666)

* Initial commit

Forked at: 4da29261bfdc13057a425c1721aeb4ec68092d42
Parent branch: origin/master

* Fix: do not initialize telemetry if telemetry is not set

* Update client/telemetry/src/lib.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update client/telemetry/src/lib.rs

* Update client/telemetry/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* fix comment (#6670)

* Regression test to ensure we don't break deterministic builds in wasm (#6597)

* Regression test to ensure we don't break deterministic builds in wasm again

* Apply suggestions from code review

Co-authored-by: s3krit <pugh@s3kr.it>

* Update .gitlab-ci.yml

Co-authored-by: s3krit <pugh@s3kr.it>

Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update substrate-networking Grafana dashboard (#6649)

* client/service/metrics: Rework metric HELP texts (#6647)

* .maintain/monitoring/alerting-rules: Remove HighCPUUsage alert (#6648)

The `HighCPUUsage` alert is based on the `cpu_usage_percentage` metric.
Instead of exposing the overall CPU usage in percent, the metric exposes
the per core usage summed over all cores.

This commit removes the alert for two reasons:

1. Substrate itself does not expose the core count and thus one can not
alert based on the `cpu_usage_percentage` metric.

2. Alerting based on CPU usage is generic and not specific to Substrate
or Blockchains. Thus any CPU usage alert suffice.

* identity: Don't let subs be re-registered (#6667)

* Fixes and tests

* Don't set subs be re-registered.

Also allow subs to de-register themselves and collect the deposit.

Also allow individual registering and removal of subs.

* Make it build

* Update frame/identity/src/lib.rs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Tests

* Add benchmarks

* Add some reasonable weights

* Docs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Treasury burning can be directed (#6671)

* Treasury burning can be directed

Also, Society is a imbalance handler

* Build

* Introduce from_permill in perthings.

* Rename to from_perthousand to avoid confusion with Permill

* Fixes

* Switch to bounded mpsc for txpool import notification stream (#6640)

* Switch to bounded mpsc for txpool import notification stream

* Update client/transaction-pool/graph/src/validated_pool.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* grandpa: report equivocations with unsigned extrinsics (#6656)

* grandpa: use unsigned extrinsics for equivocation reporting

* grandpa: allow signed reports as well

* grandpa: change runtime api for submitting unsigned extrinsics

* grandpa: fix tests

* grandpa: add test for unsigned validation

* grandpa: add benchmark for equivocation proof checking

* offences: fix grandpa benchmark

* grandpa: add proper weight for equivocation reporting extrinsic

* grandpa: fix weight unit

* Transaction pool integrated benchmarks (#6579)

* txpool benchmarks

* updated api

* Update bin/node/bench/src/txpool.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Change unknown-commit to unknown to make parsing easier (#6675)

* done (#6673)

* Fix tx-payment integrity-test (#6633)

* Fix tx-payment integrity-test

* Better error message.

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Fix line width

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Fix grumble

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* fix check polkadot companion status check (#6688)

* Fix compilation problems due to ed25519-dalek 1.0.0-pre.4 (#6676)

* Fix compilation problems due to ed25519-dalek 1.0.0-pre.4

* Update libp2p-core

* Revert rand downgrade

* More app-friendly event description (#6684)

* More app-friendly event description

* change origin -> owner

* checked all decl_event! and changed decriptions.

* annotated parameter names for remaining events

* Update libp2p to 0.22. (#6690)

* Update libp2p to 0.22.

Primarily to begin the libp2p-noise upgrade process.

See https://github.com/libp2p/rust-libp2p/issues/1631.

* CI

* Add try_mutate_exists to StorageDoubleMap. (#6694)

* add ecdsa benches (#6697)

* Fixup `cargo unleash` (#6689)

* Bumping to rc5

* Revert "Bumping to rc5"

This reverts commit 3c6767db3d7f94e164c0f08986085fb98bd883ce.

* Bumping for rc5, correctly this time

* Add sync_legacy_requests_received metric (#6698)

* Improve overall performance (#6699)

* Improve overall performance

* Clean up code

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove dead link to out-of-date style guide (#6682)

* Remove dead link to out-of-date style guide

* Replace dead link with self-hosted doc

* Use relative link to style guide

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* Format style guide

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* Formatting

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* "cargo test" jobs optimization (#6606)

* change (ci): 3 jobs in 1 decreases concurrency and is more effectiv; w/o release it's ~20% faster, but needs testing on prod; wasmtest tests are already running within cargo test --workspace

* fix (test): these ones were failing on nightly

* save: cargo profiles [skip ci]

* change (ci): one test to run them all

* change (ci): rebase

* Revert "change (ci): rebase"

This reverts commit 8a6b7ea043a460bf71526ccaa4c7a68899a3b2bc.

* fix (config): fix manifest

* change (ci): bench release

* Properly filter out duplicate voters in elections. (#6693)

* Prevent duplicate voter

* Update primitives/npos-elections/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* pallet-swap-action: Change `BalanceSwapAction` signature (#6580)

Instead of requiring `T: Trait` in `BalanceSwapAction`, we directly
depend on `AccountId`. This fixes a compilation error on wasm, where
`Runtime` does not implement `Debug`, but `BalanceSwapAction` required it.

* make impl_outer_origin default to use frame_system (#6695)

* Removing staging resources from node template README (#6701)

* Graceful shutdown for the task manager (#6654)

* Initial commit

Forked at: 60e3a693b29789045614e2ed73126695bc8b0794
Parent branch: origin/master

* Move task_manager.rs to mod.rs

* Graceful shutdown for the task manager

* Await all background task JoinHandle at the same time

* Add tests

* Make future() wait also for exit signal + fix essential task failed

Probably related to https://github.com/paritytech/cumulus/issues/111

* add comments for non-obvious code

* Use clean_shutdown() in sc-cli

* Adapt code and upgrade tokio in sc-cli

* cleanup spacing in doc

* Add license

* I guess actually running the clean shutdown would be a good idea

* fix tests

* Update client/cli/src/runner.rs

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* Improve error logging

* disable other tests (can't reproduce on my machine)

* Revert "disable other tests (can't reproduce on my machine)"

This reverts commit c133c590f33c253123ba0555ce719a71ededd60d.

* It is possible that the tasks are ended first

* Revert "It is possible that the tasks are ended first"

This reverts commit 502aba4a49fb5d892e704c412b8a81768a3f2c71.

* Use single threaded scheduler for more predictability

* enable_time

* Revert "enable_time"

This reverts commit 4e152140764a4bddeedff06a4e36ec701909e8c7.

* Revert "Use single threaded scheduler for more predictability"

This reverts commit ee5e13c5f13ff71e012dcda13579dffeb15f8ffc.

* Revert "Revert "It is possible that the tasks are ended first""

This reverts commit 1b91a8ca3eebbdc18be199c8ca188e88669ae649.

* This cannot be verified either with a threaded pool

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Benjamin Kampmann <ben@parity.io>
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Reset flaming fir network (#6703)

* Tracing values (#6679)

* Switch to serde_json::Value for Visitor values

* add tests

* switch Visitor to use Map

* refactor change names

* \n

* update integration test

* use discrete maps for each type of recorded value

* update integration test

* add docs

* Update client/tracing/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* add docs

Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update to latest sysinfo prevents leaking fd-handlers (#6708)

* Ensure we close old file handles hold by sysinfo

* Dropping is needed unfortunately

* enable process refreshing, ignore result from refresh_process

* jumping to proposed patch

* switch to latest sysinfo

* Update wasmtime to (almost) lastest master (#6662)

* update wasmtime to (almost) lastest master

* bump lock

* bump once more

* Remove the service builder (#6557)

* :)

* Slight tidy

* Remove ServiceBuilderCommand

* Remove whitespace

* Keep task manager alive for check_block/import_blocks

* Pass task_manager to run_until_exit

* WIP

* WIP

* Get rid of the macros

* Simplify a few chain components creation APIs related to the service

* Fix basic-authorship doc tests

* Remove DefaultQueue

* Update client/service/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Move ExecutionExtensions comment around

* Remove unused BlakeTwo256

* Add sc-prelude

* Rename sc-prelude to sc-service-prelude

* Rename to sc-service-types

* Improve service types

* Fix line widths

* Remove sc-service-types and move type definitions to crates

* Update bin/node-template/node/src/service.rs

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>

* Add TLightClientWithHash

* Rework types

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>

* Better handling of stable-only build (#6569)

* Better handling of stable-only build

* Fix node template build

* Fix wasm builder node-template version mismatch

* Fix load_spec error

* Add , in parameter

* Add descrptive panic messages in tests

* Add descriptive tests in node/executor benches

* Fix missing compact_code_unwrap

* Add missing wasm_binary_unwrap function for executor integration test

* Only define import_sp_io in no_std

* Small Cargo.toml styling fix

* Bump wasm-builder to 2.0.0

* Fix all `with_wasm_builder_from_crates` version in Substrate

* Use `with_wasm_builder_from_crates` for node-template

Co-authored-by: Gavin Wood <gavin@parity.io>

* Remove clear_on_drop from browser util crate (#6710)

* Add 4 as address type of ss58 for Katal Chain (#6713)

* [CI]Chaostest suite initiation (#5793)

* Initiate chaostest cli test suite: singlenodeheight on one dev node
    Added chaostest stages in CI
    Added new docker/k8s resources and environments to CI
    Added new chaos-only tag to gitlab-ci.yml

* Update .maintain/chaostest/src/commands/singlenodeheight/index.js

Co-authored-by: Max Inden <mail@max-inden.de>

* change nameSpace to namespace(one word)

* update chaos ci job to match template

* rename build-pr ci stage to docker [chaos:basic]

* test gitlab-ci [chaos:basic]

* Update .gitlab-ci.yml

* add new build-chaos-only condition

* add *default-vars to singlenodeheight [chaos:basic]

* change build-only to build-rules on substrate jobs [chaos:basic]

* test and change when:on_success to when:always [chaos:basic]

* resolve conflicts and test [chaos:basic]

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Support using system storage directly for EVM balance and nonce (#6659)

* Augmented node template docs (#6721)

* Switching from git back to released versions for wasmtime, fix cargo-unleash (#6722)

* Switching from git back to released versions for wasmtime

* filter out cratelift_codegen messages-a

Co-authored-by: NikVolf <nikvolf@gmail.com>

* adding changelog (#6728)

* fixing CI

* remove breaking excepts

* grandpa: allow noting that the set has stalled (#6725)

* grandpa: remove unused methods to convert digest

* grandpa: add root extrinsic for scheduling forced change

* grandpa: add benchmark for schedule_forced_change

* grandpa: don't take authority weight in schedule_forced_change

* grandpa: add const for default forced change delay

* grandpa: adjust weights after benchmark on ref hardware

* grandpa: fix cleanup of forced changes on standard change application

* grandpa: replace schedule_forced_change with note_stalled

* grandpa: always trigger a session change when the set is stalled

* grandpa: fix bug on set id mutation after failed scheduled change

* grandpa: take delay as parameter in note_stalled

* grandpa: fix tests

* grandpa: fix cleanup of forced changes

* grandpa: add test for forced changes cleanup

* grandpa: add test for session rotation set id

* grandpa: add test for scheduling of forced changes on new session

* Name all the tasks! (#6726)

* Remove any implementation of `Spawn` or `Executor` from our task executors

* Fix compilation

* Rename `SpawnBlockingExecutor`

* Update primitives/core/src/traits.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fix tests

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add the Substrate Service Tasks dashboard (#6665)

* seal: Fail instantiate if new contract is below subsistence threshold (#6719)

* seal: Fail instantiate if new contract is below subsistence threshold

We need each contract that exists to be above the subsistence threshold
in order to keep up the guarantuee that we always leave a tombstone behind
with the exception of a contract that called `ext_terminate`.

* Fixup executor test

* Bump runtime

* Remove Unpin requirement for Slots (#6711)

* pallet-evm: add support for tuple-based precompile declarations (#6681)

* pallet-evm: add support for tuple-based precompile declarations

* Add missing license header

* Switch to use impl_for_tuples

* Remove unnecessary impl for ()

* prometheus: don't use protobuf feature (#6744)

* Use node_template_runtime::opaque::Block instead of node_template_runtime::Block (#6737)

* Various small improvements to service construction. (#6738)

* Remove service components and add build_network, build_offchain_workers etc

* Improve transaction pool api

* Remove commented out line

* Add PartialComponents

* Add BuildNetworkParams, documentation

* Remove unused imports in tests

* Apply suggestions from code review

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Remove unused imports in node-bench

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Remove unused node template deps (#6748)

* Remove unused node template deps

Backport changes made by @c410-f3r
https://github.com/substrate-developer-hub/substrate-node-template/pull/66

* Enhancements to README

* Revert change to serde per @thiolliere

* Replace Process.toml with json (#6740)

* Replace Process.toml with json

* Trigger checks

* Revert "Trigger checks"

This reverts commit 9bdf9f135cecb92ca3859dfa211d396a48dd6a8d.

* Trigger checks

* Revert "Trigger checks"

This reverts commit b0c6f29d6aefaf7ca8b137c7d2f958a5e0929d9e.

* Update parity-scale-codec to prepare for breaking rustc release (#6746)

This updates parity-scale-codec{-derive} to prepare for a rustc release
that would otherwise break the derive implementation:
https://github.com/rust-lang/rust/pull/73084

* seal: Add benchmarks for dispatchables (#6715)

* seal: Fix syntax that confuses rust-analyzer

* seal: Add benchmarks for Dispatchables

These are only the benchmarks for the dispatchables of
the pallet. Those are not listed in the Schedule because
we do not want to pull the Schedule from storage before
dispatching.

This OK because those costs are not related to actual contract
execution. Those costs (instruction costs, ext_* costs) will
be benchmarked seperatly and entered into the default Schedule.

* seal: Add a maximum code size

* Fix comments from review

* Removed SEED constant

* client/network: Adjust wording (#6755)

Rename `NetworkWorker::from_worker` to `NetworkWorker::from_service` as
it is a channel from the `NetworkService` to the `NetworkWorker`.

* Add a back-pressure-friendly alternative to NetworkService::write_notifications 🎉 (#6692)

* Add NetworkService::send_notifications

* Doc

* Doc

* API adjustment

* Address concerns

* Make it compile

* Start implementation

* Progress in the implementation

* Change implementation strategy again

* More work before weekend

* Finish changes

* Minor doc fix

* Revert some minor changes

* Apply suggestions from code review

* GroupError -> NotifsHandlerError

* Apply suggestions from code review

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* state_transition_waker -> close_waker

* Apply suggestions from code review

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Finish renames in service.rs

* More renames

* More review suggestsions applied

* More review addressing

* Final change

* 512 -> 2048

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Cleanup our sort usage (#6754)

* Allow `PostDispatchInfo` to disable fees (#6749)

* initial mock

* add test

* remove unneeded clone

* Update frame/support/src/weights.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* fix compile

* Update frame/support/src/weights.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update frame/sudo/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* benchmarks! macro: factorize instance usage. (#6750)

* factorize benchmark!

* fix types

* fix types

* client/network: Add peers to DHT only if protocols match (#6549)

* client/network/src/discovery: Adjust to Kademlia  API changes

* client/network: Add peers to DHT only if protocols match

With https://github.com/libp2p/rust-libp2p/pull/1628 rust-libp2p allows
manually controlling which peers are inserted into the routing table.
Instead of adding each peer to the routing table automatically, insert
them only if they support the local nodes protocol id (e.g. `dot`)
retrieved via the `identify` behaviour.

For now this works around
https://github.com/libp2p/rust-libp2p/issues/1611. In the future one
might add more requirements. For example one might try to exclude
light-clients.

* Cargo.toml: Remove crates.io patch for libp2p

* client/network/src/behaviour: Adjust to PeerInfo name change

* client/network/src/discovery: Rework Kademlia event matching

* client/network/discovery: Add trace on adding peer to DHT

* client/network/discovery: Retrieve protocol name from kad behaviour

* client/network/discovery: Fix formatting

* client/network: Change DiscoveryBehaviour::add_self_reported signature

* client/network: Document manual insertion strategy

* client/network/discovery: Remove TODO for ignoring DHT address

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Add `memory-tracker` feature to `sp-trie` to fix wasm panic (#6745)

* Add memory tracker feature to sp-trie to fix wasm panic

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* support custom ss58addressformat in from_ss58check_with_version (#5526)

* support custom ss58addressformat in from_ss58check_with_version

* fix str parse

1. if can parse with u8, use u8 into.
2. if u8 can't parse, convert to str then parse

* add a test

* typo

* add error description in test

* fix the `TryFrom<u8>` for `Ss58AddressFormat`

change check logic in TryFrom<u8> to replace modified code in `from_ss58check_with_version`

* use Ss58AddressFormat::default() replace DEFAULT_VERSION

* Apply suggestions from code review

* Update primitives/core/src/crypto.rs

* Update primitives/core/src/crypto.rs

* Update primitives/core/src/crypto.rs

* Update primitives/core/src/crypto.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add a `DefaultQueue` type alias to remove the need to use `sp_api::TransactionFor` (#6761)

* Add DefaultQueue

* Add DefaultImportQueue to the top level of sp-consensus

* Ignore flaky test (#6767)

* Add "✅ Successfully mined block" log message (#6764)

* pallet-evm: add builtin support for the four basic Ethereum precompiles (#6743)

* pallet-evm: add builtin support for the four basic Ethereum precompiles

* linear_cost -> ensure_linear_cost to directly return OutOfGas error

* Rename task name to stick to the default naming scheme (#6768)

* BABE slot and epoch event notifications (#6563)

* BabeWorker -> BabeSlotWorker

* SlotWorker::notify_slot: similar to claim_slot, but called no matter authoring

* Wrap the future with a new struct BabeWorker

* Add type definition slot_notification_sinks

* Function slot_notification_streams for the receiver side

* Get a handle of slot_notification_sinks in BabeSlotWorker

* Implement notify_slot

* Switch to use bounded mpsc

* Do not drop the sink when channel is full

Only skip sending the message and emit a warning, because it is recoverable.

* Fix future type bounds

* Add must_use and sink type alias

* Update Balances Pallet to use `WeightInfo` (#6610)

* Update balance benchmarks

* Update weight functions

* Remove user component

* make componentless

* Add support for `#[extra]` tag on benchmarks

* Update balances completely

* Apply suggestions from code review

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Fix some tests

* Maybe fix to test. Need approval from @tomusdrw this is okay

* Make test better

* keep weights conservative

* Update macro for merge master

* Add headers

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Fix graceful shutdown skipped if future ends with error (#6769)

* Initial commit

Forked at: 5c34fe49f8fccb1f0e0f983bb7f8cbb8850b574b
Parent branch: origin/master

* Fix graceful shutdown skipped if future ends with error

* apply suggestion

* Fix link (#6775)

* Allow blacklisting blocks from being finalized again after block revert (#6301)

* Allow blacklisting blocks from being finalized again after block revert

* Use BlockRules for storing unfinalized and add have_state_at in revert

* Move finalization_check in finalize_block upward

* Directly mark finalization blacklist as badblocks

* Remove obselete comment

* Order delta before calculating the storage root (#6780)

We need to order the delta before calculating the storage root, because
the order is important if the storage root is calculated using a storage
proof. The problem is arises when the delta is different than at the
time the storage root was recorded, because we may require a different
node that is not part of the proof and so, the storage root can not be
calculated. The problem is solved by always order the delta to use the
same order when calculating the storage root while recording the
stroage proof and when calculating the storage root using
the storage proof.

To prevent this bug in future again, a regression test is added.

Fixes: https://github.com/paritytech/cumulus/issues/146

* Don't close inbound notifications substreams immediately (#6781)

* Don't close inbound notifications substreams immediately

* Fix not closing in return to node A closing

* client/network: Expose DHT query duration to Prometheus (#6784)

Expose duration of DHT put and get request as a Prometheus histogram.

* Fix transaction payment runtime api (#6792)

The transaction payment runtime api used its own extrinsic generic
parameter. This is wrong, because this resulted in using always the
native extrinsic. If there was a runtime upgrade that changed the
extrinsic in some way, it would result in the api breaking. The correct
way is to use the `Extrinsic` from the `Block` parameter. This is on the
node side the opaque extrinsic and on the runtime side the real extrinsic.

* Bump elliptic from 6.5.2 to 6.5.3 in /.maintain/chaostest (#6791)

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.2 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](https://github.com/indutny/elliptic/compare/v6.5.2...v6.5.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* client/network: Fix wrong metric help text (#6794)

The `sub_libp2p_kademlia_query_duration` metric only has the dimension
`type` not `protocol`.

* seal: Fix and improve error reporting (#6773)

* seal: Rework ext_transfer, ext_instantiate, ext_call error handling

* Deny calling plain accounts (must use transfer now)
* Return proper module error rather than ad-hoc strings
* Return the correct error codes from call,instantiate (documentation was wrong)
* Make ext_transfer fallible again to make it consistent with ext_call

* seal: Improve error messages on memory access failures

* seal: Convert contract trapped to module error

* seal: Add additional tests for transfer, call, instantiate

These tests verify that those functions return the error types
which are declared in its docs.

* Make it more pronounced that to_execution_result handles trap_reason

* Improve ReturnCode docs

* Fix whitespace issues in wat files

* Improve ReturnCode doc

* Improve ErrorOrigin doc and variant naming

* Improve docs on ExecResult and ExecError

* Encode u32 sentinel value as hex

* with_nested_context no longer accepts an Option for trie

* Fix successful typo

* Rename InvalidContractCalled to NotCallable

* Improve Benchmark Writer: Remove Unused Components, Remove Multiply by Zero, Files Split by Pallet (#6785)

* initial improvements

* better file management, ignore unused components

* Output warning when components unused

* update comment

* Write even when base weight is zero

* remove unwrap where possible

* Dont sort components to dedup

* undo delete

* improve clarity of unused components

* remove unused dep

* Update Process.json

* Add integrity test for slash defer duration (#6782)

* Add integrity test for slash defer duration

* Wrap in externalities

* Update frame/staking/src/lib.rs

* Convert spaces to tabs (#6799)

* Add details to legacy requests (#6747)

* Add ss58 address for Subsocial (#6800)

* Add mutate_exists to StorageDoubleMap. (#6704)

* pallet-democracy use of weightinfo (#6783)

* democracy use of weightinfo

* fix some doc and benchs

* todo generate from parity machine

* factorize and add license

* use final weights

* add slightly more sensible default weight

* refactor

* rename benchmark to avoid confusion

* just make remove_other_vote benchmark being the worst case of the extrinsic

* Use DNS hostnames for flaming fir bootnodes (#6807)

* Use dns hostnames for flaming fir bootnodes

* Remove newline

* Fix warning being printed by authority-discovery (#6820)

* Better default balances weights (#6813)

* Remove generation of instance trait by decl_storage. (#6812)

* remove generation of instance trait, no breaking change

* doc

* doc

* Update frame/support/src/traits.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update frame/support/procedural/src/storage/instance_trait.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Successful `note_imminent_preimage` is free (#6793)

* Successful `note_imminent_preimage` is free

* update docs

* Add test for duplicate preimage

* service: remove collection of system/process metrics (#6822)

* Fix legacy substream fallback not working (#6826)

* Fix legacy substream fallback not working

* Make it nicer

* grandpa: never overwrite current rounds voter state (#6823)

* grandpa: never overwrite current rounds voter state

* grandpa: add test for voter state overwrite

* Add RPC Builder to Substrate Node Template (#6808)

* Pulled RPC from node and populated the node-template's RPC builder with one example implementation

* surpress build errror

* dead_code

* Fixed module usage, removed copyright, removed rpc builder for light client + some comments

* added a comment for rpc extension

* Update bin/node-template/node/src/rpc.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update rpc.rs

* fix spacing

* more space to tabs

* more space to tabs

* Documenation nitpick

* Documentation nitpick

* Documentation nitpick

* Documentation nitpick

* Documentation nitpick

* pre-format

* Updated transaction payment API implemented for node template

* fix space and commented code

* fix long line

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Dan Forbes <dan@danforbes.dev>

* Delay network startup to after complete initialization (#6833)

* Delay network startup to after complete initialization

* Update client/service/src/builder.rs

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

* De-alias frame_system in node template runtime (#6829)

* de-alias frame_system in node template

* Fix line length

* Fix chainspec

* Allow task manager to have children (#6771)

* Initial commit

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* changelog

* Remove Box

* Make future nicer

* Revert "Make future nicer"

This reverts commit 49fb8fb6f245c3ca2c384468df14b34f34616736.

* Simplify

* Additional check

* Simplify more

Co-authored-by: Bastian Köcher <git@kchr.de>

* De-alias pallets in node template runtime (#6836)

* dealias pallets

* Restore accidentally deleted code blocks

* grandpa: fix enacting forced changes with no delay (#6828)

* grandpa: fix enacting forced changes with no delay

* grandpa: fix formatting

* Renamed add_children to add_child since it adds only one child (#6838)

* Revalidate transactions only on latest best block (#6824)

* Revalidate transactions only on latest best block

We should revalidate transactions only on the latest best block and not
on any arbitrary block. The revalidation before failed when there were
multiple blocks on the height given to the revalidation function, but no
block was imported as best block.

* Update test-utils/runtime/transaction-pool/src/lib.rs

Co-authored-by: Jaco Greeff <jacogr@gmail.com>

* Fix tests

* Only process best blocks in the transaction pool

Co-authored-by: Jaco Greeff <jacogr@gmail.com>

* Fix `wait_for_blocks` counting blocks that didn't go through consensus (#6850)

* Initial commit

Forked at: 74804b5649eccfb83c90aec87bdca58e5d5c8789
Parent branch: origin/master

* WIP

Forked at: 74804b5649eccfb83c90aec87bdca58e5d5c8789
Parent branch: origin/master

* pallet-evm: move gas price check to execute_evm (#6837)

* Update pinned wasm-bindgen dependency (#6861)

* Update dependency.

* Update readme docs

* update python webserver command

* Add Subscription RPC for Grandpa Finality (#5732)

* Rough skeleton for what I think the RPC should look like

* Create channel for sending justifications

Sends finalized header and justification from Grandpa to the
client. This lays the groundwork for hooking into the RPC module.

* WIP: Add subscribers for justifications to Grandpa

Adds the Sender end of a channel into Grandpa, through which notifications
about block finality events can be sent.

* WIP: Add a struct for managing subscriptions

Slightly different approach from the last commit, but same
basic idea. Still a rough sketch, very much doesn't compile yet.

* Make naming more clear and lock data in Arc

* Rough idea of what RPC would look like

* Remove code from previous approach

* Missed some things

* Update client/rpc-api/src/chain/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update client/rpc-api/src/chain/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Split justification subscription into sender and receiver halves

* Replace RwLock with a Mutex

* Add sample usage from the Service's point of view

* Remove code that referred to "chain_" RPC

* Use the Justification sender/receivers from Grandpa LinkHalf

* Add some PubSub boilerplate

* Add guiding comments

* TMP: comment out to fix compilation

* Return MetaIoHandler from PubSubHandler in create_full

* Uncomment pubsub methods in rpc handler (fails to build)

* node/rpc: make Metadata concrete in create_full to fix compilation

* node: pass in SubscriptionManger to grandpa rpc handler

* grandpa-rpc: use SubscriptionManger to add subscriber

* grandpa-rpc: attempt at setting up the justification stream (fails to build)

* grandpa-rpc: fix compilation of connecting stream to sink

* grandpa-rpc: implement unsubscribe

* grandpa-rpc: update older tests

* grandpa-rpc: add full prefix to avoid confusing rust-analyzer

* grandpa-rpc: add test for pubsub not available

* grandpa-rpc: tidy up leftover code

* grandpa-rpc: add test for sub and unsub of justifications

* grandpa-rpc: minor stylistic changes

* grandpa-rpc: split unit test

* grandpa-rpc: minor stylistic changes in test

* grandpa-rpc: skip returning future when cancelling

* grandpa-rpc: reuse testing executor from sc-rpc

* grandpa-rpc: don't need to use PubSubHandler in tests

* node-rpc: use MetaIoHandler rather than PubSubHandler

* grandpa: log if getting header failed

* grandpa: move justification channel creation into factory function

* grandpa: make the justification sender optional

* grandpa: fix compilation warnings

* grandpa: move justification notification types to new file

* grandpa-rpc: move JustificationNotification to grandpa-rpc

* grandpa-rpc: move JustificationNotification to its own file

* grandpa: rename justification channel pairs

* grandpa: rename notifier types

* grandpa: pass justification as GrandpaJustification to…
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants