Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

grandpa: remove light-client specific block import pipeline #7546

Merged
2 commits merged into from
Nov 23, 2020

Conversation

andresilva
Copy link
Contributor

@andresilva andresilva commented Nov 16, 2020

Fixes #6893.

tldr: This PR removes a bunch of code related to handling of finality for the light client, essentially it removes the special casing and makes the light client use the same code the full client does.

The PR has 2 commits that can be reviewed separately.

Remove LightGrandpaBlockImport

The light client used a specialized block import pipeline that would not take in regular GRANDPA justifications but instead build "finality proof" requests which are an extension over justifications that allow proving finality of a block B with a justification for one of its descendents B'. Additionally the finality proof also comes with a storage proof for the authorities key.

In practice this isn't any lighter than just importing regular justifications using the full block import pipeline. I also think the storage proof is unnecessary, the light client will read the authorities from a header digest. If we can trust the storage root we can also trust the header hash.

I have not removed the code for actually generating finality proofs since this will be used in #7339. Some more cleaning up can be done in that PR.

Remove FinalityProof network protocol

The only user of the FinalityProof network protocol was GRANDPA's light import and since we're removing it I think it makes sense to avoid the burden of maintaining an unused network protocol (for example in #7478). In the future we will have some new network protocol for "warp" syncing the light client but the interface will almost certainly not match and breaking changes to the finality proof protocol would be required regardless.

polkadot companion: paritytech/polkadot#1954

@andresilva andresilva added A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. labels Nov 16, 2020
@andresilva andresilva changed the title grandpa: remove light-client block import pipeline grandpa: remove light-client specific block import pipeline Nov 16, 2020
@andresilva andresilva requested review from bkchr and removed request for sorpaas November 16, 2020 20:38
@rphmeier
Copy link
Contributor

In practice this isn't any lighter than just importing regular justifications using the full block import pipeline. I also think the storage proof is unnecessary, the light client will read the authorities from a header digest. If we can trust the storage root we can also trust the header hash.

The only thing we have to be careful of is which authority sets we allow to prove finality. We shouldn't be trusting anything from any given authority set until we've seen finality of the change to that set.

Copy link
Contributor

@tomaka tomaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me from the networking perspective!
I've always wanted to remove this finality proofs protocol.

cc @mxinden you don't need to do finality proofs anymore in #6968

@andresilva
Copy link
Contributor Author

The only thing we have to be careful of is which authority sets we allow to prove finality. We shouldn't be trusting anything from any given authority set until we've seen finality of the change to that set.

Just to be sure I understood what you meant here: with this PR the light client is essentially doing the same thing as the full node (minus executing blocks). It will validate all authority set hand offs in-order, so the change from SetId 1 -> SetId 2 will only be enacted after we import a justification signed by the validators from SetId 1.

@rphmeier
Copy link
Contributor

It will validate all authority set hand offs in-order, so the change from SetId 1 -> SetId 2 will only be enacted after we import a justification signed by the validators from SetId 1.

Yep, that's alright.

Copy link
Contributor

@octol octol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see so much red!
(More familiar with the grandpa changes than the networking ones...)

@tomaka
Copy link
Contributor

tomaka commented Nov 23, 2020

Any reason to not merge this? 🙏

@andresilva
Copy link
Contributor Author

bot merge

@ghost
Copy link

ghost commented Nov 23, 2020

Trying merge.

@ghost ghost merged commit f16acff into master Nov 23, 2020
@ghost ghost deleted the andre/remove-grandpa-light-import branch November 23, 2020 14:28
clearloop added a commit to patractlabs/substrate that referenced this pull request Nov 25, 2020
* make ClientConfig public (paritytech#7544)

* sc-basic-authorship: remove useless dependencies (paritytech#7550)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Add slashing events to elections-phragmen. (paritytech#7543)

* Add slashing events to elections-phragmen.

* Fix build

* Apply suggestions from code review

* Update frame/elections-phragmen/src/lib.rs

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Remove necessity to pass ConsensusEngineId when registering notifications protocol (paritytech#7549)

* Remove necessity to pass ConsensusEngineId when registering notifications protocol

* Line width

* Fix tests protocol name

* Other renames

* Doc update

* Change issue in TODO

* sc-cli: replace bip39 with tiny-bip39 (paritytech#7551)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Add extra docs to on_initialize (paritytech#7552)

* Add some extra on_initialize docs.

* Address review comments.

* More Extensible Multiaddress Format (paritytech#7380)

* More extensible multiaddress format

* update name

* Don't depend on indices to define multiaddress type

* Use MultiAddress in Node Template too!

* reduce traits, fix build

* support multiple `StaticLookup`

* bump tx version

* feedback

* Fix weight template to remove ugliness in rust doc (paritytech#7565)

fixed weight template

* Cargo.lock: Run cargo update (paritytech#7553)

* Cargo.lock: Run cargo update

* Cargo.lock: Downgrade cc to v1.0.62

* Cargo.lock: Revert wasm-* updates

* .github: Add dependabot config and thus enable dependabot (paritytech#7509)

* .github: Add dependabot config and thus enable dependabot

* Update .github/dependabot.yml

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Thread-local parameter_types for testing. (paritytech#7542)

* Thread-local parameter_types for testing.

* Better docs.

* Some minors

* Merge'em

* Update frame/support/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Align more to basti's trick

* Update frame/support/src/lib.rs

* Update frame/support/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

* Bump wasm-bindgen-test from 0.3.12 to 0.3.17 (paritytech#7567)

* Bump wasm-bindgen-test from 0.3.12 to 0.3.17

Bumps [wasm-bindgen-test](https://github.com/rustwasm/wasm-bindgen) from 0.3.12 to 0.3.17.
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

Signed-off-by: dependabot[bot] <support@github.com>

* Update wasm-bindgen pin to 0.2.68

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* pallet-evm: move to Frontier (Part IV) (paritytech#7573)

* Bump secrecy from 0.6.0 to 0.7.0 (paritytech#7568)

* Bump secrecy from 0.6.0 to 0.7.0

Bumps [secrecy](https://github.com/iqlusioninc/crates) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/iqlusioninc/crates/releases)
- [Commits](iqlusioninc/crates@secrecy/v0.6.0...secrecy/v0.7.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix compilation errors

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Bump wasm-bindgen-futures from 0.4.17 to 0.4.18 (paritytech#7572)

Bumps [wasm-bindgen-futures](https://github.com/rustwasm/wasm-bindgen) from 0.4.17 to 0.4.18.
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump lru from 0.4.3 to 0.6.1 (paritytech#7577)

Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.4.3 to 0.6.1.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](jeromefroe/lru-rs@0.4.3...0.6.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump wasm-bindgen-test from 0.3.17 to 0.3.18 (paritytech#7579)

Bumps [wasm-bindgen-test](https://github.com/rustwasm/wasm-bindgen) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump wasm-timer from 0.2.4 to 0.2.5 (paritytech#7578)

Bumps [wasm-timer](https://github.com/tomaka/wasm-timer) from 0.2.4 to 0.2.5.
- [Release notes](https://github.com/tomaka/wasm-timer/releases)
- [Commits](https://github.com/tomaka/wasm-timer/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* grandpa: remove light-client specific block import pipeline (paritytech#7546)

* grandpa: remove light-client specific block import

* consensus, network: remove finality proofs

* client/authority-discovery: Publish and query on exponential interval (paritytech#7545)

* client/authority-discovery: Publish and query on exponential interval

When a node starts up publishing and querying might fail due to various
reasons, for example due to being not yet fully bootstrapped on the DHT.
Thus one should retry rather sooner than later. On the other hand, a
long running node is likely well connected and thus timely retries are
not needed. For this reasoning use an exponentially increasing interval
for `publish_interval`, `query_interval` and
`priority_group_set_interval` instead of a constant interval.

* client/authority-discovery/src/interval.rs: Add license header

* .maintain/gitlab: Ensure adder collator tests are run on CI

* sc-network: update some dependencies (paritytech#7582)

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Bump linregress and do some other cleanups (paritytech#7580)

* Wasm-builder 3.0 (paritytech#7532)

* Build every wasm crate in its own project with wasm-builder

Building all wasm crates in one workspace was a nice idea, however it
just introduced problems:

1. We needed to prune old members, but this didn't worked for old git
deps.
2. We locked the whole wasm workspace while building one crate. This
could lead to infinitely locking the workspace on a crash.

Now we just build every crate in its own project, this means we will
build the dependencies multiple times. While building the dependencies
multiple times, we still decrease the build time by around 30 seconds
for Polkadot and Substrate because of the new parallelism ;)

* Remove the requirement on wasm-builder-runner

This removes the requirement on wasm-builder-runner by using the new
`build_dep` feature of cargo. We use nightly anyway and that enables us
to use this feature. This solves the problem of not mixing
build/proc-macro deps with normal deps. By doing this we get rid off
this complicated project structure and can depend directly on
`wasm-builder`. This also removes all the code from wasm-builder-runner
and mentions that it is deprecated.

* Copy the `Cargo.lock` to the correct folder

* Remove wasm-builder-runner

* Update docs

* Fix deterministic check

Modified-by: Bastian Köcher <git@kchr.de>

* Try to make the ui test happy

* Switch to `SKIP_WASM_BUILD`

* Rename `SKIP_WASM_BINARY` to the correct name...

* Update utils/wasm-builder/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update utils/wasm-builder/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Bump tracing from 0.1.21 to 0.1.22 (paritytech#7589)

Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.21 to 0.1.22.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](tokio-rs/tracing@tracing-0.1.21...tracing-0.1.22)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* contracts: Add `salt` argument to contract instantiation (paritytech#7482)

* pallet-contracts: Fix seal_restore_to to output proper module errors

Those errors where part of the decl_error for some time but where
never actually returned. This allows proper debugging of failed
restorations. Previously, any error did return the misleading
`ContractTrapped`.

* Bind UncheckedFrom<T::Hash> + AsRef<[u8]> everywhere

This allows us to make assumptions about the AccoutId
that are necessary for testing and in order to benchmark
the module properly.

This also groups free standing functions into inherent functions
in order to minimize the places where the new bounds need to
be specified.

* Rework contract address determination

* Do not allow override by runtime author
* Instantiate gained a new parameter "salt"

This change is done now in expecation of the upcoming code rent
which needs to change the instantiation dispatchable and
host function anyways.

The situation in where we have only something that is like CREATE2
makes it impossible for UIs to help the user to create an arbitrary
amount of instantiations from the same code.

With this change we have the same functionality as ethereum with
a CREATE and CREATE2 instantation semantic.

* Remove TrieIdGenerator

The new trait bounds allows us to remove this workaround
from the configuration trait.

* Remove default parameters for config trait

It should be solely the responsiblity to determine proper values for
these parameter. As a matter of fact most runtime weren't using these
values anyways.

* Fix tests for new account id type

Because of the new bounds on the trait tests can't get away by using
u64 as accound id. Replacing the 8 byte value by a 32 byte value
creates out quite a bit of code churn.

* Fix benchmarks

The benchmarks need adaption to the new instantiate semantics.

* Fix compile errors caused by adding new trait bounds
* Fix compile errors caused by renaming storage and rent functions
* Adapt host functions and dispatchables to the new salt
* Add tests for instantiate host functions (was not possible before)

* Add benchmark results

* Adapt to the new WeightInfo

The new benchmarks add a new parameter for salt "s" to the instantiate weights
that needs to be applied.

* Fix deploying_wasm_contract_should_work integration test

This test is adapted to use the new instantiate signature.

* Break overlong line

* Break more long lines

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* */Cargo.toml: Remove unused dependencies (paritytech#7590)

* */Cargo.toml: Remove unused dependencies

Using cargo-udeps to detect unused dependencies.

* client/network/Cargo: Revert dependency removal

* Cargo.lock: Update

Co-authored-by: Andrew Plaza <aplaza@liquidthink.net>
Co-authored-by: Qinxuan Chen <koushiro.cqx@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Wei Tang <wei@that.world>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
darkfriend77 pushed a commit to mogwaicoin/substrate that referenced this pull request Jan 11, 2021
…ch#7546)

* grandpa: remove light-client specific block import

* consensus, network: remove finality proofs
This pull request was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove light-client specific GRANDPA block import
4 participants