Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependency downgrade on cargo update #5702

Open
debris opened this issue Jul 10, 2018 · 18 comments
Open

dependency downgrade on cargo update #5702

debris opened this issue Jul 10, 2018 · 18 comments
Labels
A-dependency-resolution Area: dependency resolution and the resolver Command-update S-triage Status: This issue is waiting on initial triage.

Comments

@debris
Copy link
Contributor

debris commented Jul 10, 2018

cargo 1.26.0 (0e7c5a931 2018-04-06)

I run cargo update -p snappy on parity project - rev bca100cdb047906e73d6733f43df73c7603897dd.

cargo output was ok:

    Updating git repository `https://github.com/paritytech/rust-snappy`
    Updating git repository `https://github.com/paritytech/ring`
    Updating registry `https://github.com/rust-lang/crates.io-index`
      Adding cmake v0.1.31
    Updating snappy v0.1.0 (https://github.com/paritytech/rust-snappy#40ac9a0d) -> #798408ff
    Updating snappy-sys v0.1.0 (https://github.com/paritytech/rust-snappy#40ac9a0d) -> #798408ff

but to my surprise, one of my dependencies has been downgraded with no apparent reason

diff --git a/Cargo.lock b/Cargo.lock
index c791ae4..d69a4c7 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -246,6 +246,14 @@ dependencies = [
 ]
 
 [[package]]
+name = "cmake"
+version = "0.1.31"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+dependencies = [
+ "cc 1.0.17 (registry+https://github.com/rust-lang/crates.io-index)",
+]
+
+[[package]]
 name = "common-types"
 version = "0.1.0"
 dependencies = [
@@ -1969,7 +1977,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 dependencies = [
  "num-integer 0.1.36 (registry+https://github.com/rust-lang/crates.io-index)",
  "num-traits 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
- "rand 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
+ "rand 0.3.20 (registry+https://github.com/rust-lang/crates.io-index)",
  "rustc-serialize 0.3.24 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
@@ -2765,7 +2773,7 @@ dependencies = [
  "lazy_static 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)",
  "libc 0.2.36 (registry+https://github.com/rust-lang/crates.io-index)",
  "num_cpus 1.8.0 (registry+https://github.com/rust-lang/crates.io-index)",
- "rand 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
+ "rand 0.3.20 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -3118,7 +3126,7 @@ dependencies = [
 [[package]]
 name = "snappy"
 version = "0.1.0"
-source = "git+https://github.com/paritytech/rust-snappy#40ac9a0d9fd613e7f38df800a11a589b7296da73"
+source = "git+https://github.com/paritytech/rust-snappy#798408ffef8f86dd51481673aca10f5348d7491b"
 dependencies = [
  "libc 0.2.36 (registry+https://github.com/rust-lang/crates.io-index)",
  "snappy-sys 0.1.0 (git+https://github.com/paritytech/rust-snappy)",
@@ -3127,9 +3135,9 @@ dependencies = [
 [[package]]
 name = "snappy-sys"
 version = "0.1.0"
-source = "git+https://github.com/paritytech/rust-snappy#40ac9a0d9fd613e7f38df800a11a589b7296da73"
+source = "git+https://github.com/paritytech/rust-snappy#798408ffef8f86dd51481673aca10f5348d7491b"
 dependencies = [
- "cc 1.0.17 (registry+https://github.com/rust-lang/crates.io-index)",
+ "cmake 0.1.31 (registry+https://github.com/rust-lang/crates.io-index)",
  "libc 0.2.36 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
@@ -3942,6 +3950,7 @@ dependencies = [
 "checksum chrono 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)" = "1cce36c92cb605414e9b824f866f5babe0a0368e39ea07393b9b63cf3844c0e6"
 "checksum cid 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "d85ee025368e69063c420cbb2ed9f852cb03a5e69b73be021e65726ce03585b6"
 "checksum clap 2.29.1 (registry+https://github.com/rust-lang/crates.io-index)" = "8f4a2b3bb7ef3c672d7c13d15613211d5a6976b6892c598b0fcb5d40765f19c2"
+"checksum cmake 0.1.31 (registry+https://github.com/rust-lang/crates.io-index)" = "95470235c31c726d72bf2e1f421adc1e65b9d561bf5529612cbe1a72da1467b3"
 "checksum conv 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "78ff10625fd0ac447827aa30ea8b861fead473bb60aeb73af6c1c58caf0d1299"
 "checksum crc 1.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "bd5d02c0aac6bd68393ed69e00bbc2457f3e89075c6349db7189618dc4ddc1d7"
 "checksum crossbeam 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "24ce9782d4d5c53674646a6a4c1863a21a8fc0cb649b3c94dfc16e45071dea19"

rand crate has been downgraded for num-bigint and rayon-core which require rand >= 0.3, < 0.5

@alexcrichton
Copy link
Member

Thanks for the report! This is definitely a bug in the output of cargo update at least in that it should inform you what's happening. Otherwise though this is a result of Cargo's crate graph resolution where it will pick the first crate graph that works (rather than optimizing for fewer crate versions). In that sense this is expected behavior in that this can happen, but further invocations of cargo update -p ... can help fix the issue.

cc @Eh2406, you might find this interesting as well!

@ehuss
Copy link
Contributor

ehuss commented Jul 10, 2018

I ran into the same thing with rand, see #5529.

@Eh2406
Copy link
Contributor

Eh2406 commented Jul 11, 2018

This is definitely a bug in the output of cargo update at least in that it should inform you what's happening.

Specifically #5530 with a WIP PR at #5634

@staktrace
Copy link

Similar issue: If you check out mozilla-central at 6f86cc3479f80ace97f62634e2c82a483d1ede40 and run cargo update -p webrender_api -p webrender in the root dir, it will downgrade the log 0.4.2 dependency of rsdparsa to log 0.3.9 for no apparent reason. The webrender dependency tree is completely independent of rsdparsa so it's not obvious to me why cargo update is changing the dependencies of rsdparsa at all. Note also that subsequently running cargo update -p gkrust reverts the log dependency back. gkrust is the root of the dependency tree.

@dwijnand
Copy link
Member

Note there was an attempt to fix this in #5718, that needs a bit more work.

@5chdn
Copy link

5chdn commented Oct 29, 2018

further invocations of cargo update -p ... can help fix the issue.

that's not solving the issue unfortunately.

@carols10cents
Copy link
Member

I believe we're seeing this in crates.io too, and I'm not sure why it's happening all of the sudden: rust-lang/crates.io#1607

We're not able to update any dependency right now, because whatever we change triggers the resolver to downgrade conduit-cookie and then cookie to an incompatible version, for no reason I can see.

I tried using rust 1.24.1 (an old stable i happen to have lying around) in case something changed with the resolver lately but it happens there too.

Any advice besides hand-managing Cargo.lock? Any information/logs I can supply that would be helpful?

@ehuss
Copy link
Contributor

ehuss commented Jan 28, 2019

@carols10cents I think it is definitely due to the ring yanks. All the versions were yanked on the 10th. I tried updating crates.io from an index locked before then, and it is able to update dependencies just fine.

I think the reason is because when the resolver runs, the yanked packages are no longer visible, so it decides it has no other choice but to downgrade conduit-cookie to find a version that doesn't need ring. (I'm not certain, but it looks that way to me.)

You can use some flags as a workaround. For example, to update itertools, update Cargo.toml and run cargo update -p itertools --precise 0.8.0. For log, you can update it with cargo update -p log:0.4.5.

It would be nice if when using -p that it would be more careful about not downgrading unrelated packages.

@carols10cents
Copy link
Member

carols10cents commented Jan 28, 2019

the yanked packages are no longer visible

The way I understood yanking to be is that if a crate was yanked, you couldn't start depending on a crate, but existing dependencies in your lockfile would be allowed. We had a lockfile with ring in it with a patch to a github branch+hash even, so crates.io should be allowed to continue depending on that version of ring even if it was yanked... right?

If this is a separate bug, I'm happy to move!

@ehuss
Copy link
Contributor

ehuss commented Jan 28, 2019

right?

I'm not sure. I don't know much about the resolver, so I'll leave it to @Eh2406 or @alexcrichton to explain more. At a minimum I expect cargo update -p somecrate to leave the unrelated crates untouched. Whether or not a bare cargo update should continue to use yanked entries in the lockfile is questionable.

I crated a reproduction for anyone who wants to look at it:

  1. Download: https://gist.github.com/ehuss/cd5e090d06a9af26add19650bad988b4
  2. Create an empty src/lib.rs
  3. Run cargo update -p itertools

This should only update itertools to 0.8. However, it ends up re-resolving everything else (and thus downgrading "cookie" because "ring" is yanked and unavailable). If you remove this line, then it works as expected (only updates itertools). Perhaps it is filtering out yanked summaries too early?

@alexcrichton
Copy link
Member

I extracted @ehuss's comment to a separate bug because I agree that it's unrelated to this issue specifically, as this was previously largely about "cargo does this naturally and there's little we can do, but we should message it better in the message output of cargo update"

@carols10cents you mentioned though that 1.24.1 didn't work for you, but the bug exposed in #6609 should have worked just fine on 1.24.1 so it may mean there's another bug in play here. Do you have a way I can reproduce on nightly/stable today so I can dig into that bug more?

@carols10cents
Copy link
Member

carols10cents commented Jan 28, 2019

@carols10cents you mentioned though that 1.24.1 didn't work for you, but the bug exposed in #6609 should have worked just fine on 1.24.1 so it may mean there's another bug in play here. Do you have a way I can reproduce on nightly/stable today so I can dig into that bug more?

UGH I lied. I have a rustup override set in my crates.io directory so I didn't actually switch to 1.24.1. When I actually switch to 1.24.1 and update itertools, only itertools updates as I expected. I'm sorry for messing that up.

Sorry this isn't reduced yet, but the way I'm reproducing is by checking out this commit of crates.io, change itertools to 0.7.0 in Cargo.toml, and run cargo update -p itertools. If only itertools is updated in Cargo.lock, that's the behavior I expect. If itertools is upgraded, but conduit-cookie and cookie are downgraded and ring is removed, that's the problem I'm having. I reset the Cargo.lock between each attempt.

The problem behavior is happening for me with stable 1.32.0, beta 1.33.0-beta.4, and nightly 1.34.0-nightly (da6ab956e 2019-01-27).

@alexcrichton
Copy link
Member

Ok cool, thanks for the confirmation! No need to worry about a reproduction/reduction in that case, I think #6609 has everything it needs for fixing now.

ordian added a commit to paritytech/substrate that referenced this issue Jul 15, 2020
ordian added a commit to paritytech/substrate that referenced this issue Jul 15, 2020
* sc-offchain: upgrade hyper-rustls to 0.18

* Cargo.lock: revert rand downgrade due to rust-lang/cargo/issues/5702
drewstone added a commit to hicommonwealth/substrate that referenced this issue Sep 8, 2020
* Update stale docstring with 'EnsureOneOf' introduced. (#6501)

* Update stale docstring with 'EnsureOneOf' introduced.

* Apply review suggestions.

* Add auth-label-issues.yml (#6488)

* Remove /self from mandatory rpc reviews. (#6507)

* Change contract fees to MILLICENTS (#6509)

* Re-enter runtime after resetting overlay from runtime (#6513)

This still assumes that the client did not start any transactions
before calling into runtime. This is the case for benchmarking
as long as either NativeWhenPossible or AlwaysWasm exection
strategy is chosen. Using any other will result in a panic.

* Releasing rc4 – Rhinoceros (#6515)

Co-authored-by: Martin Pugh <pugh@s3kr.it>

* Fix an extra semi-colon yielding a wrong error (#6520)

* Fix an extra semi-colon yielding a wrong error

* Update client/cli/src/commands/run_cmd.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Refactor as_sub to make things clearer. (#6503)

* Refactor as_sub to make things clearer.

- `as_sub` becomes `as_alternative`
- `as_sub_limited` becomes `as_derivative`
- `as_alternative` and `as_derivative` generate a mutually exclusive set of accounts.

* Test fix

* Add test

* Fix test

* Remove `as_alternative`.

* Docs.

* [CI] Fix warning in polkadot-companion-label action (#6514)

* Remove @cecton from CODEOWNERS (#6524)

* Initial commit

Forked at: 397068219c7a43a743f3979947efec35a8e93536
Parent branch: origin/master

* Remove @cecton from CODEOWNERS

* Impl From<UncheckedExtrinsic> for OpaqueExtrinsic (#6522)

* Implement Contains for pallet-membership (#6518)

* implement Contains for pallet-membership

* bump version

* Increase the limit for the maximum size of the telemetry name (#6523)

* Increase the limit for the maximum size of the telemetry name

* Fix test

* Fix some broken benchmarks (#6528)

* Implement `()` for `Happened` (#6529)

* Allow `retract_tip` on `tip_new` (#6511)

* Allow `retract_tip` on `tip_new`

* initial migration code

* test migration

* make pub

* bump spec

* Update to libp2p v0.20.1 (#6465)

* Update to libp2p-0.20.0

* Update to `libp2p-0.20.1`.

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Remove unecessary &mut in call argument (#6540)

* feat: Allocate ss58 address format to DataHighway (#6530)

* Fix tx-pool returning the same transaction multiple times (#6535)

* Fix tx-pool returning the same transaction multiple times

This fixes a bug that lead to returning the same transaction multiple
times when iterating the `ready` iterator. Internally the transaction
was kept in the `best` list and could be duplicated in that list be
re-inserting it again. This `best` list is using a `TransactionRef`
which internally uses a `insertion_id`. This `insertion_id` could lead
to the same transaction being inserted multiple times into the `best`
list.

* Update client/transaction-pool/src/testing/pool.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Remove the service, replacing it with a struct of individual chain components (#6352)

* WIP

* Making progress

* Almost ready

* Get service tests compiling

* Fix node screenshot

* Line widths

* Fix node cli tests

* Fix node cli warning

* ChainComponents -> ServiceComponents, fix tests

* make spawn_handle public

* Remove spawnnamed impl for taskmanager

* Move the keep alive stuff to the task manager

* Move the telemetry, base path, rpc keep_alive to the service builder

* Make the task manager keep alive an internal detail

* Rewrite the browser start_client future

* Remove run_node etc

* Revert my personal changes to browser-demo/build.sh

* use |config|

* Add a runtime_version function to SubstrateCli

* Reexport role and runtime version from sc cli

* Update Cargo.lock

* runtime_version -> native_runtime_version

* Pass chain spec to native_runtime_version for polkadot

* Fix line widths

* Traitify ServiceComponents Client

* Block production integration benchmark (#6468)

* proposer benchmark

* update cargo.lock

* Make the `OnRuntimeUpgrade` docs more clear (#6542)

* Support synching of blocks that are not `new_best` (#6508)

* Start

* Remove debug println

* Add tests

* grandpa: minor cleanups in communication module (#6371)

* grandpa: replace Result<(), ()> with Option<()>

* grandpa: replace &Option<T> with Option<&T>

* grandpa: cleanup local id and keystore usages

* grandpa: return bool on check_message_signature

* grandpa: fix erroneous log message on startup

* grandpa: fix test

* Update to make cargo-deny happy (#6547)

* Update to make cargo-deny happy

* Remove cargo deny from CI

* change (ci): run cargo deny only on tags and schedules

Co-authored-by: Denis P <denis.pisarev@parity.io>

* pallet-evm: customizable chain id (#6537)

* Fix runtime benchmarks CI (#6545)

* debug (ci): ci config [skip ci]

* debug (ci): fix runtime benchmarks

* fix identity benchmarks

* fix utility benchmarks

* Revert "debug (ci): ci config [skip ci]"

This reverts commit 081b175b5e95604520c79ea4e5822b84ea35ddaa.

* change (ci): touch ci config to run CI

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Gav Wood <gavin@parity.io>

* Fix mocking multiple http calls in the same function call (#6510)

* Fix mocking multiple http calls in the same function call

Fixes an issue where a function call would perform more than one http request and wait for each to complete before proceeding. The `RequestId` comes from the length of the `requests` collection in the `OffchainState` and if a request is completed before the next one starts it will be removed and the "next expected" will be off by one. This PR tries to fix that by using a request counter that tracks how many requests have been performed so that we can `remove()` items from the `expected_requests` at the right index.

I suspect that this is a sub-optimal soluton and perhaps requests and their mocks should live side by side in the same collection, e.g. in a tuple of `(PendingRequest, Option<ExpectedRequest>)`.

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Panic on overflow

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Use a Deque and push/pop expected requests

* fix test

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* .maintain/monitoring/alerting-rules: Adjust transaction queue size alert (#6426)

The transaction queue size alert has been firing with a constant 10
transactions in the queue. While maybe problematic those 10 transactions
don't need to be the same across scrape intervals.

Instead of alerting with a size above 10, alert based on two things:

1. Monotonically increasing queue size

2. Upper limit queue size reached

* Fix auto-label-issues.yml (#6536)

statements in github actions cannot use ", must use '

Co-authored-by: Gav Wood <gavin@parity.io>

* [CI] Add Github Action to notify devops of PRs labelled with A1-needsburnin (#6525)

* add burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* Update .github/workflows/burnin-label-notification.yml

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Gav Wood <gavin@parity.io>

* Restrict `Protected` to some heap types. (#6471)

* Restrict `Protected` to some heap types.

* Comment abut Protected usage.

* Remove Protected from crypto, use secrecy crate for existing uses.

* use a parse function

* fix error convert

* Rename and move secretY string function.

* std result

* Remove `ServiceBuilderCommand` and implement the chain ops as standalone functions instead. (#6543)

* :)

* Slight tidy

* Remove ServiceBuilderCommand

* Remove whitespace

* Keep task manager alive for check_block/import_blocks

* Pass task_manager to run_until_exit

* Make task_manager in run_until_exit and make subcommands async

* Change the async_run fn to return a future and task manager

* async_run should take a result fn

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix spaces in export_raw_state

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Drop the tokio runtime before the task_manager (#6548)

* Initial commit

Forked at: 4919c808cb75618d95762944aa6f5664c1aa3b59
Parent branch: origin/master

* Drop the tokio runtime before the task_manager

The tokio runtime must be dropped before the task_manager. Otherwise the objects the task_manager
keep alive are dropped before the tasks are finished.

* Update SubstrateCli to return String (#6550)

* Update SubstrateCli to return String

* Add default implementation for executable_name()

* Use display instead of PathBuf

* Get file_name in default impl of executable_name

* Remove String::from and use .into()

* Use default impl for executable_name()

* Use .as_str() and remove useless .to_string()

* Update only sp-io when running companion build

* Remove unneeded update of sp-io in CI

Co-authored-by: Cecile Tonglet <cecile@parity.io>

* Allow specify schedule dispatch origin (#6387)

* allow specify schedule dispatch origin

* fix tests

* use caller origin for scheduled

* fix tests

* line width

* check origin for cancel

* line width

* fix some issues for benchmarking

* fix doc test

* another way to constraint origin

* fix build issues

* fix cancel

* line width

* fix benchmarks

* bump version

* enable runtime upgrade

* add migration code and test

* Update frame/scheduler/src/lib.rs

Co-authored-by: Gavin Wood <github@gavwood.com>

* expose migration method

* add notes

* bump version

* remove on_runtime_upgrade

* fix test

Co-authored-by: Gavin Wood <github@gavwood.com>

* Move `create_inherents` into the block-builder (#6553)

* Move `create_inherents` into the block-builder

This moves the `create_inherents` call into the block-builder. This has
the advantage that `create_inherents` will be able to reuse the same
context that will be used when applying the extrinsics and we also save
one call to `on_initialize`. To make sure that `create_inherents` does
not modify any state, we execute it in a transaction that is
rolled-back after doing the runtime call.

* Feedback and build fix

* Update primitives/runtime/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Update client/block-builder/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* client/network: Remove unused Result returned by NetworkWorker (#6552)

* New testing helpers (#6555)

* Initial commit

Forked at: 8ef1ac0ee13d2a72cc1c391d4624dfaaafe641e8
Parent branch: origin/master

* Add send_transaction to RpcHandlers

* Extension trait for RpcHandlers

* Revert "Add send_transaction to RpcHandlers"

This reverts commit 03c89e13d404bae3f3123387dd50f026061bca82.

* Add an extension trait for BlockchainEvents

* Update test-utils/client/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update test-utils/client/src/lib.rs

* fix

* deps fix

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove polkadot companion detection from branch name  (#6565)

* Initial commit

Forked at: facc6741400a21fa3b5a3eedfe444f30ab5df8b0
Parent branch: origin/master

* Remove polkadot companion detection from branch name

Even though it was nice it was also error prone as there were no indication whatsoever on the PR
that a polkadot companion branch exists.

* primitives: use generic Header in testing runtime (#6561)

* primitives: use generic Header in testing runtime

* frame: remove unused imports

* Remove warning

Co-authored-by: Bastian Köcher <git@kchr.de>

* Remove polkadot companion detection from branch name  (#6568)

* Initial commit

Forked at: 7e3cf944cf87a22036cc26f03bd859855b7ed3f1
Parent branch: origin/master

* Remove polkadot companion detection from branch name

Even though it was nice it was also error prone as there were no indication whatsoever on the PR
that a polkadot companion branch exists.

* Make the encoded-Call Vec<u8> explicitly so in metadata (#6566)

* ci: fix merge on polkadot companion job (#6574)

* pallet-evm: return Ok(()) when EVM execution fails (#6493)

* pallet-evm: return Ok(()) when EVM execution fails

* Bump spec version

* Init test module

* Add fail_call_return_ok test

* Fix tests and use full match pattern

Co-authored-by: Gav Wood <gavin@parity.io>

* babe: report equivocations (#6362)

* slots: create primitives crate for consensus slots

* offences: add method to check if an offence is unknown

* babe: initial equivocation reporting implementation

* babe: organize imports

* babe: working equivocation reporting

* babe: add slot number to equivocation proof

* session: move duplicate traits to session primitives

* babe: move equivocation stuff to its own file

* offences: fix test

* session: don't have primitives depend on frame_support

* babe: use opaque type for key owner proof

* babe: cleanup client equivocation reporting

* babe: cleanup equivocation code in pallet

* babe: allow sending signed equivocation reports

* node: fix compilation

* fix test compilation

* babe: return bool on check_equivocation_proof

* babe: add test for equivocation reporting

* babe: add more tests

* babe: add test for validate unsigned

* babe: take slot number in generate_key_ownership_proof API

* babe: add benchmark for equivocation proof checking

* session: add benchmark for membership proof checking

* offences: fix babe benchmark

* babe: add weights based on benchmark results

* babe: adjust weights after benchmarking on reference hardware

* babe: reorder checks in check_and_report_equivocation

* Benchmarks Writer CLI (#6567)

* initial mockup

* add and wipe

* track writes

* start to add to pipeline

* return all reads/writes

* Log reads and writes from bench db

* causes panic

* Allow multiple commits

* commit before ending benchmark

* doesn't work???

* fix

* Update lib.rs

* switch to struct for `BenchmarkResults`

* add to output

* fix test

* line width

* @kianenigma review

* Add Whitelist to DB Tracking in Benchmarks Pipeline (#6405)

* hardcoded whitelist

* Add whitelist to pipeline

* Remove whitelist pipeline from CLI, add to runtime

* clean-up unused db initialized whitelist

* Add regression analysis to DB Tracking (#6475)

* Add selector

* add tests

* debug formatter for easy formula

* initial idea

* use all benchmarks

* broken

* working without trait

* Make work for multiple pallets

* Fix merge issues

* writer appends to file

* implement () for balances weight trait

* update name of trait

* Weights to WeightInfo

* auto trait writer

* Heap pages are configurable

* clean out runtime changes

* more clean up

* Fix string generation

* Update comments

* Update bin/node/runtime/src/lib.rs

Co-authored-by: arkpar <arkady.paronyan@gmail.com>

* Don't require module name in inherents (#6576)

* Start

* Cleanup `construct_runtime!`

* Add tests

* Fix after merge

* Update the docs

* Derive `RuntimeDebug` for `Runtime` (#6581)

* Fix UI tests (#6589)

* Use async/await in build_network_future (#6533)

* Use async/await in build_network_future

* Address concerns

* Fix test

* decl_module! macro: use 'frame_system' instead of `system` as default ident (#6500)

* Use frame_system as default ident.

* Remove unused 'frame_system' to 'system' renaming.

* Fix construct_runtime_ui tests.

* Rename system to frame_system in sudo/utility pallet test.

* Bump runtime impl_version.

* Update formatting.

* Upgrade `kvdb-*`, `trie-db` and `memory-db` (#6584)

* Upgrade `kvdb-*`, `trie-db` and `memory-db`

The updates of `trie-db` and `memory-db` are important, as they fix the
non-deterministic build of Polkadot/Substrate.

* Change `trie-db` version

* Update test-utils/runtime/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update primitives/trie/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update `Cargo.lock` and `trie-bench`

* Fix UI tests

* Switch to fixed version of memory-db

Co-authored-by: Andronik Ordian <write@reusable.software>

* client/network: Rename DebugInfoBehaviour to PeerInfoBehaviour (#6556)

Information retrieved via `DebugInfoBehaviour` is not only used for
debugging purposes, e.g. disconnecting from nodes not responding to
pings, using external addresses retrieved via indentify, ...

In order for the name to reflect the usage of the module this commit
renames it.

* Add log rotation (#6564)

* Use flexi_logger; Add log rotation

* Add default rotation; Add FlexiLogger error

* Fix compilation error

* Remove logging to stdout if it's not a tty

* Fix formatting

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove needless debug statement

* Default to unlimited size for log rotation

* Add more comments about log-age option

* Remove unused variable

* Fix typo in comment

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Send Status message on all newly-opened legacy substreams (#6593)

* Send Status message on all newly-opened legacy substreams

* Fix tests

* *: Update to libp2p v0.21.1 (#6559)

* *Cargo.toml: Update versions

* client/network/src/discovery: Adjust to Kademlia  API changes

* client/network: Adjust to one_shot.rs changes

* client/network/discovery: Log address list on trace level

* client/network/discovery: Ignore RoutablePeer and PendingRoutablePeer

* Commit Cargo.lock

* Finish update

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Improve transaction submission (#6599)

* Improve transaction submission

Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.

Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.

* Make sure we don't submit the same transaction twice from the network concurrently

* Remove added listener call

* Feedback

* Ignore banned on resubmit

* Add `WeightInfo` to all pallets with benchmarks. (#6575)

* Start adding weight info

* More weightinfo

* finish weight info

* more fixes

* inital update of node runtime

* fix the rest of the compilation

* update balances

* add docs

* fix balances tests

* Fix more tests

* Fix compile

* Fix pallet-evm tests

* Fix build when with-kvdb-rocksdb is disabled. (#6546)

* Make evm errors public (#6598)

* Improved send_transaction helper to return an error in case of error  (#6592)

* Rename `CheckEra` to `CheckMortality` (#6619)

* Add an authority_discovery_known_authorities_count metric (#6614)

* Clean exit when no space left on device (#6339)

Fixes #6305

* seal: Rework contracts API (#6573)

* Transition getter functions to not use scratch buffer

* Remove scratch buffer from ext_get_storage

* Remove scratch buffer from ext_call

* Remove scratch buffer from ext_instantiate

* Add ext_input and remove scratch buffer

* Rework error handling (changes RPC exposed data)

* ext_return passes a flags field instead of a return code
	* Flags is only for seal and not for the caller
	* flags: u32 replaced status_code: u8 in RPC exposed type
* API functions use a unified error type (ReturnCode)
* ext_transfer now traps on error to be consistent with call and instantiate

* Remove the no longer used `Dispatched` event

* Updated inline documentation

* Prevent skipping of copying the output for getter API

* Return gas_consumed from the RPC contracts call interface

* Updated COMPLEXTITY.md

* Rename ext_gas_price to ext_weight_to_fee

* Align comments with spaces

* Removed no longer used `ExecError`

* Remove possible panic in `from_typed_value`

* Use a struct as associated data for SpecialTrap::Return

* Fix nits in COMPLEXITY.md

* Renamed SpecialTrap to TrapReason

* Fix test

* Finish renaming special_trap -> trap_reason

* Remove no longer used get_runtime_storage

* fixup! Remove no longer used get_runtime_storage

* Removed tabs for comment aligment

* Simplify a few chain components creation APIs related to the service (#6611)

* Simplify a few chain components creation APIs related to the service

* Fix basic-authorship doc tests

* Remove DefaultQueue

* Update client/service/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Move ExecutionExtensions comment around

* Remove unused BlakeTwo256

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Make `init_logging` more easily usable (#6620)

Instead of requiring the `LogRotationOpt`, it now requires an
`Option<LogRotationOpt>`. This makes it much more easy to use the
interface when someone isn't interested on the `LogRotationOpt`'s

* add ios to cfg gates in client/service (#6622)

* done

* sysinfo should not be not android

* add back sysinfo dep to satisfy web wasm ci

* Node template love (#6560)

* Node template love

Better description in node/Cargo.toml
Example dispatchables should have non-zero weights
Changes to README to bring in line with current DevHub Node Template

* Update README.md

cleaned up, reworked text, ready for round 2

* Add DbWeight

* Update README.md

* Consistent Playground buttons

* "setup" -> "set up"

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Import `Get` trait to fix build

Co-authored-by: Addie Wagenknecht <addie@nortd.com>
Co-authored-by: Ricardo Rius <ricardo@parity.io>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* change ss58 format for SubstraTEE from 44 to 13 and add ChainX (#6602)

* Deduplicate some dependencies (#6628)

* Revert "Add log rotation (#6564)" (#6627)

This reverts commit 802a0d0b0ade796a3b2d4663212518315923fe8a.

* seal: Prevent contracts from going below subsistence (#6623)

* seal: Do not allow transfers to bring total balance below subsistence deposit

This also reworks the rent system to take the total balance into account
when evaluating whether the account is above the subsistence deposit.

* Fix nits from review

* Fix typo

* Do not enforce subsistence when called from EOA

* Rename CallOrigin to TransactorKind

* Add debug asserts to check the invariants of a plain account transactor

* Fix typo

Co-authored-by: Sergei Shulepov <sergei@parity.io>

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Remove warning about networking taking a lot of time (#6630)

* Remove warning about networking taking a lot of time

* Fix warnings

* Add FromStr for AccountId32 (#6621)

* Add FromStr for AccountId32

* Review grumbles

* Clarify some host interfaces (#6626)

* Remove duplicate comments (#6638)

* Make signature batching use specialized methods (#6616)

It was a mistake to use the `*_verify` methods for signature batching.
This pr move the signature batching into their own functions. This is
required, because otherwise transaction signature verification infers
with other signature verifications.

This pr also temporarily disables signature batching. The functionality
stays, but we need to make sure that all nodes have the new runtime
interface, before we can bring back signature batching.

* Check that `serde` is present in `sc-chain-spec-derive` (#6636)

We should check that `serde` is present when using
`sc-chains-spec-derive`, because we require this for the generated
struct. This also switches the generated `derive` instruction to import
`Serialize`/`Deserialize` by path to not require the user to import
these derive macros.

* ci: add deployment of polkadot prometheus rules (#6646)

* Implement apply_state flag and allow fetching return data and used gas  (#6590)

* pallet-evm: return Ok(()) when EVM execution fails

* Bump spec version

* Implement apply_state flag and allow fetching return data and used gas

* Update evm version

* Remove legacy Clogged event (#6652)

* Fix two minor warnings (#6650)

* Unused import in no_std builds
* Global attribute in non-root of a crate

* allow to specify schedule time as a relative value (#6578)

* allow to specify schedule time as a relative value

* bump version

* line width

* fix benchmarks

* rename and updates

* Move the legacy protocol handshake to the legacy substream (#5938)

* Move the legacy protocol handshake to the legacy substream

* Fix tests

* Remove line that wasn't supposed to be committed

* Remove hack

* Rework how it's done

* Some little changes

* update_chain wasn't doing its thing

* Fix service tests not calling update_chain

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* [WIP]

* Revert "[WIP]"

This reverts commit 2b892e6a7637c0b1297e6ecdbb919321c9098ff5.

* Update client/network/src/protocol.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix received message not being handshake

* Update client/network/src/protocol/generic_proto/behaviour.rs

Co-authored-by: Max Inden <mail@max-inden.de>

Co-authored-by: Max Inden <mail@max-inden.de>

* [CI] Fix check_polkadot_companion_status.sh (#6631)

* check changes_requested and approved separately

* handle checking more than 1 CHANGES_REQUESTED review

* sc-offchain: upgrade hyper-rustls to 0.18 (#6663)

* sc-offchain: upgrade hyper-rustls to 0.18

* Cargo.lock: revert rand downgrade due to rust-lang/cargo/issues/5702

* Support Component-less Runtime Benchmarks (#6645)

* Update benchmarking macro for no components

* Handle output when error

* skip when empty

* Update analysis for zero components

* add back trace logs

* Apply suggestions from code review

* remove mean value, and use median value

* Add note

* Use standard for loop

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove polkadot companion detection from url anywhere  (#6570)

* The telemetry should not be initialized if no telemetry has been provided (#6666)

* Initial commit

Forked at: 4da29261bfdc13057a425c1721aeb4ec68092d42
Parent branch: origin/master

* Fix: do not initialize telemetry if telemetry is not set

* Update client/telemetry/src/lib.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Update client/telemetry/src/lib.rs

* Update client/telemetry/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* fix comment (#6670)

* Regression test to ensure we don't break deterministic builds in wasm (#6597)

* Regression test to ensure we don't break deterministic builds in wasm again

* Apply suggestions from code review

Co-authored-by: s3krit <pugh@s3kr.it>

* Update .gitlab-ci.yml

Co-authored-by: s3krit <pugh@s3kr.it>

Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update substrate-networking Grafana dashboard (#6649)

* client/service/metrics: Rework metric HELP texts (#6647)

* .maintain/monitoring/alerting-rules: Remove HighCPUUsage alert (#6648)

The `HighCPUUsage` alert is based on the `cpu_usage_percentage` metric.
Instead of exposing the overall CPU usage in percent, the metric exposes
the per core usage summed over all cores.

This commit removes the alert for two reasons:

1. Substrate itself does not expose the core count and thus one can not
alert based on the `cpu_usage_percentage` metric.

2. Alerting based on CPU usage is generic and not specific to Substrate
or Blockchains. Thus any CPU usage alert suffice.

* identity: Don't let subs be re-registered (#6667)

* Fixes and tests

* Don't set subs be re-registered.

Also allow subs to de-register themselves and collect the deposit.

Also allow individual registering and removal of subs.

* Make it build

* Update frame/identity/src/lib.rs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Tests

* Add benchmarks

* Add some reasonable weights

* Docs

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Treasury burning can be directed (#6671)

* Treasury burning can be directed

Also, Society is a imbalance handler

* Build

* Introduce from_permill in perthings.

* Rename to from_perthousand to avoid confusion with Permill

* Fixes

* Switch to bounded mpsc for txpool import notification stream (#6640)

* Switch to bounded mpsc for txpool import notification stream

* Update client/transaction-pool/graph/src/validated_pool.rs

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* grandpa: report equivocations with unsigned extrinsics (#6656)

* grandpa: use unsigned extrinsics for equivocation reporting

* grandpa: allow signed reports as well

* grandpa: change runtime api for submitting unsigned extrinsics

* grandpa: fix tests

* grandpa: add test for unsigned validation

* grandpa: add benchmark for equivocation proof checking

* offences: fix grandpa benchmark

* grandpa: add proper weight for equivocation reporting extrinsic

* grandpa: fix weight unit

* Transaction pool integrated benchmarks (#6579)

* txpool benchmarks

* updated api

* Update bin/node/bench/src/txpool.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Change unknown-commit to unknown to make parsing easier (#6675)

* done (#6673)

* Fix tx-payment integrity-test (#6633)

* Fix tx-payment integrity-test

* Better error message.

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Fix line width

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Fix grumble

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* fix check polkadot companion status check (#6688)

* Fix compilation problems due to ed25519-dalek 1.0.0-pre.4 (#6676)

* Fix compilation problems due to ed25519-dalek 1.0.0-pre.4

* Update libp2p-core

* Revert rand downgrade

* More app-friendly event description (#6684)

* More app-friendly event description

* change origin -> owner

* checked all decl_event! and changed decriptions.

* annotated parameter names for remaining events

* Update libp2p to 0.22. (#6690)

* Update libp2p to 0.22.

Primarily to begin the libp2p-noise upgrade process.

See https://github.com/libp2p/rust-libp2p/issues/1631.

* CI

* Add try_mutate_exists to StorageDoubleMap. (#6694)

* add ecdsa benches (#6697)

* Fixup `cargo unleash` (#6689)

* Bumping to rc5

* Revert "Bumping to rc5"

This reverts commit 3c6767db3d7f94e164c0f08986085fb98bd883ce.

* Bumping for rc5, correctly this time

* Add sync_legacy_requests_received metric (#6698)

* Improve overall performance (#6699)

* Improve overall performance

* Clean up code

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove needless ::

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Remove dead link to out-of-date style guide (#6682)

* Remove dead link to out-of-date style guide

* Replace dead link with self-hosted doc

* Use relative link to style guide

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* Format style guide

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* Formatting

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* "cargo test" jobs optimization (#6606)

* change (ci): 3 jobs in 1 decreases concurrency and is more effectiv; w/o release it's ~20% faster, but needs testing on prod; wasmtest tests are already running within cargo test --workspace

* fix (test): these ones were failing on nightly

* save: cargo profiles [skip ci]

* change (ci): one test to run them all

* change (ci): rebase

* Revert "change (ci): rebase"

This reverts commit 8a6b7ea043a460bf71526ccaa4c7a68899a3b2bc.

* fix (config): fix manifest

* change (ci): bench release

* Properly filter out duplicate voters in elections. (#6693)

* Prevent duplicate voter

* Update primitives/npos-elections/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* pallet-swap-action: Change `BalanceSwapAction` signature (#6580)

Instead of requiring `T: Trait` in `BalanceSwapAction`, we directly
depend on `AccountId`. This fixes a compilation error on wasm, where
`Runtime` does not implement `Debug`, but `BalanceSwapAction` required it.

* make impl_outer_origin default to use frame_system (#6695)

* Removing staging resources from node template README (#6701)

* Graceful shutdown for the task manager (#6654)

* Initial commit

Forked at: 60e3a693b29789045614e2ed73126695bc8b0794
Parent branch: origin/master

* Move task_manager.rs to mod.rs

* Graceful shutdown for the task manager

* Await all background task JoinHandle at the same time

* Add tests

* Make future() wait also for exit signal + fix essential task failed

Probably related to https://github.com/paritytech/cumulus/issues/111

* add comments for non-obvious code

* Use clean_shutdown() in sc-cli

* Adapt code and upgrade tokio in sc-cli

* cleanup spacing in doc

* Add license

* I guess actually running the clean shutdown would be a good idea

* fix tests

* Update client/cli/src/runner.rs

Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>

* Improve error logging

* disable other tests (can't reproduce on my machine)

* Revert "disable other tests (can't reproduce on my machine)"

This reverts commit c133c590f33c253123ba0555ce719a71ededd60d.

* It is possible that the tasks are ended first

* Revert "It is possible that the tasks are ended first"

This reverts commit 502aba4a49fb5d892e704c412b8a81768a3f2c71.

* Use single threaded scheduler for more predictability

* enable_time

* Revert "enable_time"

This reverts commit 4e152140764a4bddeedff06a4e36ec701909e8c7.

* Revert "Use single threaded scheduler for more predictability"

This reverts commit ee5e13c5f13ff71e012dcda13579dffeb15f8ffc.

* Revert "Revert "It is possible that the tasks are ended first""

This reverts commit 1b91a8ca3eebbdc18be199c8ca188e88669ae649.

* This cannot be verified either with a threaded pool

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Benjamin Kampmann <ben@parity.io>
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Reset flaming fir network (#6703)

* Tracing values (#6679)

* Switch to serde_json::Value for Visitor values

* add tests

* switch Visitor to use Map

* refactor change names

* \n

* update integration test

* use discrete maps for each type of recorded value

* update integration test

* add docs

* Update client/tracing/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* add docs

Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update to latest sysinfo prevents leaking fd-handlers (#6708)

* Ensure we close old file handles hold by sysinfo

* Dropping is needed unfortunately

* enable process refreshing, ignore result from refresh_process

* jumping to proposed patch

* switch to latest sysinfo

* Update wasmtime to (almost) lastest master (#6662)

* update wasmtime to (almost) lastest master

* bump lock

* bump once more

* Remove the service builder (#6557)

* :)

* Slight tidy

* Remove ServiceBuilderCommand

* Remove whitespace

* Keep task manager alive for check_block/import_blocks

* Pass task_manager to run_until_exit

* WIP

* WIP

* Get rid of the macros

* Simplify a few chain components creation APIs related to the service

* Fix basic-authorship doc tests

* Remove DefaultQueue

* Update client/service/src/builder.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Move ExecutionExtensions comment around

* Remove unused BlakeTwo256

* Add sc-prelude

* Rename sc-prelude to sc-service-prelude

* Rename to sc-service-types

* Improve service types

* Fix line widths

* Remove sc-service-types and move type definitions to crates

* Update bin/node-template/node/src/service.rs

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>

* Add TLightClientWithHash

* Rework types

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>

* Better handling of stable-only build (#6569)

* Better handling of stable-only build

* Fix node template build

* Fix wasm builder node-template version mismatch

* Fix load_spec error

* Add , in parameter

* Add descrptive panic messages in tests

* Add descriptive tests in node/executor benches

* Fix missing compact_code_unwrap

* Add missing wasm_binary_unwrap function for executor integration test

* Only define import_sp_io in no_std

* Small Cargo.toml styling fix

* Bump wasm-builder to 2.0.0

* Fix all `with_wasm_builder_from_crates` version in Substrate

* Use `with_wasm_builder_from_crates` for node-template

Co-authored-by: Gavin Wood <gavin@parity.io>

* Remove clear_on_drop from browser util crate (#6710)

* Add 4 as address type of ss58 for Katal Chain (#6713)

* [CI]Chaostest suite initiation (#5793)

* Initiate chaostest cli test suite: singlenodeheight on one dev node
    Added chaostest stages in CI
    Added new docker/k8s resources and environments to CI
    Added new chaos-only tag to gitlab-ci.yml

* Update .maintain/chaostest/src/commands/singlenodeheight/index.js

Co-authored-by: Max Inden <mail@max-inden.de>

* change nameSpace to namespace(one word)

* update chaos ci job to match template

* rename build-pr ci stage to docker [chaos:basic]

* test gitlab-ci [chaos:basic]

* Update .gitlab-ci.yml

* add new build-chaos-only condition

* add *default-vars to singlenodeheight [chaos:basic]

* change build-only to build-rules on substrate jobs [chaos:basic]

* test and change when:on_success to when:always [chaos:basic]

* resolve conflicts and test [chaos:basic]

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Support using system storage directly for EVM balance and nonce (#6659)

* Augmented node template docs (#6721)

* Switching from git back to released versions for wasmtime, fix cargo-unleash (#6722)

* Switching from git back to released versions for wasmtime

* filter out cratelift_codegen messages-a

Co-authored-by: NikVolf <nikvolf@gmail.com>

* adding changelog (#6728)

* fixing CI

* remove breaking excepts

* grandpa: allow noting that the set has stalled (#6725)

* grandpa: remove unused methods to convert digest

* grandpa: add root extrinsic for scheduling forced change

* grandpa: add benchmark for schedule_forced_change

* grandpa: don't take authority weight in schedule_forced_change

* grandpa: add const for default forced change delay

* grandpa: adjust weights after benchmark on ref hardware

* grandpa: fix cleanup of forced changes on standard change application

* grandpa: replace schedule_forced_change with note_stalled

* grandpa: always trigger a session change when the set is stalled

* grandpa: fix bug on set id mutation after failed scheduled change

* grandpa: take delay as parameter in note_stalled

* grandpa: fix tests

* grandpa: fix cleanup of forced changes

* grandpa: add test for forced changes cleanup

* grandpa: add test for session rotation set id

* grandpa: add test for scheduling of forced changes on new session

* Name all the tasks! (#6726)

* Remove any implementation of `Spawn` or `Executor` from our task executors

* Fix compilation

* Rename `SpawnBlockingExecutor`

* Update primitives/core/src/traits.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fix tests

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add the Substrate Service Tasks dashboard (#6665)

* seal: Fail instantiate if new contract is below subsistence threshold (#6719)

* seal: Fail instantiate if new contract is below subsistence threshold

We need each contract that exists to be above the subsistence threshold
in order to keep up the guarantuee that we always leave a tombstone behind
with the exception of a contract that called `ext_terminate`.

* Fixup executor test

* Bump runtime

* Remove Unpin requirement for Slots (#6711)

* pallet-evm: add support for tuple-based precompile declarations (#6681)

* pallet-evm: add support for tuple-based precompile declarations

* Add missing license header

* Switch to use impl_for_tuples

* Remove unnecessary impl for ()

* prometheus: don't use protobuf feature (#6744)

* Use node_template_runtime::opaque::Block instead of node_template_runtime::Block (#6737)

* Various small improvements to service construction. (#6738)

* Remove service components and add build_network, build_offchain_workers etc

* Improve transaction pool api

* Remove commented out line

* Add PartialComponents

* Add BuildNetworkParams, documentation

* Remove unused imports in tests

* Apply suggestions from code review

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Remove unused imports in node-bench

Co-authored-by: Nikolay Volf <nikvolf@gmail.com>

* Remove unused node template deps (#6748)

* Remove unused node template deps

Backport changes made by @c410-f3r
https://github.com/substrate-developer-hub/substrate-node-template/pull/66

* Enhancements to README

* Revert change to serde per @thiolliere

* Replace Process.toml with json (#6740)

* Replace Process.toml with json

* Trigger checks

* Revert "Trigger checks"

This reverts commit 9bdf9f135cecb92ca3859dfa211d396a48dd6a8d.

* Trigger checks

* Revert "Trigger checks"

This reverts commit b0c6f29d6aefaf7ca8b137c7d2f958a5e0929d9e.

* Update parity-scale-codec to prepare for breaking rustc release (#6746)

This updates parity-scale-codec{-derive} to prepare for a rustc release
that would otherwise break the derive implementation:
https://github.com/rust-lang/rust/pull/73084

* seal: Add benchmarks for dispatchables (#6715)

* seal: Fix syntax that confuses rust-analyzer

* seal: Add benchmarks for Dispatchables

These are only the benchmarks for the dispatchables of
the pallet. Those are not listed in the Schedule because
we do not want to pull the Schedule from storage before
dispatching.

This OK because those costs are not related to actual contract
execution. Those costs (instruction costs, ext_* costs) will
be benchmarked seperatly and entered into the default Schedule.

* seal: Add a maximum code size

* Fix comments from review

* Removed SEED constant

* client/network: Adjust wording (#6755)

Rename `NetworkWorker::from_worker` to `NetworkWorker::from_service` as
it is a channel from the `NetworkService` to the `NetworkWorker`.

* Add a back-pressure-friendly alternative to NetworkService::write_notifications 🎉 (#6692)

* Add NetworkService::send_notifications

* Doc

* Doc

* API adjustment

* Address concerns

* Make it compile

* Start implementation

* Progress in the implementation

* Change implementation strategy again

* More work before weekend

* Finish changes

* Minor doc fix

* Revert some minor changes

* Apply suggestions from code review

* GroupError -> NotifsHandlerError

* Apply suggestions from code review

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* state_transition_waker -> close_waker

* Apply suggestions from code review

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Finish renames in service.rs

* More renames

* More review suggestsions applied

* More review addressing

* Final change

* 512 -> 2048

Co-authored-by: Roman Borschel <romanb@users.noreply.github.com>

* Cleanup our sort usage (#6754)

* Allow `PostDispatchInfo` to disable fees (#6749)

* initial mock

* add test

* remove unneeded clone

* Update frame/support/src/weights.rs

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* fix compile

* Update frame/support/src/weights.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update frame/sudo/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* benchmarks! macro: factorize instance usage. (#6750)

* factorize benchmark!

* fix types

* fix types

* client/network: Add peers to DHT only if protocols match (#6549)

* client/network/src/discovery: Adjust to Kademlia  API changes

* client/network: Add peers to DHT only if protocols match

With https://github.com/libp2p/rust-libp2p/pull/1628 rust-libp2p allows
manually controlling which peers are inserted into the routing table.
Instead of adding each peer to the routing table automatically, insert
them only if they support the local nodes protocol id (e.g. `dot`)
retrieved via the `identify` behaviour.

For now this works around
https://github.com/libp2p/rust-libp2p/issues/1611. In the future one
might add more requirements. For example one might try to exclude
light-clients.

* Cargo.toml: Remove crates.io patch for libp2p

* client/network/src/behaviour: Adjust to PeerInfo name change

* client/network/src/discovery: Rework Kademlia event matching

* client/network/discovery: Add trace on adding peer to DHT

* client/network/discovery: Retrieve protocol name from kad behaviour

* client/network/discovery: Fix formatting

* client/network: Change DiscoveryBehaviour::add_self_reported signature

* client/network: Document manual insertion strategy

* client/network/discovery: Remove TODO for ignoring DHT address

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Add `memory-tracker` feature to `sp-trie` to fix wasm panic (#6745)

* Add memory tracker feature to sp-trie to fix wasm panic

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* support custom ss58addressformat in from_ss58check_with_version (#5526)

* support custom ss58addressformat in from_ss58check_with_version

* fix str parse

1. if can parse with u8, use u8 into.
2. if u8 can't parse, convert to str then parse

* add a test

* typo

* add error description in test

* fix the `TryFrom<u8>` for `Ss58AddressFormat`

change check logic in TryFrom<u8> to replace modified code in `from_ss58check_with_version`

* use Ss58AddressFormat::default() replace DEFAULT_VERSION

* Apply suggestions from code review

* Update primitives/core/src/crypto.rs

* Update primitives/core/src/crypto.rs

* Update primitives/core/src/crypto.rs

* Update primitives/core/src/crypto.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add a `DefaultQueue` type alias to remove the need to use `sp_api::TransactionFor` (#6761)

* Add DefaultQueue

* Add DefaultImportQueue to the top level of sp-consensus

* Ignore flaky test (#6767)

* Add "✅ Successfully mined block" log message (#6764)

* pallet-evm: add builtin support for the four basic Ethereum precompiles (#6743)

* pallet-evm: add builtin support for the four basic Ethereum precompiles

* linear_cost -> ensure_linear_cost to directly return OutOfGas error

* Rename task name to stick to the default naming scheme (#6768)

* BABE slot and epoch event notifications (#6563)

* BabeWorker -> BabeSlotWorker

* SlotWorker::notify_slot: similar to claim_slot, but called no matter authoring

* Wrap the future with a new struct BabeWorker

* Add type definition slot_notification_sinks

* Function slot_notification_streams for the receiver side

* Get a handle of slot_notification_sinks in BabeSlotWorker

* Implement notify_slot

* Switch to use bounded mpsc

* Do not drop the sink when channel is full

Only skip sending the message and emit a warning, because it is recoverable.

* Fix future type bounds

* Add must_use and sink type alias

* Update Balances Pallet to use `WeightInfo` (#6610)

* Update balance benchmarks

* Update weight functions

* Remove user component

* make componentless

* Add support for `#[extra]` tag on benchmarks

* Update balances completely

* Apply suggestions from code review

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Fix some tests

* Maybe fix to test. Need approval from @tomusdrw this is okay

* Make test better

* keep weights conservative

* Update macro for merge master

* Add headers

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Fix graceful shutdown skipped if future ends with error (#6769)

* Initial commit

Forked at: 5c34fe49f8fccb1f0e0f983bb7f8cbb8850b574b
Parent branch: origin/master

* Fix graceful shutdown skipped if future ends with error

* apply suggestion

* Fix link (#6775)

* Allow blacklisting blocks from being finalized again after block revert (#6301)

* Allow blacklisting blocks from being finalized again after block revert

* Use BlockRules for storing unfinalized and add have_state_at in revert

* Move finalization_check in finalize_block upward

* Directly mark finalization blacklist as badblocks

* Remove obselete comment

* Order delta before calculating the storage root (#6780)

We need to order the delta before calculating the storage root, because
the order is important if the storage root is calculated using a storage
proof. The problem is arises when the delta is different than at the
time the storage root was recorded, because we may require a different
node that is not part of the proof and so, the storage root can not be
calculated. The problem is solved by always order the delta to use the
same order when calculating the storage root while recording the
stroage proof and when calculating the storage root using
the storage proof.

To prevent this bug in future again, a regression test is added.

Fixes: https://github.com/paritytech/cumulus/issues/146

* Don't close inbound notifications substreams immediately (#6781)

* Don't close inbound notifications substreams immediately

* Fix not closing in return to node A closing

* client/network: Expose DHT query duration to Prometheus (#6784)

Expose duration of DHT put and get request as a Prometheus histogram.

* Fix transaction payment runtime api (#6792)

The transaction payment runtime api used its own extrinsic generic
parameter. This is wrong, because this resulted in using always the
native extrinsic. If there was a runtime upgrade that changed the
extrinsic in some way, it would result in the api breaking. The correct
way is to use the `Extrinsic` from the `Block` parameter. This is on the
node side the opaque extrinsic and on the runtime side the real extrinsic.

* Bump elliptic from 6.5.2 to 6.5.3 in /.maintain/chaostest (#6791)

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.2 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](https://github.com/indutny/elliptic/compare/v6.5.2...v6.5.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* client/network: Fix wrong metric help text (#6794)

The `sub_libp2p_kademlia_query_duration` metric only has the dimension
`type` not `protocol`.

* seal: Fix and improve error reporting (#6773)

* seal: Rework ext_transfer, ext_instantiate, ext_call error handling

* Deny calling plain accounts (must use transfer now)
* Return proper module error rather than ad-hoc strings
* Return the correct error codes from call,instantiate (documentation was wrong)
* Make ext_transfer fallible again to make it consistent with ext_call

* seal: Improve error messages on memory access failures

* seal: Convert contract trapped to module error

* seal: Add additional tests for transfer, call, instantiate

These tests verify that those functions return the error types
which are declared in its docs.

* Make it more pronounced that to_execution_result handles trap_reason

* Improve ReturnCode docs

* Fix whitespace issues in wat files

* Improve ReturnCode doc

* Improve ErrorOrigin doc and variant naming

* Improve docs on ExecResult and ExecError

* Encode u32 sentinel value as hex

* with_nested_context no longer accepts an Option for trie

* Fix successful typo

* Rename InvalidContractCalled to NotCallable

* Improve Benchmark Writer: Remove Unused Components, Remove Multiply by Zero, Files Split by Pallet (#6785)

* initial improvements

* better file management, ignore unused components

* Output warning when components unused

* update comment

* Write even when base weight is zero

* remove unwrap where possible

* Dont sort components to dedup

* undo delete

* improve clarity of unused components

* remove unused dep

* Update Process.json

* Add integrity test for slash defer duration (#6782)

* Add integrity test for slash defer duration

* Wrap in externalities

* Update frame/staking/src/lib.rs

* Convert spaces to tabs (#6799)

* Add details to legacy requests (#6747)

* Add ss58 address for Subsocial (#6800)

* Add mutate_exists to StorageDoubleMap. (#6704)

* pallet-democracy use of weightinfo (#6783)

* democracy use of weightinfo

* fix some doc and benchs

* todo generate from parity machine

* factorize and add license

* use final weights

* add slightly more sensible default weight

* refactor

* rename benchmark to avoid confusion

* just make remove_other_vote benchmark being the worst case of the extrinsic

* Use DNS hostnames for flaming fir bootnodes (#6807)

* Use dns hostnames for flaming fir bootnodes

* Remove newline

* Fix warning being printed by authority-discovery (#6820)

* Better default balances weights (#6813)

* Remove generation of instance trait by decl_storage. (#6812)

* remove generation of instance trait, no breaking change

* doc

* doc

* Update frame/support/src/traits.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update frame/support/procedural/src/storage/instance_trait.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Successful `note_imminent_preimage` is free (#6793)

* Successful `note_imminent_preimage` is free

* update docs

* Add test for duplicate preimage

* service: remove collection of system/process metrics (#6822)

* Fix legacy substream fallback not working (#6826)

* Fix legacy substream fallback not working

* Make it nicer

* grandpa: never overwrite current rounds voter state (#6823)

* grandpa: never overwrite current rounds voter state

* grandpa: add test for voter state overwrite

* Add RPC Builder to Substrate Node Template (#6808)

* Pulled RPC from node and populated the node-template's RPC builder with one example implementation

* surpress build errror

* dead_code

* Fixed module usage, removed copyright, removed rpc builder for light client + some comments

* added a comment for rpc extension

* Update bin/node-template/node/src/rpc.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update rpc.rs

* fix spacing

* more space to tabs

* more space to tabs

* Documenation nitpick

* Documentation nitpick

* Documentation nitpick

* Documentation nitpick

* Documentation nitpick

* pre-format

* Updated transaction payment API implemented for node template

* fix space and commented code

* fix long line

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Dan Forbes <dan@danforbes.dev>

* Delay network startup to after complete initialization (#6833)

* Delay network startup to after complete initialization

* Update client/service/src/builder.rs

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

* De-alias frame_system in node template runtime (#6829)

* de-alias frame_system in node template

* Fix line length

* Fix chainspec

* Allow task manager to have children (#6771)

* Initial commit

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* WIP

Forked at: 19c1d9028d8d6eabef41693433b56e14da025247
Parent branch: origin/master

* changelog

* Remove Box

* Make future nicer

* Revert "Make future nicer"

This reverts commit 49fb8fb6f245c3ca2c384468df14b34f34616736.

* Simplify

* Additional check

* Simplify more

Co-authored-by: Bastian Köcher <git@kchr.de>

* De-alias pallets in node template runtime (#6836)

* dealias pallets

* Restore accidentally deleted code blocks

* grandpa: fix enacting forced changes with no delay (#6828)

* grandpa: fix enacting forced changes with no delay

* grandpa: fix formatting

* Renamed add_children to add_child since it adds only one child (#6838)

* Revalidate transactions only on latest best block (#6824)

* Revalidate transactions only on latest best block

We should revalidate transactions only on the latest best block and not
on any arbitrary block. The revalidation before failed when there were
multiple blocks on the height given to the revalidation function, but no
block was imported as best block.

* Update test-utils/runtime/transaction-pool/src/lib.rs

Co-authored-by: Jaco Greeff <jacogr@gmail.com>

* Fix tests

* Only process best blocks in the transaction pool

Co-authored-by: Jaco Greeff <jacogr@gmail.com>

* Fix `wait_for_blocks` counting blocks that didn't go through consensus (#6850)

* Initial commit

Forked at: 74804b5649eccfb83c90aec87bdca58e5d5c8789
Parent branch: origin/master

* WIP

Forked at: 74804b5649eccfb83c90aec87bdca58e5d5c8789
Parent branch: origin/master

* pallet-evm: move gas price check to execute_evm (#6837)

* Update pinned wasm-bindgen dependency (#6861)

* Update dependency.

* Update readme docs

* update python webserver command

* Add Subscription RPC for Grandpa Finality (#5732)

* Rough skeleton for what I think the RPC should look like

* Create channel for sending justifications

Sends finalized header and justification from Grandpa to the
client. This lays the groundwork for hooking into the RPC module.

* WIP: Add subscribers for justifications to Grandpa

Adds the Sender end of a channel into Grandpa, through which notifications
about block finality events can be sent.

* WIP: Add a struct for managing subscriptions

Slightly different approach from the last commit, but same
basic idea. Still a rough sketch, very much doesn't compile yet.

* Make naming more clear and lock data in Arc

* Rough idea of what RPC would look like

* Remove code from previous approach

* Missed some things

* Update client/rpc-api/src/chain/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Update client/rpc-api/src/chain/mod.rs

Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

* Split justification subscription into sender and receiver halves

* Replace RwLock with a Mutex

* Add sample usage from the Service's point of view

* Remove code that referred to "chain_" RPC

* Use the Justification sender/receivers from Grandpa LinkHalf

* Add some PubSub boilerplate

* Add guiding comments

* TMP: comment out to fix compilation

* Return MetaIoHandler from PubSubHandler in create_full

* Uncomment pubsub methods in rpc handler (fails to build)

* node/rpc: make Metadata concrete in create_full to fix compilation

* node: pass in SubscriptionManger to grandpa rpc handler

* grandpa-rpc: use SubscriptionManger to add subscriber

* grandpa-rpc: attempt at setting up the justification stream (fails to build)

* grandpa-rpc: fix compilation of connecting stream to sink

* grandpa-rpc: implement unsubscribe

* grandpa-rpc: update older tests

* grandpa-rpc: add full prefix to avoid confusing rust-analyzer

* grandpa-rpc: add test for pubsub not available

* grandpa-rpc: tidy up leftover code

* grandpa-rpc: add test for sub and unsub of justifications

* grandpa-rpc: minor stylistic changes

* grandpa-rpc: split unit test

* grandpa-rpc: minor stylistic changes in test

* grandpa-rpc: skip returning future when cancelling

* grandpa-rpc: reuse testing executor from sc-rpc

* grandpa-rpc: don't need to use PubSubHandler in tests

* node-rpc: use MetaIoHandler rather than PubSubHandler

* grandpa: log if getting header failed

* grandpa: move justification channel creation into factory function

* grandpa: make the justification sender optional

* grandpa: fix compilation warnings

* grandpa: move justification notification types to new file

* grandpa-rpc: move JustificationNotification to grandpa-rpc

* grandpa-rpc: move JustificationNotification to its own file

* grandpa: rename justification channel pairs

* grandpa: rename notifier types

* grandpa: pass justification as GrandpaJustification to…
@Boscop
Copy link

Boscop commented Jan 7, 2021

I have this in my Cargo.toml:

zip = "0.5"

https://crates.io/crates/zip

cargo update downgraded it:

> cargo update
    Updating actix-web v3.3.1 -> v3.3.2
    Updating addr2line v0.13.0 -> v0.14.1
    Updating ahash v0.4.6 -> v0.4.7
    Updating aho-corasick v0.7.14 -> v0.7.15
    Updating alsa-sys v0.3.0 -> v0.3.1
    Updating andrew v0.3.0 -> v0.3.1
    Removing arc-swap v0.4.7
    Updating arithmetic-eval v0.2.0-beta.1 -> v0.2.0
    Updating arithmetic-parser v0.2.0-beta.1 -> v0.2.0
    Updating async-executor v1.3.0 -> v1.4.0
    Updating async-global-executor v1.4.2 -> v1.4.3
    Updating async-io v1.1.10 -> v1.3.1
    Updating async-std v1.6.5 -> v1.8.0
    Updating auto_enums v0.7.8 -> v0.7.12
    Updating auto_enums_core v0.7.8 -> v0.7.12
    Updating auto_enums_derive v0.7.8 -> v0.7.12
    Updating backtrace v0.3.53 -> v0.3.55
    Updating base-x v0.2.6 -> v0.2.8
    Removing base64 v0.10.1
    Removing base64 v0.11.0
    Updating blake2b_simd v0.5.10 -> v0.5.11
    Updating bytecount v0.6.0 -> v0.6.2
    Updating cc v1.0.65 -> v1.0.66
    Removing cloudabi v0.1.0
    Updating cmake v0.1.44 -> v0.1.45
    Updating combine v4.4.0 -> v4.5.2
    Updating const_fn v0.4.2 -> v0.4.5
    Removing cookie v0.12.0
    Removing cookie v0.14.2
      Adding cookie v0.14.3
    Updating core-graphics v0.22.1 -> v0.22.2
    Updating coreaudio-sys v0.2.5 -> v0.2.6
    Updating crc32fast v1.2.0 -> v1.2.1
    Updating derivative v2.1.1 -> v2.1.3
    Updating derive_utils v0.10.0 -> v0.11.2
    Updating dtoa v0.4.6 -> v0.4.7
    Updating dtoa-short v0.3.2 -> v0.3.3
    Updating encoding_rs v0.8.24 -> v0.8.26
    Updating fetch_unroll v0.2.1 -> v0.2.2
    Updating filetime v0.2.12 -> v0.2.13
    Updating flate2 v1.0.14 -> v1.0.19
    Updating futures v0.3.6 -> v0.3.8
    Updating futures-channel v0.3.6 -> v0.3.8
    Updating futures-core v0.3.6 -> v0.3.8
    Updating futures-executor v0.3.6 -> v0.3.8
    Updating futures-io v0.3.6 -> v0.3.8
    Updating futures-lite v1.11.2 -> v1.11.3
    Updating futures-sink v0.3.6 -> v0.3.8
    Updating futures-task v0.3.6 -> v0.3.8
    Updating futures-util v0.3.6 -> v0.3.8
    Removing getrandom v0.1.15
    Removing getrandom v0.2.0
      Adding getrandom v0.1.16
      Adding getrandom v0.2.1
    Updating gimli v0.22.0 -> v0.23.0
    Updating h2 v0.2.6 -> v0.2.7
    Updating heck v0.3.1 -> v0.3.2
    Updating http v0.2.1 -> v0.2.2
    Updating hyper v0.13.8 -> v0.13.9
    Removing idna v0.1.5
    Updating image v0.23.10 -> v0.23.12
    Updating indexmap v1.6.0 -> v1.6.1
    Updating inotify-sys v0.1.3 -> v0.1.4
    Updating instant v0.1.7 -> v0.1.9
    Updating itoa v0.4.6 -> v0.4.7
    Updating js-sys v0.3.45 -> v0.3.46
    Updating libc v0.2.79 -> v0.2.82
    Updating libflate v0.1.27 -> v1.0.3
      Adding libflate_lz77 v1.0.0
    Updating libloading v0.6.4 -> v0.6.6
    Removing line_drawing v0.8.0
    Updating lock_api v0.4.1 -> v0.4.2
    Removing lzw v0.10.0
    Updating memchr v2.3.3 -> v2.3.4
    Removing memmap v0.7.0
      Adding memmap2 v0.1.0
    Updating minimp3 v0.5.0 -> v0.5.1
    Updating mio v0.6.22 -> v0.6.23
    Updating miow v0.2.1 -> v0.2.2
    Updating native-tls v0.2.4 -> v0.2.7
    Updating net2 v0.2.35 -> v0.2.37
    Updating nom_locate v2.0.0 -> v3.0.0
    Removing num-complex v0.3.0
    Updating num-integer v0.1.43 -> v0.1.44
    Updating num-iter v0.1.41 -> v0.1.42
    Updating num-rational v0.3.0 -> v0.3.2
    Updating object v0.21.1 -> v0.22.0
    Updating openssl v0.10.30 -> v0.10.32
    Updating openssl-sys v0.9.58 -> v0.9.60
    Updating ordered-float v2.0.0 (https://github.com/reem/rust-ordered-float#e8fe06a5) -> #923c571c
    Updating parking_lot v0.11.0 -> v0.11.1
    Updating parking_lot_core v0.8.0 -> v0.8.2
    Removing percent-encoding v1.0.1
    Updating png v0.16.7 -> v0.16.8
      Adding podio v0.1.7
    Updating ppv-lite86 v0.2.9 -> v0.2.10
    Updating quote v1.0.7 -> v1.0.8
    Updating reamap_config v0.1.0 (ssh://git@github.com/Boscop/reamap#6e70a920) -> #897fd60a
    Updating regex v1.4.1 -> v1.4.2
    Updating regex-syntax v0.6.20 -> v0.6.21
    Updating reqwest v0.10.8 -> v0.10.10
    Updating retain_mut v0.1.1 -> v0.1.2
    Updating rimd v0.0.2 (https://github.com/RustAudio/rimd#54fd9bd2) -> #7ac15853
    Updating ring v0.16.18 -> v0.16.19
    Updating rust-argon2 v0.8.2 -> v0.8.3
    Updating rustc-demangle v0.1.17 -> v0.1.18
    Updating rustls v0.16.0 -> v0.19.0
    Updating security-framework v0.4.4 -> v2.0.0
    Updating security-framework-sys v0.4.3 -> v2.0.0
    Updating serde-big-array v0.3.0 -> v0.3.1
    Updating serde_json v1.0.59 -> v1.0.61
    Removing serde_urlencoded v0.6.1
    Updating signal-hook-registry v1.2.1 -> v1.3.0
    Updating smallvec v1.4.2 -> v1.6.0
    Updating smithay-client-toolkit v0.12.0 -> v0.12.2
    Updating standback v0.2.11 -> v0.2.14
    Updating string_cache v0.8.0 -> v0.8.1
    Updating syn v1.0.45 -> v1.0.58
    Removing take_mut v0.2.2
    Updating termcolor v1.1.0 -> v1.1.2
    Updating thiserror v1.0.21 -> v1.0.23
    Updating thiserror-impl v1.0.21 -> v1.0.23
    Updating tiff v0.5.0 -> v0.6.1
    Updating time v0.2.22 -> v0.2.23
    Removing tinyvec v0.3.4
    Updating tokio v0.2.22 -> v0.2.24
    Updating toml v0.5.7 -> v0.5.8
    Updating tracing v0.1.21 -> v0.1.22
    Updating unicode-normalization v0.1.13 -> v0.1.16
    Updating unicode-segmentation v1.6.0 -> v1.7.1
    Updating ureq v0.11.4 -> v1.5.4
    Removing url v1.7.2
    Updating vcpkg v0.2.10 -> v0.2.11
    Updating vst v0.2.0 -> v0.2.1
    Updating wasm-bindgen v0.2.68 -> v0.2.69
    Updating wasm-bindgen-backend v0.2.68 -> v0.2.69
    Updating wasm-bindgen-futures v0.4.18 -> v0.4.19
    Updating wasm-bindgen-macro v0.2.68 -> v0.2.69
    Updating wasm-bindgen-macro-support v0.2.68 -> v0.2.69
    Updating wasm-bindgen-shared v0.2.68 -> v0.2.69
    Updating wayland-client v0.28.1 -> v0.28.3
    Updating wayland-commons v0.28.1 -> v0.28.3
    Updating wayland-cursor v0.28.1 -> v0.28.3
    Updating wayland-egl v0.28.1 -> v0.28.3
    Updating wayland-protocols v0.28.1 -> v0.28.3
    Updating wayland-scanner v0.28.1 -> v0.28.3
    Updating wayland-sys v0.28.1 -> v0.28.3
    Updating web-sys v0.3.45 -> v0.3.46
    Updating webpki v0.21.3 -> v0.21.4
    Updating webpki-roots v0.18.0 -> v0.21.0
    Updating weezl v0.1.1 -> v0.1.3
    Updating xcursor v0.3.2 -> v0.3.3
    Updating zip v0.5.8 -> v0.5.6

Updating zip v0.5.8 -> v0.5.6

before I had:

[[package]]
name = "zip"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "543adf038106b64cfca4711c82c917d785e3540e04f7996554488f988ec43124"
dependencies = [
 "byteorder",
 "bzip2",
 "crc32fast",
 "flate2 1.0.14",
 "thiserror",
 "time 0.1.44",
]

Why did it choose to downgrade zip?

Calling cargo update -p zip after this didn't cause cargo to update it!

After I wrote

zip = "0.5.9"

in my Cargo.toml file, it updated it correctly, so there was no reason to not choose this version in the first place!

Btw, why doesn't it display downgrades in a more visible way so that I can manually fix them?
(E.g. by printing "Downgrading" instead of "Updating")

I just noticed this accidentally because it was the last line in the very long cargo update output.

@Eh2406
Copy link
Contributor

Eh2406 commented Jan 7, 2021

@Boscop do you have a way for me to reproduce to try and figure out what specifically happened in the resolver?
And a big +1 on a more obvious output for downgrades.

@Boscop
Copy link

Boscop commented Jan 8, 2021

@Eh2406 Would it be sufficient if I upload the Cargo.lock file how it was before cargo update?

@Eh2406
Copy link
Contributor

Eh2406 commented Jan 9, 2021

I think the dependencies part of Cargo.toml and Cargo.lock will probably be sufficient. If you gave me them, then I would start a new project with the files in it and try a cargo update. If that gives us the line Updating zip v0.5.8 -> v0.5.6, then I can try to dig in.

rnestler added a commit to rnestler/reboot-arch-btw that referenced this issue Oct 12, 2021
This was done using `cargo update zbus --precise=1.9.2`, since `cargo
update` did only update to 1.9.1.

Also note that now runing `cargo update` will *downgrade* zbus again to 1.9.1:

    % cargo update
	Updating crates.io index
	Updating bitflags v1.2.1 -> v1.3.2
	  Adding cfg-if v0.1.10
	Removing memoffset v0.6.4
	Updating nix v0.20.2 -> v0.17.0
	  Adding void v1.0.2
	Updating zbus v1.9.2 -> v1.9.1
	Updating zbus_macros v1.9.2 -> v1.9.1

See also the following cargo issues:
 * rust-lang/cargo#7671
 * rust-lang/cargo#5702
rnestler added a commit to rnestler/reboot-arch-btw that referenced this issue Oct 12, 2021
This was done using `cargo update zbus --precise=1.9.2`, since `cargo
update` did only update to 1.9.1.

Also note that now runing `cargo update` will *downgrade* zbus again to 1.9.1:

    % cargo update
	Updating crates.io index
	Updating bitflags v1.2.1 -> v1.3.2
	  Adding cfg-if v0.1.10
	Removing memoffset v0.6.4
	Updating nix v0.20.2 -> v0.17.0
	  Adding void v1.0.2
	Updating zbus v1.9.2 -> v1.9.1
	Updating zbus_macros v1.9.2 -> v1.9.1

See also the following cargo issues:
 * rust-lang/cargo#7671
 * rust-lang/cargo#5702
@djc
Copy link
Contributor

djc commented Sep 12, 2022

Here's another case that I think might be related:

djc-2021 main test-rs $ cargo init foo
     Created binary (application) package
djc-2021 main test-rs $ cargo add chrono@0.4
    Updating crates.io index
      Adding chrono v0.4 to dependencies. [this is misleading, see https://github.com/rust-lang/cargo/issues/11073]
             Features:
             - rustc-serialize
             - serde
djc-2021 main test-rs $ rg -C2 chrono Cargo.lock
31-
32-[[package]]
33:name = "chrono"
34-version = "0.4.22"
35-source = "registry+https://github.com/rust-lang/crates.io-index"
--
147-version = "0.1.0"
148-dependencies = [
149: "chrono",
150-]
151-
djc-2021 main test-rs $ cargo add whoami        
    Updating crates.io index
      Adding whoami v1.2.2 to dependencies.
djc-2021 main test-rs $ rg -C2 chrono Cargo.lock 
22-
23-[[package]]
24:name = "chrono"
25-version = "0.4.20"
26-source = "registry+https://github.com/rust-lang/crates.io-index"
--
117-version = "0.1.0"
118-dependencies = [
119: "chrono",
120- "whoami",
121-]

So Cargo silently downgraded my chrono dependency here. If I bump the dependency to 0.4.22 in Cargo.toml to force the issue, it explains itself:

error: failed to select a version for `wasm-bindgen`.
    ... required by package `whoami v1.2.2`
    ... which satisfies dependency `whoami = "^1.2.2"` of package `test-rs v0.1.0 (/Users/djc/src/test-rs)`
versions that meet the requirements `>=0.2, <=0.2.78` are: 0.2.78, 0.2.77, 0.2.76, 0.2.75, 0.2.74, 0.2.73, 0.2.72, 0.2.71, 0.2.70, 0.2.69, 0.2.68, 0.2.67, 0.2.66, 0.2.65, 0.2.64, 0.2.63, 0.2.62, 0.2.61, 0.2.60, 0.2.59, 0.2.58, 0.2.57, 0.2.56, 0.2.55, 0.2.54, 0.2.53, 0.2.52, 0.2.51, 0.2.50, 0.2.49, 0.2.48, 0.2.47, 0.2.46, 0.2.45, 0.2.44, 0.2.43, 0.2.42, 0.2.41, 0.2.40, 0.2.39, 0.2.38, 0.2.37, 0.2.36, 0.2.35, 0.2.34, 0.2.33, 0.2.32, 0.2.31, 0.2.30, 0.2.29, 0.2.28, 0.2.27, 0.2.26, 0.2.25, 0.2.24, 0.2.23, 0.2.22, 0.2.21, 0.2.20, 0.2.19, 0.2.18, 0.2.17, 0.2.16, 0.2.15, 0.2.14, 0.2.13, 0.2.12, 0.2.11, 0.2.10, 0.2.9, 0.2.8, 0.2.7, 0.2.6, 0.2.5, 0.2.4, 0.2.3, 0.2.2, 0.2.1, 0.2.0

the package `wasm-bindgen` links to the native library `wasm_bindgen`, but it conflicts with a previous package which links to `wasm_bindgen` as well:
package `wasm-bindgen-shared v0.2.81`
    ... which satisfies dependency `wasm-bindgen-shared = "=0.2.81"` of package `wasm-bindgen-backend v0.2.81`
    ... which satisfies dependency `wasm-bindgen-backend = "=0.2.81"` of package `wasm-bindgen-macro-support v0.2.81`
    ... which satisfies dependency `wasm-bindgen-macro-support = "=0.2.81"` of package `wasm-bindgen-macro v0.2.81`
    ... which satisfies dependency `wasm-bindgen-macro = "=0.2.81"` of package `wasm-bindgen v0.2.81`
    ... which satisfies dependency `wasm-bindgen = "^0.2.81"` of package `iana-time-zone v0.1.45`
    ... which satisfies dependency `iana-time-zone = "^0.1.44"` of package `chrono v0.4.22`
    ... which satisfies dependency `chrono = "^0.4.22"` of package `test-rs v0.1.0 (/Users/djc/src/test-rs)`
Only one package in the dependency graph may specify the same links value. This helps ensure that only one copy of a native library is linked in the final binary. Try to adjust your dependencies so that only one package uses the links ='wasm-bindgen' value. For more information, see https://doc.rust-lang.org/cargo/reference/resolver.html#links.

all possible versions conflict with previously selected packages.

  previously selected package `wasm-bindgen v0.2.81`
    ... which satisfies dependency `wasm-bindgen = "^0.2.81"` of package `iana-time-zone v0.1.45`
    ... which satisfies dependency `iana-time-zone = "^0.1.44"` of package `chrono v0.4.22`
    ... which satisfies dependency `chrono = "^0.4.22"` of package `test-rs v0.1.0 (/Users/djc/src/test-rs)`

failed to select a version for `wasm-bindgen` which could resolve this conflict

bors added a commit that referenced this issue Mar 14, 2023
Accurately show status when downgrading dependencies

### What does this PR try to resolve?

A few times now I've ran into issues where Cargo ends up downgrading a dependency in order to satisfy a pinned dependency somewhere in the dependency graph. Unfortunately this is not clear in the output of `cargo update`, which just shows all the changed dependencies as `Updating`.

References:

* #5702
* [Finding a pinned dependency edge](https://users.rust-lang.org/t/finding-a-pinned-dependency-edge/81157)
* tokio-rs/axum#1814

### How should we test and review this PR?

This is a small change that tries to make dependency downgrades stand out more in the output of `cargo update`. I have not added any new tests since the existing tests seem to cover this functionality.

Some tests still fail, these refer to Git dependencies. I'm honestly not sure how to handle these, so I'd like to get some feedback on this before I fix those tests up. Git commits form a DAG, so one option is to see if the new commit is an ancestor of the old one (mark as "updating"), if the old commit is an ancestor of the new one (mark as "downgrading"), or neither (could be from parallel branches) we could compare based on timestamp in that case.

### To do

* Fix up tests for Git dependency updates
bors added a commit that referenced this issue Mar 14, 2023
Accurately show status when downgrading dependencies

### What does this PR try to resolve?

A few times now I've ran into issues where Cargo ends up downgrading a dependency in order to satisfy a pinned dependency somewhere in the dependency graph. Unfortunately this is not clear in the output of `cargo update`, which just shows all the changed dependencies as `Updating`.

References:

* #5702
* [Finding a pinned dependency edge](https://users.rust-lang.org/t/finding-a-pinned-dependency-edge/81157)
* tokio-rs/axum#1814

### How should we test and review this PR?

This is a small change that tries to make dependency downgrades stand out more in the output of `cargo update`. I have not added any new tests since the existing tests seem to cover this functionality.

Some tests still fail, these refer to Git dependencies. I'm honestly not sure how to handle these, so I'd like to get some feedback on this before I fix those tests up. Git commits form a DAG, so one option is to see if the new commit is an ancestor of the old one (mark as "updating"), if the old commit is an ancestor of the new one (mark as "downgrading"), or neither (could be from parallel branches) we could compare based on timestamp in that case.

### To do

* Fix up tests for Git dependency updates
@epage epage added A-dependency-resolution Area: dependency resolution and the resolver S-triage Status: This issue is waiting on initial triage. labels Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-dependency-resolution Area: dependency resolution and the resolver Command-update S-triage Status: This issue is waiting on initial triage.
Projects
None yet
Development

Successfully merging a pull request may close this issue.