-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
artifact deps should work when target field specified coexists with optional = true
#11183
Conversation
r? @epage (rust-highfive has picked a reviewer for you, use r? to override) |
If my understanding of this is correct, the removal of If so, I'm slightly uncomfortable with relying on that. The idea was that in the future there would not be an empty set used for things that don't have any features set. Then, the call to However, there isn't really a strong need for that (I just kind wanted to get rid of the |
Thank you for the review! Of course I will update that comment! And maybe explain more on the purpose of Just curious, besides removing |
Previously, `is_dep_activated` depends on `activated_dependencies`, which is a map of `PackageFeaturesKey` to its own activated `DepFeature`s. `PackageFeaturesKey` in feature resolver is always comprised of * A `PackageId` of a given dependency, and * A enum value that helps decoupling activated features for that dependency. Currently depending on the type of given dependency. Looking into issue 10526, it has an `activated_dependencies` of ``` { (mycrate, NormalOrDevOrArtifactTarget(None)) -> [dep:mybindep] } ``` However, this [line][1] accidentally use a parent's `pkgid` and a dependency's `FeaturesFor` to compose a key: ``` (mycrate, NormalOrDevOrArtifactTarget("x86_64-unknown-linux-gnu")) ``` That is never a valid key to query features for artifacts dependency. A valid key should be comprised of components both from the parent: ``` (mycrate, NormalOrDevOrArtifactTarget(None)) ``` Or both from the dependency itself: ``` (mybindep, NormalOrDevOrArtifactTarget("x86_64-unknown-linux-gnu")) ``` As aforementioned `activated_dependencies` only stores parent packages and their activated features. Those informations are included in `activated_features` as well, so this commit goes with the route that removes `activated_dependencies` and uses only dependency's infomation to query if itself is activated. [1]: https://github.com/rust-lang/cargo/blob/0b84a35c2c7d70df4875a03eb19084b0e7a543ef/src/cargo/core/compiler/unit_dependencies.rs#L1097-L1098
f78dbd9
to
34cdbe9
Compare
Rebased and added comments per discussion. |
src/cargo/core/resolver/features.rs
Outdated
// The presence of an (empty) set also means that the dependency is activated, | ||
// The presence of an (empty) set also means that the dependency is activated. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like these lines are duplicated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just tweak the comment a bit further.
I'm not sure. I recall having separate maps seemed important at the time, but I don't remember.
I'm not sure I follow. With this change, it must insert an empty set, so skipping that step is no longer an option? |
2d02b29
to
7b44434
Compare
7b44434
to
2b913b6
Compare
@@ -321,21 +322,14 @@ impl ResolvedFeatures { | |||
.expect("activated_features for invalid package") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like we are not on the same page, @ehuss 😆
Maybe I miss some contexts, but it seems that we can switch to unwrap_or_default nowadays even if we stop inserting empty set from now on.
I'm not sure I follow. With this change, it must insert an empty set, so skipping that step is no longer an option?
activated_features_int
returns a Result
so unwrap_or_default
can easily recover from it with a default. There is no need of the presence of an empty set, no?
cargo/src/cargo/core/resolver/features.rs
Lines 352 to 363 in 1945b44
fn activated_features_int( | |
&self, | |
pkg_id: PackageId, | |
features_for: FeaturesFor, | |
) -> CargoResult<Vec<InternedString>> { | |
let fk = features_for.apply_opts(&self.opts); | |
if let Some(fs) = self.activated_features.get(&(pkg_id, fk)) { | |
Ok(fs.iter().cloned().collect()) | |
} else { | |
bail!("features did not find {:?} {:?}", pkg_id, fk) | |
} | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I was conflating "switch to unwrap_or_default" and "remove inserting the empty set", as I was thinking of them as a single action that would be done together. I suppose it could switch to unwrap_or_default, but I like the validation that expect
has been giving.
Thanks! @bors r+ |
☀️ Test successful - checks-actions |
14 commits in 7e484fc1a766f56dbc95380f45719698e0c82749..810cbad9a123ad4ee0a55a96171b8f8478ff1c03 2022-10-27 15:20:57 +0000 to 2022-11-02 21:04:31 +0000 - Update curl-sys (rust-lang/cargo#11326) - Mention fix on build script deadlock (rust-lang/cargo#11325) - Make cargo forward pre-existing CARGO if set (rust-lang/cargo#11285) - Clean up workspace dependencies after cargo remove (rust-lang/cargo#11242) - Update the outdated link for rust-semverver (rust-lang/cargo#11322) - Fix broken link to compilation entry point (rust-lang/cargo#11317) - Only remove fingerprints and build script artifacts of the requested package (rust-lang/cargo#10621) - Newer anyhow features are required (rust-lang/cargo#11316) - Clean stale git temp files (rust-lang/cargo#11308) - Report crate size on package and publish (rust-lang/cargo#11270) - add a note that some warnings (and/or errors) can be auto-fixed (rust-lang/cargo#10989) - Update libcurl (rust-lang/cargo#11307) - artifact deps shoud works when target field specified coexists with `optional = true` (rust-lang/cargo#11183) - Fix singular verb in tests page (rust-lang/cargo#11300)
Update cargo 14 commits in 7e484fc1a766f56dbc95380f45719698e0c82749..810cbad9a123ad4ee0a55a96171b8f8478ff1c03 2022-10-27 15:20:57 +0000 to 2022-11-02 21:04:31 +0000 - Update curl-sys (rust-lang/cargo#11326) - Mention fix on build script deadlock (rust-lang/cargo#11325) - Make cargo forward pre-existing CARGO if set (rust-lang/cargo#11285) - Clean up workspace dependencies after cargo remove (rust-lang/cargo#11242) - Update the outdated link for rust-semverver (rust-lang/cargo#11322) - Fix broken link to compilation entry point (rust-lang/cargo#11317) - Only remove fingerprints and build script artifacts of the requested package (rust-lang/cargo#10621) - Newer anyhow features are required (rust-lang/cargo#11316) - Clean stale git temp files (rust-lang/cargo#11308) - Report crate size on package and publish (rust-lang/cargo#11270) - add a note that some warnings (and/or errors) can be auto-fixed (rust-lang/cargo#10989) - Update libcurl (rust-lang/cargo#11307) - artifact deps shoud works when target field specified coexists with `optional = true` (rust-lang/cargo#11183) - Fix singular verb in tests page (rust-lang/cargo#11300) r? `@ghost`
Revert #11183 This reverts commit d4c38af, reversing changes made to 92d8826. Fixes #11330 --- The root cause is that [`map_to_features_for`] takes care of `unit_for.host_features` from parent unit, but [here] we only want to know whether the dependency is for host or not. We could have an ad-hoc `FeaturesFor` construction there, but I'd rather revert it at this moment. The interaction between `UnitFor` and `FeaturesFor` bites me every time 😭. After this revert, if we want to resolve #10526 again. The fastest way is doing the following: ```diff diff --git a/src/cargo/core/compiler/unit_dependencies.rs b/src/cargo/core/compiler/unit_dependencies.rs index 9319a19e0..14cc84941 100644 --- a/src/cargo/core/compiler/unit_dependencies.rs +++ b/src/cargo/core/compiler/unit_dependencies.rs `@@` -1103,7 +1103,7 `@@` impl<'a, 'cfg> State<'a, 'cfg> { // If this is an optional dependency, and the new feature resolver // did not enable it, don't include it. if dep.is_optional() { - let features_for = unit_for.map_to_features_for(dep.artifact()); + let features_for = FeaturesFor::from_for_host(unit_for.is_for_host()); if !self.is_dep_activated(pkg_id, features_for, dep.name_in_toml()) { return false; } ``` [`map_to_features_for`]: https://github.com/rust-lang/cargo/blob/810cbad9a123ad4ee0a55a96171b8f8478ff1c03/src/cargo/core/profiles.rs#L1043 [here]: https://github.com/rust-lang/cargo/blob/810cbad9a123ad4ee0a55a96171b8f8478ff1c03/src/cargo/core/compiler/unit_dependencies.rs#L1102
Revert #11183 This reverts commit d4c38af, reversing changes made to 92d8826. Fixes #11330 --- The root cause is that [`map_to_features_for`] takes care of `unit_for.host_features` from parent unit, but [here] we only want to know whether the dependency is for host or not. We could have an ad-hoc `FeaturesFor` construction there, but I'd rather revert it at this moment. The interaction between `UnitFor` and `FeaturesFor` bites me every time 😭. After this revert, if we want to resolve #10526 again. The fastest way is doing the following: ```diff diff --git a/src/cargo/core/compiler/unit_dependencies.rs b/src/cargo/core/compiler/unit_dependencies.rs index 9319a19e0..14cc84941 100644 --- a/src/cargo/core/compiler/unit_dependencies.rs +++ b/src/cargo/core/compiler/unit_dependencies.rs `@@` -1103,7 +1103,7 `@@` impl<'a, 'cfg> State<'a, 'cfg> { // If this is an optional dependency, and the new feature resolver // did not enable it, don't include it. if dep.is_optional() { - let features_for = unit_for.map_to_features_for(dep.artifact()); + let features_for = FeaturesFor::from_for_host(unit_for.is_for_host()); if !self.is_dep_activated(pkg_id, features_for, dep.name_in_toml()) { return false; } ``` [`map_to_features_for`]: https://github.com/rust-lang/cargo/blob/810cbad9a123ad4ee0a55a96171b8f8478ff1c03/src/cargo/core/profiles.rs#L1043 [here]: https://github.com/rust-lang/cargo/blob/810cbad9a123ad4ee0a55a96171b8f8478ff1c03/src/cargo/core/compiler/unit_dependencies.rs#L1102
Revert #11183 This reverts commit d4c38af, reversing changes made to 92d8826. Fixes #11330 --- The root cause is that [`map_to_features_for`] takes care of `unit_for.host_features` from parent unit, but [here] we only want to know whether the dependency is for host or not. We could have an ad-hoc `FeaturesFor` construction there, but I'd rather revert it at this moment. The interaction between `UnitFor` and `FeaturesFor` bites me every time 😭. After this revert, if we want to resolve #10526 again. The fastest way is doing the following: ```diff diff --git a/src/cargo/core/compiler/unit_dependencies.rs b/src/cargo/core/compiler/unit_dependencies.rs index 9319a19e0..14cc84941 100644 --- a/src/cargo/core/compiler/unit_dependencies.rs +++ b/src/cargo/core/compiler/unit_dependencies.rs `@@` -1103,7 +1103,7 `@@` impl<'a, 'cfg> State<'a, 'cfg> { // If this is an optional dependency, and the new feature resolver // did not enable it, don't include it. if dep.is_optional() { - let features_for = unit_for.map_to_features_for(dep.artifact()); + let features_for = FeaturesFor::from_for_host(unit_for.is_for_host()); if !self.is_dep_activated(pkg_id, features_for, dep.name_in_toml()) { return false; } ``` [`map_to_features_for`]: https://github.com/rust-lang/cargo/blob/810cbad9a123ad4ee0a55a96171b8f8478ff1c03/src/cargo/core/profiles.rs#L1043 [here]: https://github.com/rust-lang/cargo/blob/810cbad9a123ad4ee0a55a96171b8f8478ff1c03/src/cargo/core/compiler/unit_dependencies.rs#L1102
20 commits in 7e484fc1a766f56dbc95380f45719698e0c82749..9286a1beba5b28b115bad67de2ae91fb1c61eb0b 2022-10-27 15:20:57 +0000 to 2022-11-04 06:41:49 +0000 - chore: Upgrade dependencies (rust-lang/cargo#11328) - Clean more aggressively in CI (rust-lang/cargo#11335) - Remove remove_dir_all (rust-lang/cargo#11333) - test(publish): Cover more wait-for-publish cases (rust-lang/cargo#11327) - Revert rust-lang/cargo#11183 (rust-lang/cargo#11331) - fix(semver-check): adapt to a different error for variant not covered (rust-lang/cargo#11332) - Update curl-sys (rust-lang/cargo#11326) - Mention fix on build script deadlock (rust-lang/cargo#11325) - Make cargo forward pre-existing CARGO if set (rust-lang/cargo#11285) - Clean up workspace dependencies after cargo remove (rust-lang/cargo#11242) - Update the outdated link for rust-semverver (rust-lang/cargo#11322) - Fix broken link to compilation entry point (rust-lang/cargo#11317) - Only remove fingerprints and build script artifacts of the requested package (rust-lang/cargo#10621) - Newer anyhow features are required (rust-lang/cargo#11316) - Clean stale git temp files (rust-lang/cargo#11308) - Report crate size on package and publish (rust-lang/cargo#11270) - add a note that some warnings (and/or errors) can be auto-fixed (rust-lang/cargo#10989) - Update libcurl (rust-lang/cargo#11307) - artifact deps shoud works when target field specified coexists with `optional = true` (rust-lang/cargo#11183) - Fix singular verb in tests page (rust-lang/cargo#11300)
Update cargo 20 commits in 7e484fc1a766f56dbc95380f45719698e0c82749..9286a1beba5b28b115bad67de2ae91fb1c61eb0b 2022-10-27 15:20:57 +0000 to 2022-11-04 06:41:49 +0000 - chore: Upgrade dependencies (rust-lang/cargo#11328) - Clean more aggressively in CI (rust-lang/cargo#11335) - Remove remove_dir_all (rust-lang/cargo#11333) - test(publish): Cover more wait-for-publish cases (rust-lang/cargo#11327) - Revert rust-lang/cargo#11183 (rust-lang/cargo#11331) - fix(semver-check): adapt to a different error for variant not covered (rust-lang/cargo#11332) - Update curl-sys (rust-lang/cargo#11326) - Mention fix on build script deadlock (rust-lang/cargo#11325) - Make cargo forward pre-existing CARGO if set (rust-lang/cargo#11285) - Clean up workspace dependencies after cargo remove (rust-lang/cargo#11242) - Update the outdated link for rust-semverver (rust-lang/cargo#11322) - Fix broken link to compilation entry point (rust-lang/cargo#11317) - Only remove fingerprints and build script artifacts of the requested package (rust-lang/cargo#10621) - Newer anyhow features are required (rust-lang/cargo#11316) - Clean stale git temp files (rust-lang/cargo#11308) - Report crate size on package and publish (rust-lang/cargo#11270) - add a note that some warnings (and/or errors) can be auto-fixed (rust-lang/cargo#10989) - Update libcurl (rust-lang/cargo#11307) - artifact deps shoud works when target field specified coexists with `optional = true` (rust-lang/cargo#11183) - Fix singular verb in tests page (rust-lang/cargo#11300) r? `@ghost`
ci: Bump to nightly-2022-11-07 to address rust-lang/cargo#11183
> Adapted from rust-lang#11183 Previously, `is_dep_activated` depends on `activated_dependencies`, which is a map of `PackageFeaturesKey` to its own activated `DepFeature`s. `PackageFeaturesKey` in feature resolver is always comprised of * A `PackageId` of a given dependency, and * A enum value that helps decoupling activated features for that dependency. Currently depending on the type of given dependency. Looking into issue 10526, it has an `activated_dependencies` of ``` { (mycrate, NormalOrDev) -> [dep:mybindep] } ``` However, this [line][1] accidentally use a parent's `pkgid` and a dependency's `FeaturesFor` to compose a key: ``` (mycrate, ArtifactDep("x86_64-unknown-linux-gnu")) ``` That is never a valid key to query features for artifacts dependency. A valid key should be comprised of components both from the parent: ``` (mycrate, NormalOrDev) ``` Or both from the dependency itself: ``` (mybindep, ArtifactDep("x86_64-unknown-linux-gnu")) ``` By not passing artifact of dep in `map_to_features_for`, we can get a new `FeaturesFor` for parent `unit_for`. [1]: https://github.com/rust-lang/cargo/blob/a2ea66bea6fe8156444144e98911d073d56c2c0c/src/cargo/core/compiler/unit_dependencies.rs#L1106-L1107
> Adapted from rust-lang#11183 Previously, `is_dep_activated` depends on `activated_dependencies`, which is a map of `PackageFeaturesKey` to its own activated `DepFeature`s. `PackageFeaturesKey` in feature resolver is always comprised of * A `PackageId` of a given dependency, and * A enum value that helps decoupling activated features for that dependency. Currently depending on the type of given dependency. Looking into issue 10526, it has an `activated_dependencies` of ``` { (mycrate, NormalOrDev) -> [dep:mybindep] } ``` However, this [line][1] accidentally use a parent's `pkgid` and a dependency's `FeaturesFor` to compose a key: ``` (mycrate, ArtifactDep("x86_64-unknown-linux-gnu")) ``` That is never a valid key to query features for artifacts dependency. A valid key should be comprised of components both from the parent: ``` (mycrate, NormalOrDev) ``` Or both from the dependency itself: ``` (mybindep, ArtifactDep("x86_64-unknown-linux-gnu")) ``` This `unit_for` should already contain its own artifact dep information inside `artifact_target_for_features`, so we don't need to map any `dep.artifact()` here. [1]: https://github.com/rust-lang/cargo/blob/a2ea66bea6fe8156444144e98911d073d56c2c0c/src/cargo/core/compiler/unit_dependencies.rs#L1106-L1107
> Adapted from rust-lang#11183 Previously, `is_dep_activated` depends on `activated_dependencies`, which is a map of `PackageFeaturesKey` to its own activated `DepFeature`s. `PackageFeaturesKey` in feature resolver is always comprised of * A `PackageId` of a given dependency, and * A enum value that helps decoupling activated features for that dependency. Currently depending on the type of given dependency. Looking into issue 10526, it has an `activated_dependencies` of ``` { (mycrate, NormalOrDev) -> [dep:mybindep] } ``` However, this [line][1] accidentally use a parent's `pkgid` and a dependency's `FeaturesFor` to compose a key: ``` (mycrate, ArtifactDep("x86_64-unknown-linux-gnu")) ``` That is never a valid key to query features for artifacts dependency. A valid key should be comprised of components both from the parent: ``` (mycrate, NormalOrDev) ``` Or both from the dependency itself: ``` (mybindep, ArtifactDep("x86_64-unknown-linux-gnu")) ``` This `unit_for` should already contain its own artifact dep information inside `artifact_target_for_features`, so we don't need to map any `dep.artifact()` here. [1]: https://github.com/rust-lang/cargo/blob/a2ea66bea6fe8156444144e98911d073d56c2c0c/src/cargo/core/compiler/unit_dependencies.rs#L1106-L1107
> Adapted from rust-lang#11183 Previously, `is_dep_activated` depends on `activated_dependencies`, which is a map of `PackageFeaturesKey` to its own activated `DepFeature`s. `PackageFeaturesKey` in feature resolver is always comprised of * A `PackageId` of a given dependency, and * A enum value that helps decoupling activated features for that dependency. Currently depending on the type of given dependency. Looking into issue 10526, it has an `activated_dependencies` of ``` { (mycrate, NormalOrDev) -> [dep:mybindep] } ``` However, this [line][1] accidentally use a parent's `pkgid` and a dependency's `FeaturesFor` to compose a key: ``` (mycrate, ArtifactDep("x86_64-unknown-linux-gnu")) ``` That is never a valid key to query features for artifacts dependency. A valid key should be comprised of components both from the parent: ``` (mycrate, NormalOrDev) ``` Or both from the dependency itself: ``` (mybindep, ArtifactDep("x86_64-unknown-linux-gnu")) ``` This `unit_for` should already contain its own artifact dep information inside `artifact_target_for_features`, so we don't need to map any `dep.artifact()` here. [1]: https://github.com/rust-lang/cargo/blob/a2ea66bea6fe8156444144e98911d073d56c2c0c/src/cargo/core/compiler/unit_dependencies.rs#L1106-L1107
> Adapted from rust-lang#11183 Previously, `is_dep_activated` depends on `activated_dependencies`, which is a map of `PackageFeaturesKey` to its own activated `DepFeature`s. `PackageFeaturesKey` in feature resolver is always comprised of * A `PackageId` of a given dependency, and * A enum value that helps decoupling activated features for that dependency. Currently depending on the type of given dependency. Looking into issue 10526, it has an `activated_dependencies` of ``` { (mycrate, NormalOrDev) -> [dep:mybindep] } ``` However, this [line][1] accidentally use a parent's `pkgid` and a dependency's `FeaturesFor` to compose a key: ``` (mycrate, ArtifactDep("x86_64-unknown-linux-gnu")) ``` That is never a valid key to query features for artifacts dependency. A valid key should be comprised of components both from the parent: ``` (mycrate, NormalOrDev) ``` Or both from the dependency itself: ``` (mybindep, ArtifactDep("x86_64-unknown-linux-gnu")) ``` This `unit_for` is from parent and should already contain its own artifact dep information inside `artifact_target_for_features`, so we don't need to map any dependency artifact from `dep.artifact()`. [1]: https://github.com/rust-lang/cargo/blob/a2ea66bea6fe8156444144e98911d073d56c2c0c/src/cargo/core/compiler/unit_dependencies.rs#L1106-L1107
optional = true
optional = true
optional = true
optional = true
What does this PR try to resolve?
fixes #10526
Previously,
is_dep_activated
depends onactivated_dependencies
,which is a map of
PackageFeaturesKey
to its own activatedDepFeature
s.PackageFeaturesKey
in feature resolver is always comprised ofPackageId
of a given dependency, andCurrently depending on the type of given dependency.
Looking into issue 10526, it has an
activated_dependencies
ofHowever, this line accidentally use a parent's
pkgid
and a dependency's
FeaturesFor
to compose a key:That is never a valid key to query features for artifacts dependency.
A valid key should be comprised of components both from the parent:
Or both from the dependency itself:
As aforementioned
activated_dependencies
only stores parent packagesand their activated features. Those informations are included in
activated_features
as well, so this commit goes with the route thatremoves
activated_dependencies
and uses only dependency's infomationto query if itself is activated.
How should we test and review this PR?
This first commit logs the current "incorrect" behaviour.
The second commit contain the fix.
The last commit update some docs.
Additional information
There could be some other refactors to make the interaction between
-Zbindeps
and feature resolver a bit better. I'll try to send more follow-ups if life don't bite me again.