Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Unused transitive dependencies got built #3001

Open
fa93hws opened this issue Nov 15, 2024 · 3 comments
Open

[Bug] Unused transitive dependencies got built #3001

fa93hws opened this issue Nov 15, 2024 · 3 comments

Comments

@fa93hws
Copy link

fa93hws commented Nov 15, 2024

Mini reproducible repository: https://github.com/fa93hws/rules_rust_transitive_deps

Step to reproduce:

  1. Checkout repo
  2. rustup target add wasm32-wasi
  3. cd swc_plugin
  4. cargo build-wasi and it passes
  5. bazel build :wasm, and it failed with
...
Running Cargo build script onig_sys failed
...

--stderr:


error occurred: Failed to find tool. Is `DUMMY_GCC_TOOL` installed?

Step to fix the error

fa93hws/rules_rust_transitive_deps#1 fixes the issue, by removing miette = { version = "7.2.0", features = ["fancy", "syntect-highlighter"] } from the closure.

Where I think goes wrong

I'm pretty new to rust and rules_rust so I maybe wrong. I'm not sure about the GCC tooling error but I think it's irrelevant here. The real problem is that onig_sys shouldn't be included in the build graph.
If I run cargo build-wasi under swc_plugin dir, the output looks like this, and there's no onig_sys at all.
QQ_1731652246337
But it's somehow built by rules_rust.

For context, onig_sys is a tranistive deps of miette with features = ["fancy", "syntect-highlighter"]. When there's only swc_plugin in the workspace, miette only has

dependencies = [
 "cfg-if",
 "miette-derive",
 "owo-colors",
 "textwrap",
 "thiserror",
 "unicode-width",
]

in the lock file.

But when miette = { version = "7.2.0", features = ["fancy", "syntect-highlighter"] } is added to another project in the workspace (another_rust_project), miette's dependencies becomes:

dependencies = [
  ...
 "supports-unicode",
 "syntect", <- it depends on onig_sys
 "terminal_size 0.3.0",
  ...
]

So I feel that the issue is that, rules_rust will take a look at the lock file, and build all transitive dependencies no matter whether they are actually needed or not? As a comparison, cargo build builds the needed dependencies only.

Other info:

Platform: MacOS 15.1 (24B83)
Rust version:

rustc --version
rustc 1.82.0 (f6e511eec 2024-10-15)
@max-heller
Copy link
Contributor

This is caused by Cargo feature unification:

When building multiple packages in a workspace (such as with --workspace or multiple -p flags), the features of the dependencies of all of those packages are unified. If you have a circumstance where you want to avoid that unification for different workspace members, you will need to build them via separate cargo invocations.

The working behavior you're seeing is because you're invoking cargo for just one of the packages.

  • What you're doing: cd swc_plugin && cargo build

    rules_rust_transitive_deps % cargo tree --manifest-path swc_plugin/Cargo.toml --depth 1 -p miette
    miette v7.2.0
    ├── cfg-if v1.0.0
    ├── miette-derive v7.2.0 (proc-macro)
    ├── owo-colors v4.1.0
    ├── textwrap v0.16.1
    ├── thiserror v1.0.69
    └── unicode-width v0.1.14
  • What rules_rust is doing:

    rules_rust_transitive_deps % cargo tree --depth 1 -p miette
    miette v7.2.0
    ├── backtrace v0.3.74
    ├── backtrace-ext v0.2.1
    ├── cfg-if v1.0.0
    ├── miette-derive v7.2.0 (proc-macro)
    ├── owo-colors v4.1.0
    ├── supports-color v3.0.1
    ├── supports-hyperlinks v3.0.0
    ├── supports-unicode v3.0.0
    ├── syntect v5.2.0
    ├── terminal_size v0.3.0
    ├── textwrap v0.16.1
    ├── thiserror v1.0.69
    └── unicode-width v0.1.14

    (It's actually inspecting the entire workspace, but this is the relevant part.)

Why is rules_rust doing things this way? Unlike cargo, which resolves crate and feature dependencies in the context of the target it's building (e.g., cargo build -p foo resolves dependencies differently from cargo build --workspace), rules_rust needs to resolve the entire set of crate and feature dependencies upfront to define bazel targets for each crate. As a result, it gets the "bigger picture" set of dependencies including the "unused" transitive dependency because as far as rules_rust knows it is used through the other crate.

@max-heller
Copy link
Contributor

As for fixing your issue, I don't fully understand the context of what's going wrong, but if it's a matter of compiling for WASI vs not you may want to look into platform specific dependencies.

@fa93hws
Copy link
Author

fa93hws commented Nov 17, 2024

Many thanks for your kindly explanation and helpful link, I think platform specific dependencies is exactly what I want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants