Allow a build script (build.rs) in a virtual manifest Cargo.toml

[stanislav-tkach]

Describe the problem you are trying to solve

I have a package with several crates and "virtual" Cargo.toml (virtual manifest). I would like to execute some code before building this package or any containing crate. Currently it isn't allowed to have build.rs in the virtual manifest.

Describe the solution you'd like

Allow the build field to appear in the virtual manifest.

Notes

As a workaround I have created an empty crate with build.rs and made all other crates depend on it.

[alexcrichton]

Could you describe more how this might work? When would this be built? What crates would have access to it? (etc)

[stanislav-tkach]

I'm looking at this from the user perspective, so I cannot answer these questions to the full extent.

├── a
│   ├── Cargo.toml
│   └── src
│       └── main.rs
├── b
│   ├── Cargo.toml
│   └── src
│       └── main.rs
├── build.rs
└── Cargo.toml

Initially build.rs should be built and executed when any of cargo build, cargo build -p a or cargo build -p b commands are executed.
Additionally such build script should be controlled by regular rerun-if-changed rules.

I guess that in addition to OUT_DIR something like WORKSPACE_OUT_DIR can be added so all crates can access it.

Unfortunately I cannot share more details about my use-case.

[legenddcr]

This is a feature that I hope workspace could support. I would like to check rustc version in rust way with build.rs in workspace.

[nathan-at-least]

I just ran into this. I have a workspace which includes crates and a ./doc directory. I want cargo build to render the documentation and build all the crates.

My motivations are as follows:

• Don't expect other devs/users to run multiple commands to build all crates and render documentation.
• Don't rely on another tool like make since I want the workspace to be dead simple for rust users and I also don't want to introduce new build dependencies beyond cargo.

My guess is that this use of build.rs isn't expected or anticipated by cargo design, because it feels like it's oriented entirely towards building prerequisites for crates. Is there a different mechanism in cargo to capture "Build the crates and also do this other stuff to produce a full fledged set of artifacts"?

Separate from the issue of this ticket, there's a cargo documentation / usability problem here: I quickly skimmed the Cargo Reference Build Scripts section, and created a diagnostic / experiment build.rs which would print some info like the OUT_DIR environment then exit with an error. Then I ran cargo build, and to my surprise, the project built without those diagnostic outputs or error exits as I expected. It took me several minutes of googling things like "why isn't my build.rs running" before I realized maybe build.rs cannot exist for a workspace. This feels like a separate ticket for the cargo docs to me. I wouldn't be surprised if this trips up others with some frequency. A good solution, IMO, is somewhere in the build.rs intro docs it should say: "Notice that build.rs is for individual crates, not workspaces. Putting a build.rs file in a workspace has no effect." An even more user friendly solution would be for cargo to notice a workspace-scoped build.rs and warn the user it has no effect.

(Extra detail for my specific project: The docs in ./doc are an mdbook, and I actually anticipated using the mdbook crate rather than the binary, so that a user can simply run git clone ; cd ./project ; cargo build and get rendered docs without any separate local toolchain dependency (ie, no need to run cargo install mdbook). This is in service of both motivations above.)

[ckaran]

I'm running into this issue right now. My use case is that I have a large workspace which contains different sets of crates. Some crates compile on systems with configuration A, some on B, some on both, etc., etc., etc. Unfortunately, the configurations aren't as simple as Linux or Windows; a given configuration is guaranteed to have a particular set of modules and libraries, but a different configuration isn't. I can reliably detect the configuration I'm on at compile time, which would make a build.rs file ideal in the workspace root if it was possible to put one there, and if it was possible for the root build script to tell cargo which crates should be compiled.

Why is this important? Wasted time, and false compilation failures. Based on observation, it appears that cargo calculates the union of all dependent crates in a workspace before starting a compile. That means that rustc still has to deal with calculating the cfg attributes across all of those dependent crates. In some cases, this can take a very long time as some of the crates do compile time generation of bindings, which are then immediately thrown away because we're not on the right configuration. In other cases, the crate's dependencies fail to compile, causing a spurious compilation error. We already know that those crates can't compile on those systems, but we can't prevent cargo trying to set them up for compilation anyways.

If the workspace could have a build.rs file that was run before cargo attempted to compile the workspace's children, then the build.rs file could emit a list of crates that should be compiled. This would save some heartache and time while compiling.

N.B.: I know I can write a python script, or some other script to handle this outside of cargo, but that really feels counter to the Rust ethos...

[weihanglo]

My guess is that this use of build.rs isn't expected or anticipated by cargo design, because it feels like it's oriented entirely towards building prerequisites for crates.

I would say this is correct. From my understanding, build.rs was designed for building some native C libraries and Rust then consume it. Now people trying to use Cargo as a fully-fledged build system and that's unexpected. Having a workspace-level “pre-build hook” doesn't sound entirely wrong, but it feels like not a thing Cargo team is going to pursue in the near future.

Having a “pre-build” hook mechanism looks fine in a locak workspace development. However, looking into it deeper some devils in the details.

• If your package depends on a workspace member from a git repository (aka git dependency), and the workspace hook requires some stuff outside the package root directory. How should it work? You may end up building something out of the scope of the member your package is depending on.
• When running cargo publish, cargo packs a tarball and upload to crates.io. The tarball is expected to be self-contained and can compile. If building a member depending on a workspace-level pre-build hook, how do we ensure that the package can be built independently?

These questions are more or less the same — how to verify a package can be built without being a workspace member. To be fair, today people already are able to write a build script that fails to build a package. We discourage this kind of use of build script. However, if there is a workspace-level pre-build hook, the situation might be worse.

That being said, discussions about a workspace-level pre-build hook are pretty much welcome. More use cases can lead us to somewhere I believe.

---

I would like to check rustc version in rust way with build.rs in workspace.

Can rust-toolchain.toml meet your need? Or maybe the rust-version field in Cargo.toml?

I want cargo build to render the documentation and build all the crates.

Is it possible to write an xtask as a workspace member, and run it like cargo run -p myxtask. Even further you can configure alias in .cargo/config.toml so that other developers can run something simple like cargo gendoc. (Cargo the project itself does that)

… if it was possible for the root build script to tell cargo which crates should be compiled.

Something like #11313 or #9650 might help? Or do you need a runtime detection? Then I'd also recommend xtask at this moment.

[weihanglo]

Based on the complexity of the design work, and wanting more use cases, I am going to label them as

@rustbot label +S-needs-info +E-hard.