Forwarding a build.rs env var and running `cargo build` within the binary causes rebuilds #12403

rukai · 2023-07-27T00:18:15Z

Problem

In some of my projects I do the following:

forward the env vars from build.rs up to my actual application
invoke cargo build from a test or example, this is useful for certain integration tests.

However the combination of these things is causing my application to be rebuilt.

Steps

build.rs:

fn main() {
    let profile = std::env::var("PROFILE").unwrap();
    println!("cargo:rustc-env=PROFILE={profile}");
    println!("cargo:rerun-if-env-changed=PROFILE");
}

main.rs:

fn main() {
    let output = std::process::Command::new(env!("CARGO"))
        .args(["build", "-vv"])
        .output()
        .unwrap();
    let stdout = String::from_utf8(output.stdout).unwrap();
    let stderr = String::from_utf8(output.stderr).unwrap();
    println!("stdout:\n{stdout}\nstderr:\n{stderr}");
}

Then run cargo run -vv to get the following output every time it reruns:

~2/Projects/Crates/foo_bin> cargo run -vv
       Dirty foo v0.1.0 (/home/rukai2/Projects/Crates/foo_bin): the env variable PROFILE changed
   Compiling foo v0.1.0 (/home/rukai2/Projects/Crates/foo_bin)
     Running `/home/rukai2/Projects/Crates/foo_bin/target/debug/build/foo-9cd20279bf15ecca/build-script-build`
[foo 0.1.0] cargo:rustc-env=PROFILE=debug
[foo 0.1.0] cargo:rerun-if-env-changed=PROFILE
     Running `CARGO=/home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_BIN_NAME=foo CARGO_CRATE_NAME=foo CARGO_MANIFEST_DIR=/home/rukai2/Projects/Crates/foo_bin CARGO_PKG_AUTHORS='Rukai <rubickent@gmail.com>' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=foo CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/rukai2/Projects/Crates/foo_bin/target/debug/deps:/home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib:/home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' OUT_DIR=/home/rukai2/Projects/Crates/foo_bin/target/debug/build/foo-b67a63e31a813826/out PROFILE=debug /home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name foo --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=191 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=3c9e043e35410f2e -C extra-filename=-3c9e043e35410f2e --out-dir /home/rukai2/Projects/Crates/foo_bin/target/debug/deps -C linker=clang -C incremental=/home/rukai2/Projects/Crates/foo_bin/target/debug/incremental -L dependency=/home/rukai2/Projects/Crates/foo_bin/target/debug/deps -C link-arg=-fuse-ld=/usr/bin/mold`
    Finished dev [unoptimized + debuginfo] target(s) in 0.08s
     Running `target/debug/foo`
stdout:
[foo 0.1.0] cargo:rustc-env=PROFILE=debug
[foo 0.1.0] cargo:rerun-if-env-changed=PROFILE

stderr:
       Dirty foo v0.1.0 (/home/rukai2/Projects/Crates/foo_bin): the env variable PROFILE changed
   Compiling foo v0.1.0 (/home/rukai2/Projects/Crates/foo_bin)
     Running `/home/rukai2/Projects/Crates/foo_bin/target/debug/build/foo-9cd20279bf15ecca/build-script-build`
     Running `CARGO=/home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_BIN_NAME=foo CARGO_CRATE_NAME=foo CARGO_MANIFEST_DIR=/home/rukai2/Projects/Crates/foo_bin CARGO_PKG_AUTHORS='Rukai <rubickent@gmail.com>' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=foo CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/rukai2/Projects/Crates/foo_bin/target/debug/deps:/home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib:/home/rukai2/Projects/Crates/foo_bin/target/debug/deps:/home/rukai2/Projects/Crates/foo_bin/target/debug:/home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' OUT_DIR=/home/rukai2/Projects/Crates/foo_bin/target/debug/build/foo-b67a63e31a813826/out PROFILE=debug /home/rukai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name foo --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=3c9e043e35410f2e -C extra-filename=-3c9e043e35410f2e --out-dir /home/rukai2/Projects/Crates/foo_bin/target/debug/deps -C linker=clang -C incremental=/home/rukai2/Projects/Crates/foo_bin/target/debug/incremental -L dependency=/home/rukai2/Projects/Crates/foo_bin/target/debug/deps -C link-arg=-fuse-ld=/usr/bin/mold`
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s

the reason for the crate being considered dirty is because the env variable PROFILE changed, however you can see in both the outer and inner cargo it is set to the same debug value.

Possible Solution(s)

No response

Notes

No response

Version

~2/Projects/Crates/foo_bin> cargo --version --verbose
cargo 1.71.0 (cfd3bbd8f 2023-06-08)
release: 1.71.0
commit-hash: cfd3bbd8fe4fd92074dfad04b7eb9a923646839f
commit-date: 2023-06-08
host: x86_64-unknown-linux-gnu
libgit2: 1.6.4 (sys:0.17.1 vendored)
libcurl: 8.0.1-DEV (sys:0.4.61+curl-8.0.1 vendored ssl:OpenSSL/1.1.1t)
ssl: OpenSSL 1.1.1t  7 Feb 2023
os: Arch Linux Rolling Release [64-bit]

The text was updated successfully, but these errors were encountered:

weihanglo · 2023-07-27T11:00:03Z

The first run of build.rs doesn't contain PROFILE_FORWARD but the second run has, so build script observed the env changed.

Not sure what you would like to achieve. Just call out the notice here in cargo:rustc-env.

Note: These environment variables are also set when running an
executable with cargo run or cargo test. However, this usage is
discouraged since it ties the executable to Cargo's execution environment.
Normally, these environment variables should only be checked at compile-time
with the env! macro.

rukai · 2023-07-27T11:18:05Z

Ah, I think I messed up minimising the issue, I'll look at it again tomorrow morning

rukai · 2023-07-27T23:18:26Z

Sorry about that I changed the example at the last minute to try to make the issue clearer but ended up making it into a non issue.

I've now edited the example back to what I think is an actual issue.
Because PROFILE is always provided by cargo it should never change and so it should not trigger a rebuild.
Please take another look, thankyou.

weihanglo · 2023-08-11T22:40:20Z

Sorry for the late reply.

If you set the env var, everything works as expected — PROFILE=debug cargo run -vv.

Let's break down it a bit.

No PROFLIE set for outer cargo run.
PROFILE set by outer build script.
Inner cargo build observes PROFILE changed (from empty to debug). Rebuild.
Go back to 1st step, outer cargo run observes PROFILE changed (from debug to empty), Rebuild.

weihanglo · 2023-08-11T22:44:03Z

I'd say it works perfectly correct on how env changes are detected. If you can share what you were working on, we may come up with a better idea :)

rukai · 2023-08-11T23:11:37Z

Ah! Thankyou for writing it out like that, I've now found my misunderstanding.
I thought that cargo:rerun-if-env-changed would check the env vars as accessible by build.rs, considering I im in a build.rs after all.
But what you are saying, and what makes sense given the functionality here is that it checks the env vars as received by cargo itself.

I think we should document this on https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorerun-if-env-changedname
Here it suggests you may not need it for TARGET but it does not explicitly say that such environment variables will not be accessible to rerun-if-env-changed

weihanglo · 2023-08-11T23:23:29Z

Sorry I don't really understand what isn't accessible to rerun-if-env-changed. Every environment variable you set when invoking cargo will take into account. What do you look for to document?

FWIW, here is a list of environment variables Cargo sets for build scripts.

weihanglo · 2023-08-11T23:32:51Z

Maybe this?

- The rerun-if-env-changed instruction tells Cargo to re-run the build script if the value of an environment variable of the given name has changed.
+ The rerun-if-env-changed instruction tells Cargo to re-run the build script if the value of an environment variable of the given name that Cargo received has changed.

rukai · 2023-08-11T23:39:22Z

I was thinking like this:

The `rerun-if-env-changed` instruction tells Cargo to re-run the build script
 if the value of an environment variable of the given name has changed.
 
 Note that the environment variables here are intended for global environment
-variables like `CC` and such, it is not necessary to use this for environment
+variables like `CC` and such, it is not possible to use this for environment
 variables like `TARGET` that Cargo sets.
+That is because the environment variables used are those received by cargo
+not those received by the build.rs executable.

The main thing for me is that it is not necessary becomes it is not possible.

weihanglo · 2023-08-12T00:16:21Z

Hmm… now I see what you mean. Thanks for the reply! It is not necessary since Cargo already handles recompile when target changes. The rerun-if-env-changed detection happens before cargo sets env for build script, so it is also true that it is not possible.

What about this?

 Note that the environment variables here are intended for global environment
-variables like `CC` and such, it is not necessary to use this for environment
-variables like `TARGET` that Cargo sets.
+variables like `CC` and such, it is not possible to use this for environment
+variables like `TARGET` that [Cargo sets for build scripts][build-env]. The
+environment variables in use are those received by `cargo` invocations, not
+those received by the executable of the build script.

(Sorry a bit picky and feel free to submit a pull request!!)

rukai · 2023-08-12T00:21:57Z

that looks excellent, thanks for working through this with me, I'll make a PR.

rukai added C-bug Category: bug S-triage Status: This issue is waiting on initial triage. labels Jul 27, 2023

weihanglo added S-needs-info Status: Needs more info, such as a reproduction or more background for a feature request. and removed S-triage Status: This issue is waiting on initial triage. labels Aug 11, 2023

rukai mentioned this issue Aug 12, 2023

docs: make the env var source of rerun-if-env-changed clearer #12482

Merged

bors closed this as completed in 65cee64 Aug 12, 2023

weihanglo mentioned this issue Aug 13, 2023

rerun-if-env-changed=CARGO_MANIFEST_DIR changing doesn't trigger a rebuild on change #8693

Open

rukai mentioned this issue Aug 14, 2023

docs: further tweak rerun-if-env-changed docs #12489

Closed

CraftSpider mentioned this issue Mar 26, 2024

Don't rebuild tectonic(bin) when non-binary files are changed tectonic-typesetting/tectonic#1173

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Forwarding a build.rs env var and running `cargo build` within the binary causes rebuilds #12403

Forwarding a build.rs env var and running `cargo build` within the binary causes rebuilds #12403

rukai commented Jul 27, 2023 •

edited

Loading

weihanglo commented Jul 27, 2023

rukai commented Jul 27, 2023

rukai commented Jul 27, 2023 •

edited

Loading

weihanglo commented Aug 11, 2023

weihanglo commented Aug 11, 2023

rukai commented Aug 11, 2023

weihanglo commented Aug 11, 2023 •

edited

Loading

weihanglo commented Aug 11, 2023

rukai commented Aug 11, 2023 •

edited

Loading

weihanglo commented Aug 12, 2023

rukai commented Aug 12, 2023

Forwarding a build.rs env var and running cargo build within the binary causes rebuilds #12403

Forwarding a build.rs env var and running cargo build within the binary causes rebuilds #12403

Comments

rukai commented Jul 27, 2023 • edited Loading

Problem

Steps

Possible Solution(s)

Notes

Version

weihanglo commented Jul 27, 2023

rukai commented Jul 27, 2023

rukai commented Jul 27, 2023 • edited Loading

weihanglo commented Aug 11, 2023

weihanglo commented Aug 11, 2023

rukai commented Aug 11, 2023

weihanglo commented Aug 11, 2023 • edited Loading

weihanglo commented Aug 11, 2023

rukai commented Aug 11, 2023 • edited Loading

weihanglo commented Aug 12, 2023

rukai commented Aug 12, 2023

Forwarding a build.rs env var and running `cargo build` within the binary causes rebuilds #12403

Forwarding a build.rs env var and running `cargo build` within the binary causes rebuilds #12403

rukai commented Jul 27, 2023 •

edited

Loading

rukai commented Jul 27, 2023 •

edited

Loading

weihanglo commented Aug 11, 2023 •

edited

Loading

rukai commented Aug 11, 2023 •

edited

Loading