Toolchain paths are being compiled into binaires #116948
Comments
rustbot
added
the
needs-triage
|
Could you do |
|
Also can you check which section actually contains these paths? Is it a single section or are there multiple? |
The flags are indeed being set |
Sorry but can you advise on how I would determine this? |
|
I don't have a mac, but |
I'm not entirely sure what I'm looking for, below is the result of |
|
For what it's worth I see host paths in linux binaries as well. I did the same thing mentioned in the original post in a ubuntu docker container but |
|
I think |
|
|
Seems to be |
|
Could you try dwarfdump on the executable and paste say 50 lines on either side of a mention of a path that should have been remapped? Be aware that dwarfdump prints a lot, so you may want to use grep with |
|
I'm assuming you mean for linux?
```
root@2355bc16a6a7:~/Code/hostpaths# dwarfdump ./target/debug/hostpaths | grep root/Code -C50
.debug_info COMPILE_UNIT :< 0><0x0000000b> DW_TAG_compile_unit DW_AT_producer clang LLVM (rustc version 1.73.0 (cc66ad4 2023-10-03)) DW_AT_language DW_LANG_Rust DW_AT_name src/main.rs/@/1x7unpbrf03a8pv0 DW_AT_stmt_list 0x00000000 DW_AT_comp_dir /root/Code/hostpaths DW_AT_GNU_pubnames yes(1) DW_AT_low_pc 0x00006fd4 DW_AT_high_pc 32 <highpc: 0x00006ff4> LOCAL_SYMBOLS:
|
|
Yeah, Linux. Looks like both the $cd /tmp
$ cargo new foo
$ cd foo
$ mkdir .cargo
$ cat > .cargo/config.toml <<EOF
$ [build]
$ rustflags = ["--remap-path-prefix=/tmp/foo=/bar"]
$ EOF
$ cat > src/main.rs <<EOF
fn main() {
println!("Hello, world!");
}
EOF
$ cargo build
$ dwarfdump target/debug/foo | grep /tmp
$ dwarfdump target/debug/foo | grep /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_decl_file 0x00000001 /bar/src/main.rs
0x00008a70 [ 1, 0] NS uri: "/bar/src/main.rs"
DW_AT_decl_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_decl_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_decl_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_call_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_call_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_call_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_call_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_call_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_call_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_call_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_decl_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_decl_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
DW_AT_call_file 0x000000db /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs
0x00000000 [ 63, 0] NS uri: "/rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs"
0x00000048 [ 131, 9] NS uri: "/rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs"
0x00000079 [ 141,13] NS uri: "/rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs"
0x000000e6 [ 132,12] NS uri: "/rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs"
0x000001ac [ 136,39] NS uri: "/rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs"
0x000001c6 [ 135,19] NS uri: "/rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs"
0x00000000 [ 150, 0] NS uri: "/rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sync/barrier.rs"
name at offset 0x00000058, length 4 is '/bar'
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar
DW_AT_comp_dir /bar |
Crap, I messed up the repro on linux. Linux appears to be working correctly for this case, though I have some other much more complicated cases where I see this. I'll try to get info there. Otherwise, the macOS case is still very reproducible if someone out there knows how to dig deeper. I'll do my best to learn about the Mach-O format too |
|
Could you send me a macOS executable with this issue? I could try if I can dig into it locally with llvm-objdump or some other tool that can read Mach-O files on Linux. |
|
|
Looks like all references to the https://milen.me/writings/apple-linker-ld64-deterministic-builds-oso-prefix/ Could you try passing |
bjorn3
added
O-macos
|
Building debug still includes these symbols but not for release |
|
Great! Release builds being reproducible is much more important than debug builds being reproducible, right? I could take a look at the debug case to see if that one can be made reproducible too if you want though. I will need the debug mode executable for that. |
|
Also could you try if |
This works for the release and debug variants. Incredible to know! Does this mean the debug info is not controlled by rustc? Also, how did you identify where the username was in the macOS binary? |
|
I looked at it using |
saethlin
removed
the
needs-triage
|
The issue is that macOS default linker always generates Let's first see how other tools do. Buck build system use a "postprocess" approach to "relativize" those paths. Buck2 and Bazel, clever enough, just uses Like bjorn3 says, I am seeing this as a bug of BTW, this is a small reproducer if people want to play with the options: mkdir rustc-oso
pushd rustc-oso
echo "fn main() {}" >> main.rs
rustc main.rs -g
nm -pa main | grep "$HOME" # find both `SO` and `OSO` symbols
rustc main.rs -g --remap-path-prefix="$HOME"=
nm -pa main | grep "$HOME" # find only `OSO` symbols
rustc main.rs -g -Clink-arg="-Wl,-oso_prefix,$HOME"
nm -pa main | grep "$HOME" # find only `SO` symbols
rustc main.rs -g --remap-path-prefix="$HOME"= -Clink-arg="-Wl,-oso_prefix,$HOME"
nm -pa main | grep "$HOME" # all paths were removed!
popd |
Is it that |
Not sure about the latter since I don't have any personal device old enough to verify that, but OSO symbols iiuc are for debugging purpose so should be fine if you don't debug? |
I noticed binaries produced by different users on the same machine resulting in binaries that were not byte-for-byte identical even though it was expected. Upon investigation, I found that absolute paths of components in the toolchain were being written into the binaries.
I created a new project
hostpathswithcargo init hostpathsand added the following.cargo/config.tomlfileI tried this code on a macOS machine:
Then I ran
xxdon the compiled binary (xxd target/debug/hostpaths) and in the output see sections with the paths I was trying to stripThe expectation based on the use of
--remap-path-prefixis that I would not see any references toandrebriscoin the binary at all.Meta
rustc --version --verbose:rustc +nightly --version --verboseThe text was updated successfully, but these errors were encountered: