cmd/go: aix-ppc64 distpack is not reproducible

[heschi]

While testing http://go.dev/cl/502537 I discovered that the aix-ppc64 distpack isn't reproducible. All the toolchain binaries differ, and the only difference I can see is the build ID:

$ diff <(xxd a/go/pkg/tool/aix_ppc64/nm) <(xxd b/go/pkg/tool/aix_ppc64/nm)
118,121c118,121
< 00000750: 5444 5f4a 516e 5763 5561 4364 434e 3335  TD_JQnWcUaCdCN35
< 00000760: 7137 6665 2f37 4349 6231 3137 5042 634f  q7fe/7CIb117PBcO
< 00000770: 4344 3969 6575 6b55 5f2f 5546 646b 5665  CD9ieukU_/UFdkVe
< 00000780: 6379 3541 7233 7153 5930 5437 6e78 2f69  cy5Ar3qSY0T7nx/i
---
> 00000750: 4f42 3670 4947 305f 4348 6b49 6c66 7058  OB6pIG0_CHkIlfpX
> 00000760: 4138 5948 2f63 6f52 4679 6f41 4648 4d71  A8YH/coRFyoAFHMq
> 00000770: 5a35 7130 2d6b 344e 702f 7a50 7330 6463  Z5q0-k4Np/zPs0dc
> 00000780: 7478 5a31 4f6e 4369 6366 4e4b 456a 2f69  txZ1OnCicfNKEj/i

I can reproduce the problem on my workstation now that https://go.dev/cl/502699 has landed:

$ GOOS=aix GOARCH=ppc64 ./make.bash -distpack                                                
[...]
Packaging archives for aix/ppc64.
distpack: c541533292224f52 go1.21beta1.src.tar.gz
distpack: 8f3728909c393af9 go1.21beta1.aix-ppc64.tar.gz
distpack: 75acc5c5c32adf65 v0.0.1-go1.21beta1.aix-ppc64.zip
distpack: 58528cce1848ddf4 v0.0.1-go1.21beta1.aix-ppc64.mod
distpack: 8f1a0d16e664ea04 v0.0.1-go1.21beta1.aix-ppc64.info
---
Installed Go for aix/ppc64 in /usr/local/google/home/heschi/tmp/repro/go
Installed commands in /usr/local/google/home/heschi/tmp/repro/go/bin
*** You need to add /usr/local/google/home/heschi/tmp/repro/go/bin to your PATH.
$ GOOS=aix GOARCH=ppc64 ./make.bash -distpack
[...]
Packaging archives for aix/ppc64.
distpack: c541533292224f52 go1.21beta1.src.tar.gz
distpack: 9973486e551de3b2 go1.21beta1.aix-ppc64.tar.gz
distpack: cac88cd492aa35cd v0.0.1-go1.21beta1.aix-ppc64.zip
distpack: 58528cce1848ddf4 v0.0.1-go1.21beta1.aix-ppc64.mod
distpack: 8f1a0d16e664ea04 v0.0.1-go1.21beta1.aix-ppc64.info
---
Installed Go for aix/ppc64 in /usr/local/google/home/heschi/tmp/repro/go
Installed commands in /usr/local/google/home/heschi/tmp/repro/go/bin
*** You need to add /usr/local/google/home/heschi/tmp/repro/go/bin to your PATH.

cc @rsc @bcmills @golang/aix @golang/ppc64

[bcmills]

That's curious. Does the aix/ppc64 release have any C dependencies, or tools linked using the system linker? (I saw aix/ppc64 reproducibility issues in #56896, but it looked like a cgo-specific issue.)

[bcmills]

@golang/compiler can correct me if I'm wrong, but I believe that aix is the only platform that generates XCOFF executable files. Perhaps there is some instability in the compiler or linker's support for that format?

[bcmills]

The Ftimedat field in the XCOFF header seems suspicious:
https://cs.opensource.google/go/go/+/master:src/internal/xcoff/xcoff.go;l=21;drc=9d81ccbf51b08bb29efad5fa9acc8fa69bbd8f6b

Perhaps we should clear out that field (or write a bogus, deterministic value) when building the release binaries?

Hmm, but it looks like we already do:
https://cs.opensource.google/go/go/+/master:src/cmd/link/internal/ld/xcoff.go;l=1535;drc=d8d22664c65b0054fc9ea6ba7e945481348fc8fc

[rsc]

I'm slowly narrowing it down. It's nothing to do with xcoff.

[rsc]

Sort is introducing non-determinism.

[gopherbot]

Change https://go.dev/cl/502755 mentions this issue: cmd/internal/obj: make aix/ppc64 builds reproducible