Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Unset buildid for reproducible builds #593

Closed
imjasonh opened this issue Feb 11, 2022 · 4 comments
Closed

RFC: Unset buildid for reproducible builds #593

imjasonh opened this issue Feb 11, 2022 · 4 comments

Comments

@imjasonh
Copy link
Member

If the same source is built on two different machines, ideally, they should produce the same binary, same image, same layer digests, same config diffids, etc.

This isn't the case today. If you take a recent ghcr.io/google/ko image produced by CI, and try to build the same image from the same commit, using the same Go compiler version, and using the same base image by digest (aided by #592), you still end up with different images.

$ git checkout f8d0aca1abeab6cf3d1800d15acc0592c7c976b7
$ ci=ghcr.io/google/ko@sha256:8fa68c86562684f946e92086a6a4d1f17ee116fff32e157e68f96dbf948b0783
# set base image to golang:1.17@digest in .ko.yaml, based on CI build logs
$ local=$(KO_DOCKER_REPO=gcr.io/imjasonh go run ./ build ./ --platform=all)
...
2022/02/11 11:01:22 Published gcr.io/imjasonh/ko-98b8c7facdad74510a7cae0cd368eb4e@sha256:73d4746ef0423a24b2e36705d5f648f8f6ed8ad2f70a74c43a1d799ea4135936

Note these images have different digests.

Only considering --platform=linux/amd64 for now, their manifests differ in the top layers produced by ko:

$ function manifest-platform-diff {
  diff <(crane manifest $1 --platform=linux/amd64 | jq) <(crane manifest $2 --platform=linux/amd64 | jq)
}
$ manifest-platform-diff $ci $local
7c7
<     "digest": "sha256:52fbe2d8d6f679aaf218c9696bd52cf46a0acf5734e55f477f767fcfc8171fac"
---
>     "digest": "sha256:431bf119280969354ea1b7f01d5057886cde0c5f415a9d6a702fb415b1f9e795"
52,53c52,53
<       "size": 14447415,
<       "digest": "sha256:103dce014357d1447a0ab5bdb2e0820ebe5b3825dc39ec094bec0833641c6f3c"
---
>       "size": 14451848,
>       "digest": "sha256:e73e6295963605d46fa599595e807f93d78035bbab37d5055c284d5fa597fe1b"

Configs diff only in the last diffID:

$ function config-diff {
  diff <(crane config $1 --platform=linux/amd64 | jq) <(crane config $2 --platform=linux/amd64 | jq)
}
$ config-diff $ci $local
91c91
<       "sha256:26b11d7436027c1a8b68d2f263694cb5d004cbaf82cf12fabd3e8bc57b83b675"
---
>       "sha256:a744dfc522c644d427686a3111398e19ce8ba62eb4d1091f7e4cf9773ab2dacc"

The Go binaries in each image:

$ function extract-file {
  crane export $1 - | tar -Oxf - $2      
}
$ extract-file $ci /ko-app/ko > cibin
$ extract-file $local /ko-app/ko > localbin

These are different sizes, and have different buildids embedded:

$ go tool buildid cibin
wfsXdWzess1HPZwqxpza/w1W8Gpkeb188ZJ6UexW7/fAQZlgOa-X8oaRDhGERu/OOojyWok9JTAbBpACeWS
$ go tool buildid localbin
fURyWN8_9H1KmClFNJrE/HWiELKLTvC1MFSvyxVmG/dsAR_RpmjFrxsRtRLtDJ/n5VsQTn6NFmq0lhuM8r4

We use buildid to avoid rebuilding/compressing/digesting binaries when KOCACHE is set, but I think maybe if KOCACHE isn't set we should consider also passing -ldflags=-buildid= as recommended here

Or, perhaps making it an opt-in choice like ko build ./ --reproducible?

@jonjohnsonjr
Copy link
Collaborator

Is the only diff the buildid?

@github-actions
Copy link

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Keep fresh with the 'lifecycle/frozen' label.

@dprotaso
Copy link
Contributor

I believe this was fixed in go1.14

golang/go#33772

@dprotaso
Copy link
Contributor

dprotaso commented Jul 4, 2023

It seems like it wasn't fixed.

Compiling two binaries on two different machines with -trimpath resulted in different checksums. Dropping the buildid gave me identical binaries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants