-
-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce a reproducible build system for Linux #6945
Conversation
Thank you for doing this! I need a arm64 linux build. |
Sorry that was a typo, I only managed to build |
@hackerbirds thank you very much for this investigation! |
No problem. When possible, could any of you please let me know if this idea/PR will go forward or not? Otherwise I'll close this PR and open a GitHub issue instead. I'm asking (no pressure though) because without approval and involvement from the devs, there isn't much users can do to reproduce 100% of an official build. |
@hackerbirds We've discussed reproducible builds and want to move forward with this plan, to use Docker similarly to the Android builds and to start with reproducible builds on Linux. We need to integrate it with the build process, and it's possible we'll need to make small adjustments to the PR, so please bear with us while we do that. Our hope and plan is to merge this PR. Thanks for your help! |
Thank you so much that's great news!! I'm really glad to see reproducible builds getting attention. Of course Debian is the least used supported OS but it's a step in the right direction. Let me know if you need help or want me to change/add anything. Currently this PR breaks |
We have merged this internally and are beginning to use this build system for our development builds! Some changes were needed to work with our build pipeline, so you will see that as a follow-on when the merge releases around next week. It's a work on progress, and we're looking forward to getting things working well enough so beta and prod builds can use this system as well. Thanks so much for your help! |
That's awesome, thank you for the updates and for putting the work into completing this PR! I'm guessing reproducibility has been successful on your machines (outside of CI/CD)? EDIT: I have (nearly) successfully reproduced the staging version |
I realized I have made a typo in the following section of [...] For instance, if you are trying to build 7.18.0, then do:
$ cd Signal-Desktop/
$ git checkout tags/7.18.0 I wrote I would add a commit to this PR to fix it, but since it was merged internally, I don't think I can do that anymore. |
@hackerbirds the md file is already in the public repo https://github.com/signalapp/Signal-Desktop/tree/main/reproducible-builds#building . Let me know if you'd like to open a PR, it sounds like a great change! |
Thank you so much again. I think the change at ec36ae7#diff-0968de0f82c073ea664db3244836eef7626ce26e2cdb4b431a4c2dfc9894f292R8 may cause reproducibility problems with the
If you got it working like that though that's cool. I guess I'll just wait for this week's merges to see if I can reproduce the build or not and let you know if I can and make another PR for the fix if needed |
Ah thanks for pointing out the timestamp detail and sorry for missing it! It shouldn't affect reproducibility of builds as the build script will use the latest git commit timestamp: https://github.com/signalapp/Signal-Desktop/blob/main/reproducible-builds/build.sh#L22, but we can update the default so builds without git can also be reproducible. |
Oh yeah, I forgot about that bit. Guess it shouldn't matter too much then |
Hopefully the last message I'll send here: I just wanted to confirm I have successfully reproduced Now we need more users trying it on their own computers... time will tell if this new build mechanism is stable enough for release builds and all. Thank you again for working on this (and the other related PRs and fixes) so quickly! |
First time contributor checklist:
Contributor checklist:
main
branchnpm run ready
run passes successfully (more about tests here)Description
This PR attempts to introduces reproducible builds on Linux.
Motivation for reproducible builds
What needs to be reproducible on Desktop
From what I gathered, there are three things making Signal Desktop non-reproducible:
.deb
, FPM was inserting timestamps in the.deb
files and tar archives (metadata). I have written more about it on Make release build's timestamp more deterministic #6940@signalapp/better-sqlite3
and@nodert-win10-rs4
. This is why I decided to use Docker, to ensure the C compilers were identical for everyone.What has been done in this PR
.deb
builds onamd64
.exe
builds: not done by this PR. Windows builds have to be built on Windows, and the Docker image is Linux..dmg
builds: same issue as above.build/dns-fallback.json
from.gitignore
to make it the same for everyone (though again there will need to be approval from the Desktop team because that would mean manually updating the file with say annpm
command, as I had suggested in the issue above).I have essentially repurposed Signal Android's reproducible build system to make Signal Desktop linux builds, for several reasons:
I have written a quick guide in
reproducible-builds/README.md
, though it will need more work. Essentially,reproducible-builds/build.sh
should do everything on its own, you just need Docker and git.I understand that this PR only covers Linux, at least for now, but in my opinion it is better than nothing. This is something that can only be improved, and is parallel to the existing build CI so it won't break it.
How to verify the builds
I tried to make it so that the entire
.deb
file is reproducible. Currently, the resulting.deb
build on my tree should have the SHA256 hash:34441fec941c4cb2450e50f3b0a2ce9170d2ded6ae73a8812bdfc37c29805050
. I invite more people to try building this themselves and verify that they are also getting the same hash. If you don't, let me know.If that doesn't work out (perhaps because of post-build signatures in the
.deb
file, but I don't think that's the case), then the contents ofsignal-desktop...deb/control.tar.gz/md5sums
can also be compared. That file contains the md5 hashes of all the files inside the.deb
.The downsides of this PR
Of course, this PR only makes sense if the Desktop team agrees to start building the official apps on this kind of build system. I am pretty sure this can be done automatically in CI (I don't know how Desktop is actually built/where the build comes from though), but this still adds extra complexity and effort, notably because of the DNS fallback issue.
I also understand that this is unfortunately not a high priority issue, which makes justifying drastic changes to the build process a lot harder, but I hope this PR will get the ball rolling.