Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve zip file reproducibility #1602

Merged
merged 2 commits into from
Jul 30, 2024

Conversation

stevedlawrence
Copy link
Contributor

We currently use the ZipArchiveEntry(File, String) constructor when creating a zip file entry for Universal/packageBin. This constructor reads mtime, atime, and ctime from the File and adds them to the 5455 extended header in the zip file. When we call setTime on the entry, it only changes the mtime field--the atime and ctime are the same values from the file and are likely to be different across builds and break reproducibility.

To fix this, we use the ZipArchiveEntry(String) constructor which does not read any file metadata, and only uses information we directly provided to it. We now provie the source epoch via setLastModifiedTime since that continues to use the 5455 extended header but only for mtime. We also ensure directories have a trailing slash in the entry name, since that was previously done by the other constructor.

With this change, when using SOURCE_DATE_EPOCH, zip files created with Universal/packageBin are now byte-for-byte exactly the same.

We currently use the ZipArchiveEntry(File, String) constructor when
creating a zip file entry for Universal/packageBin. This constructor
reads mtime, atime, and ctime from the File and adds them to the 5455
extended header in the zip file. When we call setTime on the entry, it
only changes the mtime field--the atime and ctime are the same values
from the file and are likely to be different across builds and break
reproducibility.

To fix this, we use the ZipArchiveEntry(String) constructor which does
not read any file metadata, and only uses information we directly
provided to it. We now provie the source epoch via setLastModifiedTime
since that continues to use the 5455 extended header but only for mtime.
We also ensure directories have a trailing slash in the entry name,
since that was previously done by the other constructor.

With this change, when using SOURCE_DATE_EPOCH, zip files created with
Universal/packageBin are now byte-for-byte exactly the same.
@muuki88 muuki88 added universal Zip, tar.gz, tgz and bash issues patch release drafter version labels Jul 30, 2024
@muuki88
Copy link
Contributor

muuki88 commented Jul 30, 2024

This is awesome! Thanks a lot @stevedlawrence and thanks for your patience on this. Will try to release this ASAP

@muuki88 muuki88 merged commit 1d369b4 into sbt:master Jul 30, 2024
15 checks passed
@stevedlawrence stevedlawrence deleted the zip-source-date-epoch branch August 1, 2024 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
patch release drafter version universal Zip, tar.gz, tgz and bash issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants