Skip to content

Commit

Permalink
Make Maven build reproducible (google#2543)
Browse files Browse the repository at this point in the history
However, it might still be necessary to use the same OS and JDK version
to actually be able to create identical artifacts.

This commit also formats the `pom.xml` files in the way the Maven Release
Plugin would.
  • Loading branch information
Marcono1234 authored and tibor-universe committed Aug 17, 2024
1 parent 9ee906e commit 342f2d5
Show file tree
Hide file tree
Showing 9 changed files with 154 additions and 13 deletions.
29 changes: 27 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
cache: 'maven'
- name: Build with Maven
# This also runs javadoc:jar to detect any issues with the Javadoc generated during release
run: mvn --batch-mode --update-snapshots --no-transfer-progress verify javadoc:jar
run: mvn --batch-mode --no-transfer-progress verify javadoc:jar

native-image-test:
name: "GraalVM Native Image test"
Expand All @@ -42,4 +42,29 @@ jobs:
- name: Build and run tests
# Only run tests in `graal-native-image-test` (and implicitly build and run tests in `gson`),
# everything else is covered already by regular build job above
run: mvn test --batch-mode --update-snapshots --no-transfer-progress --activate-profiles native-image-test --projects graal-native-image-test --also-make
run: mvn test --batch-mode --no-transfer-progress --activate-profiles native-image-test --projects graal-native-image-test --also-make

verify-reproducible-build:
name: "Verify reproducible build"
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: "Set up JDK 17"
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: 17
cache: 'maven'

- name: "Verify no plugin issues"
run: mvn artifact:check-buildplan --batch-mode --no-transfer-progress

- name: "Verify reproducible build"
# See https://maven.apache.org/guides/mini/guide-reproducible-builds.html#how-to-test-my-maven-build-reproducibility
run: |
mvn clean install --batch-mode --no-transfer-progress -Dproguard.skip -DskipTests
# Run with `-Dbuildinfo.attach=false`; otherwise `artifact:compare` fails because it creates a `.buildinfo` file which
# erroneously references the existing `.buildinfo` file (respectively because it is overwriting it, a file with size 0)
# See https://issues.apache.org/jira/browse/MARTIFACT-57
mvn clean verify artifact:compare --batch-mode --no-transfer-progress -Dproguard.skip -DskipTests -Dbuildinfo.attach=false
66 changes: 66 additions & 0 deletions ReleaseProcess.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,72 @@ This section was borrowed heavily from [Doclava release process](https://code.go

See [OSSRH Publish Guide](https://central.sonatype.org/publish/publish-guide/).

## Testing Maven release workflow locally

The following describes how to perform the steps of the release locally to verify that they work as desired.

**Warning:** Be careful with this, these steps might be outdated or incomplete. Doublecheck that you are working on a copy of your local Gson Git repository and make sure you have followed all steps. To be safe you can also temporarily turn off your internet connection to avoid accidentally pushing changes to the real remote Git or Maven repository.\
As an alternative to the steps described below you can instead [perform a dry run](https://maven.apache.org/maven-release/maven-release-plugin/usage.html#do-a-dry-run), though this might not behave identical to a real release.

1. Make a copy of your local Gson Git repository and only work with that copy
2. Make sure you are on the `main` branch
3. Create a temp directory outside the Gson directory\
In the following steps this will be called `#gson-remote-temp#`; replace this with the actual absolute file path of the directory, using only forward slashes. For example under Windows `C:\my-dir` becomes `C:/my-dir`.
4. Create the directory `#gson-remote-temp#/git-repo`
5. In that directory run

```sh
git init --bare --initial-branch=main .
```

6. Create the directory `#gson-remote-temp#/maven-repo`
7. Edit the root `pom.xml` of Gson
1. Change the `<developerConnection>` to

```txt
scm:git:file:///#gson-remote-temp#/git-repo
```

2. Change the `<url>` of the `<distributionManagement>` to

```txt
file:///#gson-remote-temp#/maven-repo
```

3. If you don't want to use GPG, remove the `maven-gpg-plugin` entry from the 'release' profile.\
There is also an entry under `<pluginManagement>`; you can remove that as well.
8. Commit the changes using Git
9. Change the remote repository of the Git project
<!-- Uses `txt` instead of `sh` to avoid the `#` being highlighted in some way -->
```txt
git remote set-url origin file:///#gson-remote-temp#/git-repo
```
10. Push the changes
```sh
git push origin main
```
Now you can perform the steps of the release:
1. ```sh
mvn release:clean
```
2. ```sh
mvn release:prepare
```
3. ```sh
mvn release:perform
```
4. Verify that `#gson-remote-temp#/git-repo` and `#gson-remote-temp#/maven-repo` contain all the desired changes
5. Afterwards delete all Gson files under `${user.home}/.m2/repository/com/google/code/gson` which have been installed in your local Maven repository during the release.\
Otherwise Maven might not download the real Gson artifacts with these version numbers, once they are released.
## Running Benchmarks or Tests on Android
* Download vogar
Expand Down
6 changes: 6 additions & 0 deletions extras/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@
<name>Gson Extras</name>
<description>Google Gson grab bag of utilities, type adapters, etc.</description>

<properties>
<!-- Make the build reproducible, see root `pom.xml` -->
<!-- This is duplicated here because that is recommended by `artifact:check-buildplan` -->
<project.build.outputTimestamp>2023-01-01T00:00:00Z</project.build.outputTimestamp>
</properties>

<licenses>
<license>
<name>Apache-2.0</name>
Expand Down
8 changes: 5 additions & 3 deletions graal-native-image-test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,7 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>

Expand All @@ -27,6 +25,10 @@
<artifactId>graal-native-image-test</artifactId>

<properties>
<!-- Make the build reproducible, see root `pom.xml` -->
<!-- This is duplicated here because that is recommended by `artifact:check-buildplan` -->
<project.build.outputTimestamp>2023-01-01T00:00:00Z</project.build.outputTimestamp>

<!-- GraalVM is JDK >= 17, however for build with regular JDK these tests
are also executed with JDK 11, so for them exclude JDK 17 specific tests -->
<maven.compiler.testRelease>11</maven.compiler.testRelease>
Expand Down
6 changes: 6 additions & 0 deletions gson/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@
</license>
</licenses>

<properties>
<!-- Make the build reproducible, see root `pom.xml` -->
<!-- This is duplicated here because that is recommended by `artifact:check-buildplan` -->
<project.build.outputTimestamp>2023-01-01T00:00:00Z</project.build.outputTimestamp>
</properties>

<dependencies>
<dependency>
<groupId>junit</groupId>
Expand Down
6 changes: 6 additions & 0 deletions metrics/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@
<name>Gson Metrics</name>
<description>Performance Metrics for Google Gson library</description>

<properties>
<!-- Make the build reproducible, see root `pom.xml` -->
<!-- This is duplicated here because that is recommended by `artifact:check-buildplan` -->
<project.build.outputTimestamp>2023-01-01T00:00:00Z</project.build.outputTimestamp>
</properties>

<licenses>
<license>
<name>Apache-2.0</name>
Expand Down
32 changes: 27 additions & 5 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@
This file has been modified by Happeo Oy.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"
child.project.url.inherit.append.path="false">
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd" child.project.url.inherit.append.path="false">
<modelVersion>4.0.0</modelVersion>

<groupId>com.google.code.gson</groupId>
Expand All @@ -43,6 +42,10 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.release>8</maven.compiler.release>
<maven.compiler.testRelease>17</maven.compiler.testRelease>

<!-- Make the build reproducible, see https://maven.apache.org/guides/mini/guide-reproducible-builds.html -->
<!-- Automatically updated by Maven Release Plugin -->
<project.build.outputTimestamp>2023-01-01T00:00:00Z</project.build.outputTimestamp>
</properties>

<!-- These attributes specify that the URLs should be inherited by the modules as is, to avoid constructing
Expand Down Expand Up @@ -171,8 +174,8 @@
<!-- For Markdown files removing trailing whitespace causes issues for hard line breaks,
which use two trailing spaces. However, the trailing spaces are difficult to notice anyway;
prefer a trailing `\` instead of two spaces. -->
<trimTrailingWhitespace/>
<endWithNewline/>
<trimTrailingWhitespace />
<endWithNewline />
<indent>
<spaces>true</spaces>
<!-- This seems to mostly (or only?) affect the suggested fix in case code contains tabs -->
Expand All @@ -194,10 +197,29 @@
<reorderImports>true</reorderImports>
<formatJavadoc>true</formatJavadoc>
</googleJavaFormat>
<formatAnnotations/> <!-- Puts type annotations immediately before types. -->
<formatAnnotations /> <!-- Puts type annotations immediately before types. -->
</java>
</configuration>
</plugin>

<!-- Attaches a `.buildinfo` file which contains information for reproducing the build,
such as OS, JDK version, ...
Since this is a multi-module Maven project, only one aggregated file will be created for
the last module, see the note on https://maven.apache.org/plugins/maven-artifact-plugin/usage.html#recording-buildinfo-file -->
<!-- The other goals of this plugin are run by the GitHub workflow to verify that
the build is reproducible (see `artifact:...` usage in the workflow) -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-artifact-plugin</artifactId>
<version>3.5.0</version>
<executions>
<execution>
<goals>
<goal>buildinfo</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>

<pluginManagement>
Expand Down
6 changes: 6 additions & 0 deletions proto/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@
<name>Gson Protobuf Support</name>
<description>Gson support for Protobufs</description>

<properties>
<!-- Make the build reproducible, see root `pom.xml` -->
<!-- This is duplicated here because that is recommended by `artifact:check-buildplan` -->
<project.build.outputTimestamp>2023-01-01T00:00:00Z</project.build.outputTimestamp>
</properties>

<licenses>
<license>
<name>Apache-2.0</name>
Expand Down
8 changes: 5 additions & 3 deletions shrinker-test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,7 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
Expand All @@ -28,6 +26,10 @@
<artifactId>shrinker-test</artifactId>

<properties>
<!-- Make the build reproducible, see root `pom.xml` -->
<!-- This is duplicated here because that is recommended by `artifact:check-buildplan` -->
<project.build.outputTimestamp>2023-01-01T00:00:00Z</project.build.outputTimestamp>

<maven.compiler.release>8</maven.compiler.release>
</properties>

Expand Down

0 comments on commit 342f2d5

Please sign in to comment.