From d967417fad478648a63a5b688fa014f48ee36f9c Mon Sep 17 00:00:00 2001 From: Roberto Tyley Date: Fri, 26 Apr 2024 15:03:54 +0100 Subject: [PATCH] Build library with Java version specified in .tool-versions This allows projects to specify, with an `asdf` (https://asdf-vm.com/)-formatted `.tool-versions` file, the Java version to be used by the workflow for building the project- `gha-scala-library-release-workflow` has always used a recent LTS version Java for the build (eg Java 17), and this has sometimes been _too recent_ (https://github.com/guardian/atom-maker/pull/94) for some projects at the Guardian. We _want_ projects to update to Java 21 (see https://github.com/guardian/support-frontend/pull/5792, https://github.com/guardian/scala-steward-public-repos/pull/67 etc), but tying dozens of projects tightly to what `gha-scala-library-release-workflow` is using will make updating that version pretty hard. If, at some later point, _some_ projects want to experiment with Java 25, we shouldn't have to force all other projects to be compatible with that first. ## How to express the version of Java required... ### Configuration proliferation is a problem... One option would have been to simply add a new input parameter to the workflow, specifying the required Java version, but that has a downside: it proliferates the number of places in a project where the desired Java version is declared - and there are many in use at the Guardian, with no well-agreed canonical source-of-truth: * GHA `ci.yml`, in the [`java-version`](https://github.com/guardian/etag-caching/blob/7ecc04981f5a42a0f2ecb10631f28da571a49836/.github/workflows/ci.yml#L22) field of `actions/setup-java` - this has been my favourite in the past, as whatever CI runs with is usually pretty close to the truth * In sbt, the `scalacOptions` of `-target`, `-release`, `-java-output-version` (currently `-release` preferred, tho' once [support](https://github.com/scala/scala/pull/10654) is pervasive, `-java-output-version` is probably best) and the `javacOptions` of `-target` & `-source` - these all effectively set lower-bounds on the version of Java supported, rather than guaranteeing a minimum upper bound of Java version the way CI does. * In apps running on Amigo AMI images; the Java version baked into the AMI, usually [referenced](https://github.com/guardian/mobile-apps-api/blob/3231e6bf064163c6d0e72c8dc862678c68eb0b62/mobile-fronts/conf/riff-raff.yaml#L10) by `riff-raff.yaml` * In AWS Lambdas; the cloudformation [`Runtime`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.html#cfn-lambda-function-runtime) parameter, often set [by CDK](https://github.com/guardian/mobile-save-for-later/blob/1ac12e4c0100edb976ebae9e2a9975ad2321e26e/cdk/lib/mobile-save-for-later.ts#L44) ### ...`asdf`'s `.tool-versions` flle offers some consolidation Benefits of using `.tool-versions`: * Developers will automatically get the right Java version if they have `asdf` installed * https://github.com/actions/setup-java/pull/606 has added early support for reading the version of Java used in CI by `setup-java` from the `.tool-versions` flle - unfortunately, it's of limited use to us at the moment because of https://github.com/actions/setup-java/issues/615, but it's a promising start _(`setup-java` also has support for `jEnv` files, but they are not commonly used at the Guardian)_ * Format of the file is simple enough that it can be easily understood and used, even if `asdf` is not in use - **including the file doesn't _force_ developers to use `asdf`** (I know some devs have been having some problems with it, and maybe there are/will be better alternatives out there), but it clearly documents the Java version to be used. This does mean that **all library repos need to add a `.tool-versions` file** before this PR is merged. Helpful error messaging has been added with this PR to handle cases where the file is missing or invalid: #### Only Java _major_ version is guaranteed Note that although `asdf` requires a fully-specified Java version (eg `21.0.3.9.1`) in the `.tool-versions` file, currently the workflow will only match the *major* version of Java specified in the file (eg `21`), and will _always_ use the AWS Corretto distribution of Java. This is due to [limitations](https://github.com/actions/setup-java/issues/615) in [`actions/setup-java`](https://github.com/actions/setup-java). --- .github/workflows/reusable-release.yml | 30 +++++++++++++++++++---- docs/configuration.md | 33 ++++++++++++++++++++++---- 2 files changed, 55 insertions(+), 8 deletions(-) diff --git a/.github/workflows/reusable-release.yml b/.github/workflows/reusable-release.yml index f3a7386..ee384a6 100644 --- a/.github/workflows/reusable-release.yml +++ b/.github/workflows/reusable-release.yml @@ -110,12 +110,31 @@ jobs: name: 🎊 Test & Version needs: init runs-on: ubuntu-latest + outputs: + library_build_major_java_version: ${{ steps.establish_java_for_library_build.outputs.library_build_major_java_version }} steps: - uses: actions/checkout@v4 + - id: establish_java_for_library_build + name: Establish library build Java version + run: | + if [ ! -f .tool-versions ]; then + echo "::error title=Missing .tool-versions file::gha-scala-library-release-workflow requires an asdf-format .tool-versions file to establish the Java version for the build." + exit 1 + fi + LIBRARY_BUILD_MAJOR_JAVA_VERSION=$( grep -Eo 'java [[:alnum:]-]+-[[:digit:]]+' .tool-versions | rev | cut -d'-' -f1 | rev ) + echo "Using Java $LIBRARY_BUILD_MAJOR_JAVA_VERSION" + if [ -z "${LIBRARY_BUILD_MAJOR_JAVA_VERSION}" ]; then + echo "::error title=Missing Java version in .tool-versions file::Could not establish the library's required Java version - the '.tool-versions' file should have a line like 'java corretto-21.0.3.9.1'." + exit 1 + fi + + cat << EndOfFile >> $GITHUB_OUTPUT + library_build_major_java_version=$LIBRARY_BUILD_MAJOR_JAVA_VERSION + EndOfFile - uses: actions/setup-java@v4 # don't 'cache: sbt', at least until https://github.com/actions/setup-java/pull/564 is merged with: distribution: corretto - java-version: 17 + java-version: ${{ steps.establish_java_for_library_build.outputs.library_build_major_java_version }} # - name: Debug MIMA assessment # run: | # sbt "show versionPolicyFindIssues" @@ -140,7 +159,10 @@ jobs: ls -lR $GITHUB_WORKSPACE - name: Job summary run: | - echo "# Release $(git describe --tags --abbrev=0)" >> $GITHUB_STEP_SUMMARY + cat << EndOfFile >> $GITHUB_STEP_SUMMARY + # Release $(git describe --tags --abbrev=0) + Library built with Java ${{ steps.establish_java_for_library_build.outputs.library_build_major_java_version }}. + EndOfFile - uses: actions/cache/save@v4 with: path: repo-with-unsigned-version-update-commits.git @@ -239,7 +261,7 @@ jobs: create-artifacts: name: 🎊 Create artifacts - needs: [init, push-release-commit] + needs: [init, generate-version-update-commits, push-release-commit] runs-on: ubuntu-latest outputs: ARTIFACT_SHA256SUMS: ${{ steps.record-hashes.outputs.ARTIFACT_SHA256SUMS }} @@ -250,7 +272,7 @@ jobs: - uses: actions/setup-java@v4 # don't 'cache: sbt', at least until https://github.com/actions/setup-java/pull/564 is resolved with: distribution: corretto - java-version: 17 + java-version: ${{ needs.generate-version-update-commits.outputs.library_build_major_java_version }} - name: Generate artifacts run: | cat << EndOfFile > sbt-commands.txt diff --git a/docs/configuration.md b/docs/configuration.md index 3d7506f..b9641e6 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -43,6 +43,30 @@ Your repo will require access to [release credentials](credentials/supplying-cre [pass on those secrets](https://github.com/guardian/etag-caching/blob/9935da29e76b8b89759bcfe967cc7c1c02aa1814/.github/workflows/release.yml#L11-L13) to the workflow. +## Java version + +[Example `.tool-versions`](https://github.com/guardian/etag-caching/blob/main/.tool-versions) + +Your repository *must* contain an [`asdf`](https://asdf-vm.com/)-formatted `.tool-versions` file +in the root of the repository, specifying the Java version to be used by the workflow for +building your project, eg: + +``` +java corretto-21.0.3.9.1 +``` + +Note that although `asdf` requires a fully-specified Java version (eg `21.0.3.9.1` - use +`asdf list-all java` to list all possible Java versions), currently the workflow will only +match the *major* version of Java specified in the file (eg `21`), and will _always_ use the +AWS Corretto distribution of Java. This is due to +[limitations](https://github.com/actions/setup-java/issues/615) in +[`actions/setup-java`](https://github.com/actions/setup-java). + +As recommended [below](#recommended-sbt-settings), you should also specify a `-release` flag in +`scalacOptions` to ensure that your library is compiled for any older versions of Java you wish +to support, even if you're taking advantage of a more recent version of Java for _building_ the +library. + ## `sbt` ### Recommended `sbt` plugins @@ -79,10 +103,11 @@ to the workflow. to Maven Central. * `scalacOptions` should include `-release:11` (available with Scala [2.13.9](https://www.scala-lang.org/news/2.13.9) and above, also known as `-java-output-version` - [in Scala 3](https://www.scala-lang.org/blog/2022/04/12/scala-3.1.2-released.html#changes-to-other-compatibility-flags)) - - the workflow will always use one of the most recent LTS releases of Java - [supported by Scala](https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html), - but the generated class files will be compatible with whichever version of Java you target. + [in Scala 3](https://www.scala-lang.org/blog/2022/04/12/scala-3.1.2-released.html#changes-to-other-compatibility-flags)), or whatever minimum version of Java you want to support. + The workflow will _build_ your project with whatever Java version you declare in [`.tool-versions`](#java-version) - + but while this can be a relatively new version of Java, in order for your compiled code to support + _older_ versions of Java, and avoid `UnsupportedClassVersionError` errors, you'll + need to set this flag. See also [Scala/Java compatibility](https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html). * Top-level 'release' module - if your project has a [multi-module](https://www.scala-sbt.org/1.x/docs/Multi-Project.html) build this could be called 'root', or, if your project only has one module, it and your artifact-producing module could be the same thing, and just use top-level settings.