-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support publishing preview releases #19
Conversation
77a0559
to
b3f8aab
Compare
runs-on: ubuntu-latest | ||
permissions: | ||
contents: write | ||
pull-requests: write |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In order for the workflow to comment on PRs (notifying that a preview has been published), we need the pull-requests: write
permission.
This also means updating all the release.yml
files in the projects currently using gha-scala-library-release-workflow
, so that they look like this:
permissions: { contents: write, pull-requests: write }
...I've now done this, as you can see in guardian/etag-caching#33 and the associated commits on all the other repos.
10950c6
to
c7f56f8
Compare
f300d2f
to
ddfbda2
Compare
version_suffix="" | ||
else | ||
release_type="PREVIEW_FEATURE_BRANCH" | ||
version_suffix="-PREVIEW.${GITHUB_REF_NAME//[^[:alnum:-_]]/}.$(date +%Y-%m-%dT%H%M).${GITHUB_SHA:0:8}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lots of stuff going on in the version_suffix
! :
- Starting
-
hyphen - https://semver.org/#spec-item-9 says "9. A pre-release version MAY be denoted by appending a hyphen ..." PREVIEW
- this is recognised by IntelliJ & Scala Steward to mean 'non-stable-release, not suitable for auto-upgrading to', which is what we want 👍${GITHUB_REF_NAME//[^[:alnum:-_]]/}
- this is the PR branch name, with most non-alphanumeric characters stripped out - in particular, stripping out slash (/
), which some people use in Git branch names but which we couldn't have as part of the version, because Maven uses the version as part of the artifact url path.$(date +%Y-%m-%dT%H%M)
- this is the date, to the minute (eg2024-01-04T1230
), helping to differentiate and order mulitple preview releases for one PR.${GITHUB_SHA:0:8}
- this is the first 8 digits of the hash id for the PR commit that the preview release was based on. This helps to differentiate and identify multiple preview release for one PR, but also is recognised by Scala Steward, again meaning 'non-stable-release, not suitable for auto-upgrading to', which is what we want 👍
Choice of version-suffix for preview releases
https://semver.org/#spec-item-9 says:
A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. Identifiers MUST NOT be empty. Numeric identifiers MUST NOT include leading zeroes. Pre-release versions have a lower precedence than the associated normal version. A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version. Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92, 1.0.0-x-y-z.--.
Build metadata MAY be denoted by appending a plus sign and a series of dot separated identifiers immediately following the patch or pre-release version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. Identifiers MUST NOT be empty. Build metadata MUST be ignored when determining version precedence. Thus two versions that differ only in the build metadata, have the same precedence. Examples: 1.0.0-alpha+001, 1.0.0+20130313144700, 1.0.0-beta+exp.sha.5114f85, 1.0.0+21AF26D3----117B344092BD.
How can we prevent tooling from thinking our preview releases are stable releases?
If we're not using the -SNAPSHOT
suffix, there's a risk that tooling will assume that our preview releases are stable releases, and attempt to auto-upgrade to them.
- IntelliJ automatically suggests dependency upgrades - it uses
PackageVersionNormalizer
(tests) with specific stability tokens that include 'preview' - Scala Steward raises dependency upgrade PRs - it uses
isPreRelease
which recognisesHash
(6+ or 8 hex chars) & specificAlpha
components that include 'preview'. See also Only consider versions with pre-release identifiers as pre-releases scala-steward-org/scala-steward#1033, Treat versions with hashes as pre-releases scala-steward-org/scala-steward#1549 etc - Scaladex uses
PreRelease.scala
Dont show invalid sbt and scalajs versions scalacenter/scaladex#614
Consequently, to be certain of being recognised as a pre-release, it seems wise to include these components in the version number:
- 'PREVIEW'
- a commit hash of at least 8 characters
NPM version numbers...
Some Guardian libraries are released simultaneously for both Scala and other platforms like NPM (for instance, content-api-models
, see guardian/content-api-models#229).
Both NPM and sbt/Maven work well with simple three-number semver version numbers, but how well will NPM handle extended version number like 1.0.7-PREVIEW.feature1.2024-01-04T1230.42ed11d4
?
For the time being, I'm happy enough to go ahead with this version-number structure, but alter it later if it causes problems.
'PREVIEW', 'BETA', 'ALPHA', or...?
Justin points out the 'PREVIEW' has a particular meaning for the Content Pipeline team ('Preview' vs 'Live' content) - could potentially cause some confusion.
I initially chose 'PREVIEW' (from the identifiers that IntelliJ & Scala Steward understand) partially because both 'BETA' & 'ALPHA' imply some meaning about the stage of development in the software release
cycle that may or may not be appropriate - it's not obvious which one truly reflects what we're doing when we make an early release from a PR, and it's annoying to have to choose. Additionally, looking at
the Wikipedia article that describes the different stages - the label 'pre-alpha' might even be more appropriate: https://en.wikipedia.org/wiki/Software_release_life_cycle
b23c6cf
to
3e490f9
Compare
@@ -143,12 +173,24 @@ jobs: | |||
|
|||
cat << EndOfFile >> $GITHUB_OUTPUT | |||
release_tag=$RELEASE_TAG | |||
release_version=${RELEASE_TAG#"v"} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here we're using Bash Shell Parameter Expansion to strip the v
off the front of the release tag (eg v1.0.6-PREVIEW.feature1.2023-12-31T1334.fd006d63
) to get the release version (which is what you'd use when declaring the library as a dependency, eg 1.0.6-PREVIEW.feature1.2023-12-31T1334.fd006d63
).
The RELEASE_VERSION
is used later on in the workflow in the PR comment, as that's probably what people want, rather than the tag:
cf021fa
to
6d51728
Compare
This addresses #10, providing support for publishing *preview* releases based off feature branches, ie PRs. The UX of doing a preview release is very similar to the existing process for doing a full release (https://github.com/guardian/gha-scala-library-release-workflow/blob/main/docs/making-a-release.md), with only these differences: * The developer needs to select the PR branch before clicking the green `Run workflow` button * The version number will be the **upcoming** version number, but with a suffix that clearly indicates this is a preview release, eg: `1.0.7-PREVIEW.feature1.2024-01-04T1230.42ed11d4`. Note that this is _not_ a `-SNAPSHOT` release, the workflow does not support `SNAPSHOT` releases. * No branches are updated by the release (ie, not the PR's feature branch, and not the default `main` branch) - the preview release commit exists as its own tagged commit, taking the latest PR commit as its parent. * GitHub release notes will not be created, but instead the PR using that branch will be updated with a comment providing details of the new release (version number, etc). Internally, these 'preview release' changes take place if a non-default branch (ie a feature branch, not `main`) is used: * The `🔒 Init` job `release_type` output is `PREVIEW_FEATURE_BRANCH` rather than `FULL_MAIN_BRANCH` * Only 1 commit is pushed by the workflow, rather than 2, and _not_ onto the branch - the single commit exists as a tagged leaf to the side of the PR branch. The 2nd commit normally needed by the full release process (incrementing the version number and adding the `-SNAPSHOT` suffix) is not needed for preview releases - there are already enough details in the `-PREVIEW` version-suffix to keep preview releases unique, even if you do many releases for 1 commit in 1 PR. * When that 1 commit is pushed, it's initially pushed with a _disposable_ Git tag - not the _release_ tag. Pushing _any_ commit requires either a branch or tag for the `git push` command to work on (you can't just push a commit id - I've tried), and there is no pre-existing suitable branch (we don't want to modify the PR feature branch) or tag (the release tag has an annotation message including the hashes of all artifact files generated by the release, and at the point when the commit is pushed, those artifacts & their hashes are not available yet), so we have to use a new, different, disposable, Git tag instead. ## Choice of version-suffix for preview releases https://semver.org/#spec-item-9 says: > 9. A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. Identifiers MUST NOT be empty. Numeric identifiers MUST NOT include leading zeroes. Pre-release versions have a lower precedence than the associated normal version. A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version. Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92, 1.0.0-x-y-z.--. > > 10. Build metadata MAY be denoted by appending a plus sign and a series of dot separated identifiers immediately following the patch or pre-release version. Identifiers MUST comprise only ASCII alphanumerics and hyphens [0-9A-Za-z-]. Identifiers MUST NOT be empty. Build metadata MUST be ignored when determining version precedence. Thus two versions that differ only in the build metadata, have the same precedence. Examples: 1.0.0-alpha+001, 1.0.0+20130313144700, 1.0.0-beta+exp.sha.5114f85, 1.0.0+21AF26D3----117B344092BD. ### How can we prevent tooling from thinking our preview releases are stable releases? If we're not using the `-SNAPSHOT` suffix, there's a risk that tooling will assume that our preview releases are stable releases, and attempt to auto-upgrade to them. * IntelliJ automatically suggests dependency upgrades - it uses `PackageVersionNormalizer` with specific stability tokens that include 'preview' * Scala Steward raises dependency upgrade PRs - it uses `isPreRelease` which recognises `Hash` (6+ or 8 hex chars) & specific `Alpha` components that include 'preview'. See also scala-steward-org/scala-steward#1033, scala-steward-org/scala-steward#1549 etc * Scaladex uses PreRelease.scala - scalacenter/scaladex#614 Consequently, to be certain of being recognised as a pre-release, it seems wise to include these components in the version number: * 'PREVIEW' * a commit hash of at least 8 characters ### NPM version numbers... Some Guardian libraries are released simultaneously for both Scala and other platforms like NPM (for instance, `content-api-models`, see guardian/content-api-models#229). Both NPM and sbt/Maven work well with simple three-number semver version numbers, but how well will NPM handle extended version number like `1.0.7-PREVIEW.feature1.2024-01-04T1230.42ed11d4` ? ### 'PREVIEW', 'BETA', 'ALPHA', or...? Justin points out the 'PREVIEW' has a particular meaning for the Content Pipeline team ('Preview' vs 'Live' content) - could potentially cause some confusion there. I initially chose 'PREVIEW' (from the identifiers that IntelliJ & Scala Steward understand) partially because both 'BETA' & 'ALPHA' imply some meaning about the stage of development in the software release cycle that may or may not be appropriate - it's not obvious which one truly reflects what we're doing when we make an early release from a PR, and it's annoying to have to choose. Additionally, looking at the Wikipedia article that describes the different stages - the label 'pre-alpha' might even be more appropriate: https://en.wikipedia.org/wiki/Software_release_life_cycle ## Problems with backticks Ideally we would be generating a markdown message for the PR comment with lots of backticks for styling: https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks ...like this: rtyley/sample-project-using-gha-scala-library-release-workflow#1 (comment) ...but they get interpreted by BASH, and cause trouble... https://github.com/rtyley/sample-project-using-gha-scala-library-release-workflow/actions/runs/7399435634/job/20130944058 ...so for the time being this PR avoids them in the generated PR comment.
6d51728
to
c573f19
Compare
|
||
cat comment_body.txt | ||
|
||
gh pr comment ${{ github.ref_name }} --body-file comment_body.txt >> $GITHUB_STEP_SUMMARY |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Problems with Bash interpreting backticks have meant that the comment message isn't quite as pretty as I'd like - ideally there would be backticks for styling code blocks, like this:
For the time being this PR avoids them in the generated PR comment.
guardian/gha-scala-library-release-workflow#19 introduces support for publishing preview releases, but note this does use the `pull-requests: write` permission in order to comment on the PR to announce that the release has been published - so we need to additionally grant that permission: https://github.com/guardian/gha-scala-library-release-workflow/pull/19/files#r1441589383
Now that guardian/gha-scala-library-release-workflow#19 is merged, we don't need to be using this branch of the workflow anymore.
Since guardian/gha-scala-library-release-workflow#19, we've been able to do preview releases too!
One update to this - Scaladex badges were failing to ignore pre-releases (even though the Scaladex does have the concept of a pre-release), but this has been fixed with scalacenter/scaladex#1330 |
The Maven badges we were using don't understand about pre-release version numbers, thankfully the Scaladex badges do! See also: * guardian/gha-scala-library-release-workflow#19 (review) * scalacenter/scaladex#1330
This should fix #37 - ie, a Full Main-Branch release should still succeed, even if someone adds additional commits to the `main` branch while the release workflow is running. Note that this change in behaviour only applies to Full Main-Branch releases, not Preview releases (#19), as Preview releases don't ever attempt to update any branch. To avoid the noise of an additional merge commit (ie a 3rd commit) we prefer a fast-forward, and only create a merge commit if it's _necessary_ - ie because we have to merge on top of some commits that someone else has added. ## Two full Main-Branch releases can't run concurrently Although we can now handle extra work on the `main` branch, running **two releases at once** is _not_ recommended! ## Using the GitHub API to create & merge commits, rather than `git` We're now using the GitHub API, rather than `git`, to do both these operations: * [Update the branch](https://docs.github.com/en/rest/git/refs?apiVersion=2022-11-28#update-a-reference) to try fast-forwarding - using the GitHub API means we don't need to clone a deep history of the repo to make that update * [Merge the branch](https://docs.github.com/en/rest/branches/branches?apiVersion=2022-11-28#merge-a-branch) if the fast-forward fails - using the GitHub API means that the merge commit will be signed/verified by GitHub too, [just like other commits created by the GitHub App](#26).
This addresses issue #10, providing support for publishing preview releases based off feature branches, ie Pull Requests!
It doesn't provide
SNAPSHOT
releases, but hopefully preview releases should serve most of the use cases.UX
The UX of doing a preview release is very similar to the existing process for doing a full release, with only these differences:
Run workflow
button1.0.7-PREVIEW.feature1.2024-01-04T1230.42ed11d4
.main
branch) - the preview release commit exists as its own tagged commit, taking the latest PR commit as its parent.Internal changes
Internally, these changes take place if a non-default branch (ie a feature branch, not
main
) is used:🔒 Init
jobrelease_type
output isPREVIEW_FEATURE_BRANCH
rather thanFULL_MAIN_BRANCH
-SNAPSHOT
suffix), is not needed for preview releases - there are already enough details in the-PREVIEW...
version-suffix to keep preview releases unique, even if you publish many releases for 1 commit in 1 PR.git push
command to work on (you can't just push a commit id - I've tried), and there is no pre-existing suitable branch (we don't want to modify the PR feature branch) or tag (the release tag has an annotation message including the hashes of all artifact files generated by the release, and at the point when the commit is pushed, those artifacts & their hashes are not available yet), so we have to use a new, different, disposable, Git tag instead.