Skip to content

Commit

Permalink
Fix collision between macOS workflow artifacts in release workflows
Browse files Browse the repository at this point in the history
GitHub Workflows are used to automatically generate and publish production and nightly releases of the project. This is
done for a range of host architectures, including macOS. The macOS builds are then put through a notarization process in
a dedicated workflow job.

GitHub Actions workflow artifacts are used to transfer the generated files between sequential jobs in the workflow. The
"actions/upload-artifact" and "actions/download-artifact" actions are used for this purpose.

The workflow artifact handling had to be reworked recently in order to handle a breaking change in the 4.0.0 release of
the "actions/upload-artifact". Previously, a single artifact was used for the transfer of the builds for all hosts.
However, support for uploading multiple times to a single artifact was dropped in version 4.0.0 of the
"actions/upload-artifact" action. So it is now necessary to use a dedicated artifact for each of the builds. These are
downloaded in aggregate in a subsequent job by using the artifact name globbing and merging features which were
introduced in version 4.1.0 of the "actions/download-artifact" action.

A regression was introduced at that time. The chosen approach was to use a separate set of artifacts for the
non-notarized and notarized files. An overview of the sequence (the prefixes are the workflow job names):

1. create-release-artifacts/create-nightly-artifacts: Generate builds.
2. create-release-artifacts/create-nightly-artifacts: Upload builds to workflow artifacts
3. notarize-macos: Download workflow artifacts.
4. notarize-macos: Notarize macOS build from downloaded artifact.
5. notarize-macos: Upload notarized build to workflow artifact with a different name than the source artifact.
6. create-release/publish-nightly: Download workflow artifacts.
7. create-release/publish-nightly: Publish builds.

The problem with this is that the artifacts for the non-notarized (uploaded by the
create-release-artifacts/create-nightly-artifacts job) and notarized (created by the notarize-macos job) files are then
downloaded and merged by the create-release/publish-nightly job. Since each artifact contains a file with the same path
in the merged output, the contents of the last downloaded artifact overwrite the contents of the first. It happens that
the non-notarized artifact is downloaded after the notarized artifact, so this file path collision results in
non-notarized macOS builds being published instead of the notarized builds as intended, and as done by the workflow
prior to the regression:

```
% wget https://downloads.arduino.cc/arduino-cli/nightly/arduino-cli_nightly-latest_macOS_ARM64.tar.gz

[...]

% tar -xf arduino-cli_nightly-latest_macOS_ARM64.tar.gz

% spctl -a -vvv -t install arduino-cli
arduino-cli: rejected
```

```
% wget https://downloads.arduino.cc/arduino-cli/arduino-cli_latest_macOS_ARM64.tar.gz

[..]

% tar -xf arduino-cli_latest_macOS_ARM64.tar.gz

% spctl -a -vvv -t install arduino-cli
arduino-cli: rejected
```

The chosen solution is to delete the non-notarized artifacts after downloading each in the notarize-macos jobs. An
overview of the new sequence (the prefixes are the workflow job names):

1. create-release-artifacts/create-nightly-artifacts: Generate builds.
2. create-release-artifacts/create-nightly-artifacts: Upload builds to workflow artifacts
3. notarize-macos: Download macOS x86 or Apple Silicon workflow artifact.
4. notarize-macos: Delete macOS x86 or Apple Silicon workflow artifact.
5. notarize-macos: Notarize macOS build from downloaded artifact.
6. notarize-macos: Upload notarized build to workflow artifact.
7. create-release/publish-nightly: Download workflow artifacts.
8. create-release/publish-nightly: Publish builds.

The result is that there is no file path collision when the create-release/publish-nightly job downloads and merges the
artifacts.
  • Loading branch information
per1234 committed Oct 14, 2024
1 parent d2cd387 commit cb2d6fe
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 12 deletions.
18 changes: 12 additions & 6 deletions .github/workflows/publish-go-nightly-task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,11 @@ jobs:
strategy:
matrix:
artifact:
- name: darwin_amd64
- artifact-suffix: macOS_64bit
name: darwin_amd64
path: "macOS_64bit.tar.gz"
- name: darwin_arm64
- artifact-suffix: macOS_ARM64
name: darwin_arm64
path: "macOS_ARM64.tar.gz"

steps:
Expand All @@ -94,10 +96,14 @@ jobs:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
pattern: ${{ env.ARTIFACT_NAME }}-*
merge-multiple: true
name: ${{ env.ARTIFACT_NAME }}-${{ matrix.artifact.artifact-suffix }}
path: ${{ env.DIST_DIR }}

- name: Remove non-notarized artifact
uses: geekyeggo/delete-artifact@v5
with:
name: ${{ env.ARTIFACT_NAME }}-${{ matrix.artifact.artifact-suffix }}

- name: Import Code-Signing Certificates
env:
KEYCHAIN: "sign.keychain"
Expand Down Expand Up @@ -167,11 +173,11 @@ jobs:
-C ../../ LICENSE.txt
echo "PACKAGE_FILENAME=$PACKAGE_FILENAME" >> $GITHUB_ENV
- name: Upload artifact
- name: Upload notarized artifact
uses: actions/upload-artifact@v4
with:
if-no-files-found: error
name: ${{ env.ARTIFACT_NAME }}-notarized-${{ matrix.artifact.name }}
name: ${{ env.ARTIFACT_NAME }}-${{ matrix.artifact.artifact-suffix }}
path: ${{ env.DIST_DIR }}/${{ env.PACKAGE_FILENAME }}

create-windows-installer:
Expand Down
18 changes: 12 additions & 6 deletions .github/workflows/release-go-task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,11 @@ jobs:
strategy:
matrix:
artifact:
- name: darwin_amd64
- artifact-suffix: macOS_64bit
name: darwin_amd64
path: "macOS_64bit.tar.gz"
- name: darwin_arm64
- artifact-suffix: macOS_ARM64
name: darwin_arm64
path: "macOS_ARM64.tar.gz"

steps:
Expand All @@ -94,10 +96,14 @@ jobs:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
pattern: ${{ env.ARTIFACT_NAME }}-*
merge-multiple: true
name: ${{ env.ARTIFACT_NAME }}-${{ matrix.artifact.artifact-suffix }}
path: ${{ env.DIST_DIR }}

- name: Remove non-notarized artifact
uses: geekyeggo/delete-artifact@v5
with:
name: ${{ env.ARTIFACT_NAME }}-${{ matrix.artifact.artifact-suffix }}

- name: Import Code-Signing Certificates
env:
KEYCHAIN: "sign.keychain"
Expand Down Expand Up @@ -167,11 +173,11 @@ jobs:
-C ../../ LICENSE.txt
echo "PACKAGE_FILENAME=$PACKAGE_FILENAME" >> $GITHUB_ENV
- name: Upload artifact
- name: Upload notarized artifact
uses: actions/upload-artifact@v4
with:
if-no-files-found: error
name: ${{ env.ARTIFACT_NAME }}-notarized-${{ matrix.artifact.name }}
name: ${{ env.ARTIFACT_NAME }}-${{ matrix.artifact.artifact-suffix }}
path: ${{ env.DIST_DIR }}/${{ env.PACKAGE_FILENAME }}

create-windows-installer:
Expand Down

0 comments on commit cb2d6fe

Please sign in to comment.