Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only upload builder artifacts from workflow runs triggered after merge #18

Merged
merged 43 commits into from
Feb 19, 2024

Conversation

James-Mart
Copy link
Member

@James-Mart James-Mart commented Jan 12, 2024

Changes

  • Artifacts generated during PRs will no longer be uploaded to ghcr.io, and will instead only be uploaded as local artifacts to the github action.
  • Workflows are significantly more intelligent now. For example, a workflow has the logic to know whether it needs to download one of its dependency images from a local github action artifact, or whether it should pull from ghcr.io. And if it pulls from ghcr.io, it knows how to look up the latest image (not using a latest tag).
  • The builder workflows were combined into one, parameterized by the desired base version of ubuntu.
  • A single dispatcher workflow is the only thing automatically triggered on PR or merge, and it is responsible for determining what other workflows to trigger (and how to parameterize them) based on the changed files.

Notes on centralizing dispatching

There were lots of complexities trying to manage each of the separate workflows and trigger them at the correct times. For example, if files were changes that would cause the psibase_contributor image to be rebuilt, then I should trigger that workflow. But I should not trigger it if files were also changed that would trigger a rebuild of the 2204 builder image, since that is a dependency of the psibase_contributor and therefore psibase_contributor would need to run afterwards. Building such logic like this into a workflow file using normal path exclusion doesn't work the way I expected, and I decided that the most straightforward way to accomplish our complicated build strategies here is to simply extract all the logic into a central _dispatcher.yml workflow that figures everything out and dispatches the workflows appropriately.

@James-Mart James-Mart marked this pull request as draft January 15, 2024 16:32
@James-Mart James-Mart requested a review from swatanabe February 12, 2024 20:51
@James-Mart James-Mart marked this pull request as ready for review February 12, 2024 20:56
.github/scripts/check-patterns.sh Outdated Show resolved Hide resolved
.github/workflows/_dispatcher.yml Show resolved Hide resolved
.github/workflows/_dispatcher.yml Outdated Show resolved Hide resolved
.github/workflows/_dispatcher.yml Outdated Show resolved Hide resolved
.github/workflows/_dispatcher.yml Outdated Show resolved Hide resolved
.github/scripts/free-disk-space.sh Outdated Show resolved Hide resolved
@James-Mart James-Mart force-pushed the fix-workflow-runs branch 5 times, most recently from 6d86eb1 to 2fce398 Compare February 15, 2024 17:30
@James-Mart James-Mart marked this pull request as draft February 15, 2024 19:04
@James-Mart James-Mart force-pushed the fix-workflow-runs branch 2 times, most recently from b1e499a to 2f875a1 Compare February 15, 2024 20:45
@James-Mart James-Mart marked this pull request as ready for review February 16, 2024 16:43
@James-Mart James-Mart changed the title Fix workflow runs Only upload builder artifacts from workflow runs triggered after merge Feb 16, 2024
@James-Mart James-Mart merged commit 0adb578 into main Feb 19, 2024
6 checks passed
@swatanabe swatanabe deleted the fix-workflow-runs branch July 12, 2024 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants