-
-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build Docker image and push to GHCR #230
base: unstable/v1
Are you sure you want to change the base?
Conversation
This looks.. intriguing! I don't remember if I ever considered combining composite+docker actions (I did play with having two composites in the same repo in the past, though). I'll need to take some time to think about it and look through the patch more closely. Please, don't expect an immediate review, however it does look very promising at glance! Originally I thought that I'd have a workflow where I trigger a release, that release adds a commit that hardcodes an update to This looks like a better idea so far. Thanks again! |
That sounds great. Take your time. Thanks for your consideration. If you do decide to accept this change, I'm happy to help maintain the workflows in the future. Feel free to mention me @br3ndonland and I will help address any issues that come up. |
Thanks! I've hit "rebase" on the UI to get this on top of the recent changes/linting/lockfile bumps but haven't yet looked into it deeper. |
pull_request: | ||
workflow_run: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two separate workflow runs is often hard to track. Instead, I adopted a practice of modularizing the workflow pieces as reusable workflows having the reusable-
prefix in their names. This allows embedding everything in all the right places. Let's try this, WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will return to this suggestion at a later time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should be able to circle back to this already...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two separate workflow runs is often hard to track. Instead, I adopted a practice of modularizing the workflow pieces as reusable workflows having the
reusable-
prefix in their names. This allows embedding everything in all the right places. Let's try this, WDYT?
@webknjaz I removed workflow_call
and switched to a reusable workflow instead.
Note that the required status checks for the repo will need to be updated.
steps: | ||
- name: Reset path if needed | ||
run: | | ||
# Reset path if needed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why would this be needed outside the container?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because you've set up a test that modifies the $PATH
(#112, 1350b8b).
gh-action-pypi-publish/.github/workflows/self-smoke-test-action.yml
Lines 85 to 90 in 699cd61
- name: ✅ Smoke-test the locally checked out action | |
uses: ./test | |
env: | |
DEBUG: >- | |
true | |
PATH: utter-nonsense |
I'm done with the initial review. More is needed, but I'd rather accept what I can through separate PRs to make this one smaller. And the suggested refactoring could be done in parallel. I think that generating the file is a good idea. It should be possible to write the file without bringing in the PyYAML dependency. But it's not that easy for reading it. Can we make use of |
@webknjaz thank you for your detailed review. I've addressed most of your comments so far. |
Commits like 213c885ac41d769527ac150e2e633bb1ccd886d4 aren't really necessary since Git would automatically absorb changes applied separately. FYI. In fact, this one may have a harmful effect — when the label change is merged into the default branch, and this one rebased on top, Git will keep the removal commit and attempt deleting the label 🤷♂️ Anyway, we'll address this later on. |
@br3ndonland I'm sure my review is incomplete, but hopefully it gives you enough ideas to try out until the next time. I think it'd be nice to get this in before #236 if at all possible. |
@br3ndonland could you also rebase this branch locally? The GH button doesn't work, meaning there's going to be some conflicts to resolve. |
if: runner.os != 'Linux' | ||
run: | | ||
>&2 echo This action is only able to run under GNU/Linux environments | ||
exit 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add CI jobs testing that this fails under corresponding envs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added CI jobs that test failure on macOS and Windows.
tag: | ||
description: Docker image tag |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we incorporate the prior art from the discussions in #45 and accept the 3-segment version as an input, then deduce everything else from that + push the Git tag and advance the proper branches? Perhaps, this could also happen before this PR so it exists as a separate base.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we incorporate the prior art from the discussions in #45 and accept the 3-segment version as an input, then deduce everything else from that + push the Git tag and advance the proper branches? Perhaps, this could also happen before this PR so it exists as a separate base.
I don't understand what you're asking here. You want separate inputs for major, minor, and patch versions? What if you're building the Docker image from the release/v1
branch? What is the "3-segment version" in that case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@br3ndonland I was talking about the workflow inputs. Everything else should be computed from them.
The current workflow for me is as follows:
- Tag
v1.10.1
on top ofunstable/v1
- Fast-forward
release/v1
tov1.10.1
- Fast-forward
release/v1.10
tov1.10.1
(or create it) git push --atomic
everything in one go- Create a GitHub Release from the UI
With this, my 3-segment “input” is 1.10.1
and v1
+v1.10
are being extracted from that. Do you think we could bake containers into this flow with not many changes? This will involve tagging the same image with multiple tags, as I understand. And the end-users referencing versions by both tags and branches should be able to retrieve whatever container there is, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@webknjaz thanks for explaining that. I think we're actually pretty close to the desired workflow. There's just a small update needed to the Docker build workflow so it pushes tags with the major and minor version numbers. I've pushed that change.
Your updated workflow would look like this:
- Tag
v1.10.1
on top ofunstable/v1
- Fast-forward
release/v1
tov1.10.1
- Fast-forward
release/v1.10
tov1.10.1
(or create it) git push --atomic
everything in one go- Trigger the 🏗️ workflow from the Git tag. The workflow will push the Docker image with three Docker tags:
ghcr.io/pypa/gh-action-pypi-publish:v1.10.1
ghcr.io/pypa/gh-action-pypi-publish:v1.10
ghcr.io/pypa/gh-action-pypi-publish:v1
- Create a GitHub Release from the UI
pypa#230 (comment) Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
This project uses pypa/gh-action-pypi-publish to publish Python packages to PyPI with an OIDC trusted publisher (6e532c6). pypa/gh-action-pypi-publish is set up as a Docker action referencing the Dockerfile. The downside to using the Dockerfile for the action is that the Docker image must be built every time the action is used. This will hopefully change in the near future if Docker images are pre-built and pushed to a registry (pypa/gh-action-pypi-publish#230). In the meantime, this commit will move related steps to a dedicated GitHub Actions job so that the Docker image is not built every time GitHub Actions jobs run. 6e532c6 https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action https://docs.pypi.org/trusted-publishers/ https://github.com/pypa/gh-action-pypi-publish
This project uses pypa/gh-action-pypi-publish to publish Python packages to PyPI with an OIDC trusted publisher (59ec546). pypa/gh-action-pypi-publish is set up as a Docker action referencing the Dockerfile. The downside to using the Dockerfile for the action is that the Docker image must be built every time the action is used. This will hopefully change in the near future if Docker images are pre-built and pushed to a registry (pypa/gh-action-pypi-publish#230). In the meantime, this commit will move related steps to a dedicated GitHub Actions job so that the Docker image is not built every time GitHub Actions jobs run. 59ec546 https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action https://docs.pypi.org/trusted-publishers/ https://github.com/pypa/gh-action-pypi-publish
@br3ndonland this also needs to be rebased now that the attestations PR is in. |
Up to this point, the project has been set up as a Docker action referencing the Dockerfile. The downside to using the Dockerfile for the action is that the Docker image must be built every time the action is used. This commit will set up the project to build the Docker image and push it to GitHub Container Registry (GHCR). This change will speed up user workflows every time the action is used because the workflows will simply pull the Docker image from GHCR instead of building again. Changes: - Add required metadata to Dockerfile - Build container image with GitHub Actions - Push container image to GHCR Docker actions support pulling in pre-built Docker images. The downside is that there's no way to specify the correct Docker tag because the GitHub Actions `image` and `uses:` keys don't accept any context. For example, if a user's workflow has `uses: pypa/gh-action-pypi-publish@release/v1.8`, then the action should pull in a Docker image built from the `release/v1.8` branch, something like `ghcr.io/pypa/gh-action-pypi-publish:release-v1.8` (Docker tags can't have `/`). The workaround is to switch the top-level `action.yml` to a composite action that then calls the Docker action, substituting the correct image name and tag.
pypa#230 (comment) Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Description
Closes #58
Up to this point, the project has been set up as a Docker action referencing the Dockerfile.
gh-action-pypi-publish/action.yml
Lines 86 to 88 in 3fbcf7c
The downside to using the Dockerfile for the action is that the Docker image must be built every time the action is used (#58).
This PR will set up the project to build the Docker image and push it to GitHub Container Registry (GHCR). This change will speed up user workflows every time the action is used because the workflows will simply pull the Docker image from GHCR instead of building again.
Changes
Build container image with GitHub Actions
This PR will build Docker images with the Docker CLI (
docker build
). Builds will include inline cache metadata so layers can be reused by future builds.This PR only proposes to build container images for
x86_64
(linux/amd64
) because GitHub Actions Linux runners currently only supportx86_64
CPU architectures (actions/runner-images#5631), and this project only supports GitHub Actions Linux runners. The README explains:Push container image to GHCR
The workflow will log in to GHCR using the built-in GitHub token and push the Docker image. Workflow runs triggered by pull requests will build the Docker image and run the smoke tests but will not push the Docker image.
Update action to pull container image from GHCR
Docker actions support pulling in pre-built Docker images by supplying a registry address to the
image:
key. The downside to this syntax is that there's no way to specify the correct Docker tag because the GitHub Actionsimage:
anduses:
keys don't accept any context. For example, if a user's workflow hasuses: pypa/gh-action-pypi-publish@release/v1.8
, then the action should pull in a Docker image built from therelease/v1.8
ref, something likeghcr.io/pypa/gh-action-pypi-publish:release-v1.8
(Docker tags can't have/
).The workaround is to switch the top-level
action.yml
to a composite action that then calls the Docker action, substituting the correct image name and tag.Related
github.action_repository
andgithub.action_ref
are empty inrun
for composite actions actions/runner#2473