-
-
Notifications
You must be signed in to change notification settings - Fork 623
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump workflow actions for node20 #21133
Conversation
…-java, actions/setup-node, and actions/setup-python.
This is a potentially breaking change. See https://github.com/actions/upload-artifact/blob/main/docs/MIGRATION.md
Hmm.. so, this is still a good change, but it doesn't address the underlying issue, which seems to be node20 being broken on the linux runners we use. Release run to build wheels using this workflow: https://github.com/pantsbuild/pants/actions/runs/9817274677/job/27108461020 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The actual change here looks reasonable... although I have not checked each action for breaking changes other than the upload-artifact
one I already knew about.
@@ -19,6 +19,23 @@ | |||
|
|||
from pants.util.strutil import softwrap | |||
|
|||
ACTION = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice idea to centralise these versions 👍
"setup-protoc": "arduino/setup-protoc@9b1ee5b22b0a3f1feb8c2ff99b32c89b3c3191e9", | ||
"setup-python": "actions/setup-python@v5", | ||
"slack-github-action": "slackapi/slack-github-action@v1.24.0", | ||
"upload-artifact": "actions/upload-artifact@v4", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At $WORK, we were tripped up by one of the breaking changes in this upgrade: https://github.com/actions/upload-artifact?tab=readme-ov-file#breaking-changes
Uploading to the same named Artifact multiple times.
Due to how Artifacts are created in this new version, it is no longer possible to upload to the same named Artifact multiple times. You must either split the uploads into multiple Artifacts with different names, or only upload once. Otherwise you will encounter an error.
Does Pants encounter this?
Maybe that is this why there's the overwrite: true
added in upload_log_artifacts
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yea, that was my attempt at handling that, suspecting that we want the "latest" logs always. Other than that 🤷🏽 so figured I run some workflows manually using this version, once we're passed the other CI issues (the failing build wheels jobs).
Argh, looks like we might end up somewhat jammed by platform support.
pants/src/python/pants_release/generate_github_workflows.py Lines 854 to 865 in 9d77334
Thus, since 2.17 < 2.23, we're stuck... An idea we might need to separate the "build orchestration" (and all the various actions) and the actual builds:
I'm not sure the best way to actually do this. |
Want to hold off merging until we've tested these workflows properly, as there are breaking changes in at least the |
Can we run in a new container than manylinux2014 ? |
Oh, and why do we care about node versions? Is this our own JS backend, or related to various actions being written in JS? |
At the cost of reducing platform support. I believe Pants will currently runs on Linux platforms with glibc 2.17 - 2.22, but may not if we bump the image? |
Github Actions: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ |
And we're running actions inside the container? I guess we need the action to run natively but build in a container? |
Well, the linux wheel build jobs specify a container image, so I guess the entire job is executed in it, and thus the GH actions to clone the repo sources: build_wheels_linux_x86_64:
container:
image: quay.io/pypa/manylinux2014_x86_64:latest
...
build_wheels_linux_arm64:
container:
image: ghcr.io/pantsbuild/wheel_build_aarch64:v3-8384c5cf
... |
Right, but we can't continue to do that, so I guess the fix has to be something like "run the build in docker inside a native GHA job" |
Maybe |
Possibly. The actual "build wheels" step is: pants/src/python/pants_release/release.py Lines 475 to 487 in 7b2dcad
... which invokes:
The |
So, But wont also the native_client need to be built in the container? That's not "inside pants" atm, meaning even using |
Ah, good catch. Maybe it's best to do some variant of |
Yes, the wheel is not the main problem, the engine is |
seems practical if all we're looking for is an unblock?
Edit: Doesn't seem to have any effect when applied ontop of the changes herein, maybe newer JS actions do not honor it? Edit2: Indeed, checkout@v4 unconditionally requires node20. |
Oooh yes, I would take an unblock right now. |
My attempt at the temporary fix: #21172 @benjyw Edit: It passed with some warnings, but none of them seem critical. |
Merging this and then #21172 |
Renders github actions and env settings to continue to use node16 inside the containers that match the glibc (2.17) of manylinux_2014 builder images. The new default node version for gha requires a much higher version. I checked if https://github.com/nodejs/unofficial-builds had node binaries we could use. It does not. Based on #21133
We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809
…sbuild#21172) Renders github actions and env settings to continue to use node16 inside the containers that match the glibc (2.17) of manylinux_2014 builder images. The new default node version for gha requires a much higher version. I checked if https://github.com/nodejs/unofficial-builds had node binaries we could use. It does not. Based on pantsbuild#21133
We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809
I tried to automatically cherry-pick this change back to each relevant milestone, so that it is available in those older releases of Pants. ❌ 2.20.xI was unable to cherry-pick this PR to 2.20.x, likely due to merge-conflicts. Steps to Cherry-Pick locallyTo resolve:
Please note that I cannot re-run CI if a job fails. Please work with your PR approver(s) to re-run CI if necessary. ❌ 2.21.xI was unable to cherry-pick this PR to 2.21.x, likely due to merge-conflicts. Steps to Cherry-Pick locallyTo resolve:
Please note that I cannot re-run CI if a job fails. Please work with your PR approver(s) to re-run CI if necessary. ✔️ 2.22.xSuccessfully opened #21198. When you're done manually cherry-picking, please remove the Thanks again for your contributions! |
We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809 Co-authored-by: Andreas Stenius <git@astekk.se>
We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809
We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809
We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809 Co-authored-by: Andreas Stenius <git@astekk.se>
We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809 Co-authored-by: Andreas Stenius <git@astekk.se>
Renders github actions and env settings to continue to use node16 inside the containers that match the glibc (2.17) of manylinux_2014 builder images. The new default node version for gha requires a much higher version. I checked if https://github.com/nodejs/unofficial-builds had node binaries we could use. It does not. Based on #21133
…ry-pick of #21172) (#21209) Renders github actions and env settings to continue to use node16 inside the containers that match the glibc (2.17) of manylinux_2014 builder images. The new default node version for gha requires a much higher version. I checked if https://github.com/nodejs/unofficial-builds had node binaries we could use. It does not. Based on #21133 Co-authored-by: Tobias Nilsson <tobias.jens.nilsson@gmail.com>
Renders github actions and env settings to continue to use node16 inside the containers that match the glibc (2.17) of manylinux_2014 builder images. The new default node version for gha requires a much higher version. I checked if https://github.com/nodejs/unofficial-builds had node binaries we could use. It does not. Based on #21133
Renders github actions and env settings to continue to use node16 inside the containers that match the glibc (2.17) of manylinux_2014 builder images. The new default node version for gha requires a much higher version. I checked if https://github.com/nodejs/unofficial-builds had node binaries we could use. It does not. Based on #21133
…ry-pick of #21172) (#21211) Renders github actions and env settings to continue to use node16 inside the containers that match the glibc (2.17) of manylinux_2014 builder images. The new default node version for gha requires a much higher version. I checked if https://github.com/nodejs/unofficial-builds had node binaries we could use. It does not. Based on #21133 Co-authored-by: Tobias Nilsson <tobias.jens.nilsson@gmail.com>
…ry-pick of #21172) (#21212) Renders github actions and env settings to continue to use node16 inside the containers that match the glibc (2.17) of manylinux_2014 builder images. The new default node version for gha requires a much higher version. I checked if https://github.com/nodejs/unofficial-builds had node binaries we could use. It does not. Based on #21133 Co-authored-by: Tobias Nilsson <tobias.jens.nilsson@gmail.com>
We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809