Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github Actions Self-Hosted Runner issue with node 20 #3373

Open
biswalc opened this issue Jul 4, 2024 · 4 comments
Open

Github Actions Self-Hosted Runner issue with node 20 #3373

biswalc opened this issue Jul 4, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@biswalc
Copy link

biswalc commented Jul 4, 2024

Describe the bug

I am running Github Actions Self Hosted Runner on a Ubuntu VM:

~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"

I am running Actions Runner version 2.317.0

In the context of the notice by the Github team: Notice

I am using the GitHub Actions:

jobs:
  my-job:
      runs-on: [my-ubuntu]
      timeout-minutes: 120
      container:
        image: my-image-path:latest
        credentials:
          username: ${{ secrets.USERNAME }}
          password: ${{ secrets.PASSWORD }}

      steps:
        - name: Checkout Source Code
          uses: actions/checkout@v4.1.7

But when the job executes, I get this Error in the Job logs:

/usr/bin/docker exec  XXXX sh -c "cat /etc/*release | grep ^ID"
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)

On the Ubuntu VM, I have node installed already, and it is running as expected; not sure why the runner is complaining about not being able to run node 20

# Self-Hosted Runner Ubuntu 20.04
$ node -v
v20.15.0

Additional info:

$ ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.16) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

Currently to circumvent the downtime of CI, I am running the scripts as below:

bash -c 'ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true run.sh' > nohup.out 2>&1 &

## and github actions
      steps:
        - name: Checkout Source Code
          uses: actions/checkout@v3

To Reproduce
Provided in the description.

Expected behavior
Since node version v20.15.0 is already running on the ubuntu VM, the runner should not have issues with running with node20

Runner Version and Platform

Github runner version 2.317.0

~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"

OS of the machine running the runner?

NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"

What's not working?

Provided in description.

Job Log Output

Run actions/checkout@v4.1.7
/usr/bin/docker exec  7ac705625cb4517179f0c29eb36101b7221b70f40f0d76ede8fcbdad305a8012 sh -c "cat /etc/*release | grep ^ID"
/__e/node[20](https://github.com/Roche-DIA-RDS-CSI/bio-nlp-py/actions/runs/9799219159/job/27061796743#step:3:21)/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)

Runner and Worker's Diagnostic Logs

None

@biswalc biswalc added the bug Something isn't working label Jul 4, 2024
@hardikns
Copy link

hardikns commented Jul 5, 2024

Same issue here with self-hosted runner. I am still running checkout@v2 and still its failing for node20.

is it something to do with https://github.com/actions/runner/releases/tag/v2.317.0?

@Bercon
Copy link

Bercon commented Jul 5, 2024

Dropping support for official ubuntu:18.04 docker image is unfortunate and https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ only lists ubuntu:16.04 as unsupported

Are there plans to fix the ubuntu:18.04 support?

@eli-schwartz
Copy link

eli-schwartz commented Jul 5, 2024

Since node-based actions spread into containers, it seems very unwise to allow the node interpreter to rely on software inside the container. That appears to be the root of this mismatch. A container may contain older glibc which breaks things as basic as actions/checkout, which can leave users pretty trapped.

Node should instead be:

  • built statically, so it doesn't care about the libc inside the container.
  • Alternatively, make sure to build it in a special build environment specifically for node, which uses a very old glibc version so that it is compatible with a wide range of glibc versions, and document the minimum glibc requirement for using container:

loadams added a commit to microsoft/DeepSpeed that referenced this issue Jul 8, 2024
With changes from GitHub [finally
deprecating](actions/checkout#1474) [node16
based
runners](https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/)
(which the checkout@v3 action uses) we need to make changes to support
this.

To do this, there are two changes. First we remove the python 3.6 check
as with the changes in pydantic v2 that will be merged soon, we will be
removing this check there, so we can more easily remove it now so that
future PRs are cleaner and it is clear why some changes have been made.

Second, node16 is the default on some of our self-hosted runners. To
work around tests failing on these, we [set the GitHub env var to
override this
check](https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/).

Other relevant links:
actions/checkout#1474
https://github.com/easybuilders/easybuild-framework/pull/4574/files
actions/checkout#1809
actions/runner#3373
actions/checkout#1809
mauryaavinash95 pushed a commit to DataStates/DeepSpeed that referenced this issue Jul 10, 2024
…rosoft#5756)

With changes from GitHub [finally
deprecating](actions/checkout#1474) [node16
based
runners](https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/)
(which the checkout@v3 action uses) we need to make changes to support
this.

To do this, there are two changes. First we remove the python 3.6 check
as with the changes in pydantic v2 that will be merged soon, we will be
removing this check there, so we can more easily remove it now so that
future PRs are cleaner and it is clear why some changes have been made.

Second, node16 is the default on some of our self-hosted runners. To
work around tests failing on these, we [set the GitHub env var to
override this
check](https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/).

Other relevant links:
actions/checkout#1474
https://github.com/easybuilders/easybuild-framework/pull/4574/files
actions/checkout#1809
actions/runner#3373
actions/checkout#1809
@eli-schwartz
Copy link

It appears this has actually been proposed in #3128 back in February.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants