checkout@v3 using node20 instead of node16 #1809

jcooke-impinj · 2024-07-03T20:12:44Z

Running on rhel7 with self-hosted runners and starting today we're running into issues where @V3 is using node20 instead of node16.
Happening on runner versions 2.316.1 and 2.317.0.
Oddly some of our Repos don't run into this issue when executing checkout@v3 on the same runners.

Run actions/checkout@v3
/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)
/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version CXXABI_1.3.9' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)
/ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: version GLIBC_2.28' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node) /ci_storage_agent_1/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: version GLIBC_2.25' not found (required by /ci_storage_agent_1/actions-runner/externals/node20/bin/node)

The text was updated successfully, but these errors were encountered:

jcooke-impinj · 2024-07-03T21:10:42Z

Our team found that setting these before starting the runners get's things going again. Still unsure what's causing this.

export ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION=node16
export ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION=node16

adpaco-aws · 2024-07-03T21:12:20Z

This doesn't seem to be specific to actions/checkout@v3 since we tried to upgrade to actions/checkout@v4 (here) and got the same failure (here).

biswalc · 2024-07-04T00:18:33Z

I am having the same issue while running below versions, on ubuntu 20.04 VM:

actions/checkout@v3.6.0
actions/checkout@v3
actions/checkout@v4
actions/checkout@v4.1.7

Error:

/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)

Issue started today, i.e. 3 Jul 10am PDT.

junghyun-apr · 2024-07-04T02:22:20Z

Hello! Our development team is also experiencing the same issue. We are using AWS instances as self-hosted.
We temporarily resolved the issue by simply changing the node16 directory to node20 in the same path. 🥲

AWS AMI :

amzn2-ami-kernel-5.10-hvm-2.0.20240223.0-x86_64-gp2

Git Action Error :

/home/ec2-user/actions-runner/externals/node20/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /home/ec2-user/actions-runner/externals/node20/bin/node)
/home/ec2-user/actions-runner/externals/node20/bin/node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /home/ec2-user/actions-runner/externals/node20/bin/node)

GuyAv46 · 2024-07-04T06:23:33Z

Following this blog post: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Adding this to the envs that require node16 fixed the issue

ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true

Flamefire · 2024-07-04T07:03:21Z

Our team found that setting these before starting the runners get's things going again. Still unsure what's causing this.

export ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION=node16
export ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION=node16

I tried setting this inside a workflow running on GH hosted runners and it didn't work.
So it looks like this only works when the runner itself was started with those set, i.e. you cannot set them for a running runner which includes the GH hosted ones.

This doesn't seem to be specific to actions/checkout@v3 since we tried to upgrade to actions/checkout@v4 (here) and got the same failure (here).

v4 always used Node 20

I am having the same issue while running below versions, on ubuntu 20.04 VM:
actions/checkout@v3.6.0
actions/checkout@v3
actions/checkout@v4
actions/checkout@v4.1.7

See above

david-batranu · 2024-07-04T07:52:11Z

Fixed by renaming the node20 folder and symlinking node16 to node20:

cd $RUNNER_HOME/actions-runner/externals
mv node20 node20.bak
ln -s node16 node20

eshogo · 2024-07-04T08:01:05Z

I tried modifying /home/runner/actions-runner/.env and env.sh and it works fine.

% cat .env
ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION=node16
ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION=node16

and adding their environment variables into 'varCheckList' array in env.sh.
(not needed?)

varCheckList=(
    'LANG'
    'JAVA_HOME'
    'ANT_HOME'
    'M2_HOME'
    'ANDROID_HOME'
    'ANDROID_SDK_ROOT'
    'GRADLE_HOME'
    'NVM_BIN'
    'NVM_PATH'
    'LD_LIBRARY_PATH'
    'PERL5LIB'
    'ACTIONS_RUNNER_FORCED_INTERNAL_NODE_VERSION'
    'ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION'
    )

Then restart runner service

% systemctl restart github-actions-runner.service

david-batranu · 2024-07-04T08:30:19Z

Looks like this is happening because of: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/

actions/checkout#1590 actions/checkout#1809

n1ngu · 2024-07-04T14:29:38Z

Following this blog post: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/ Adding this to the envs that require node16 fixed the issue
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true

Quoting the post paragraph about that workaround: "This will only work until we upgrade the runner removing Node16 later in the spring"

We are walking a tightrope.

Just wanted to raise some awareness about that. Plan to actually upgrade your github actions in the short term or move away from them. E.g. I think I'll quit my job. 😄

biswalc · 2024-07-04T19:05:58Z

I am having the same issue while running below versions, on ubuntu 20.04 VM:
actions/checkout@v3.6.0
actions/checkout@v3
actions/checkout@v4
actions/checkout@v4.1.7
Error:
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
Issue started today, i.e. 3 Jul 10am PDT.

My glibc version:

ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.16) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

So why do I get this error? I am using checkout@v4.1.7. I do not understand what I have to do to upgrade.

ayizhi · 2024-07-05T09:11:43Z

Here's a critical issue: Firstly, you shouldn't upgrade such fundamental environments arbitrarily. Even if you do upgrade, you must ensure the environment is correct. Upgrading to Node 20 and causing a glib error is absolutely unacceptable, especially in a product from a company like Microsoft.

FelixSchwarz · 2024-07-05T09:55:32Z

I wish there was an alternative version of the checkout action, maybe written in rust so we could just deploy a static binary. Then we would just compile that against a really old version of glibc. This would also simplify running GitHub Actions with nektos/act which does not map node into the container.

Update: I found checkout-action in GitHub's market place and added support for private repos. Not sure if that is a permanent solution but at least our CI jobs are passing again...

hasufell · 2024-07-05T10:42:19Z

This means you now can't run any docker image that has a glibc older than 2.27. That includes:

debian:9
fedora:27
ubuntu:18.04

What a huge f*ckup. I'm baffled by the amount of incompetence. What's wrong with providing a statically linked node binary? Do you need help with redistribution github?

See actions/checkout#1809

actions/checkout#1809

matthinea · 2024-07-10T00:43:51Z

@Kobzol thank you for that link, your setup is the only thing that worked for me.

daiyam · 2024-07-10T08:27:39Z

in my workflow: https://github.com/mmomtchev/hadron/blob/main/.github/workflows/os_comp.yml#L32-L33

It's working fine (only ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION needed)

Adds an option to deal with the current glibc errors when running the tests. Issue was reported actions/checkout#1809 and solution comes from actions/checkout#1809 (comment)

Now adding it to the correct place. (issue actions/checkout#1809)

Adds an option to deal with the current glibc errors when running the tests. Issue was reported actions/checkout#1809 and solution comes from actions/checkout#1809 (comment)

Bypasses unsupported node versions check Node 16 is required to build linux binaries compatible with old distros (see #449) relates-to: actions/checkout#1809

…rosoft#5756) With changes from GitHub [finally deprecating](actions/checkout#1474) [node16 based runners](https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/) (which the checkout@v3 action uses) we need to make changes to support this. To do this, there are two changes. First we remove the python 3.6 check as with the changes in pydantic v2 that will be merged soon, we will be removing this check there, so we can more easily remove it now so that future PRs are cleaner and it is clear why some changes have been made. Second, node16 is the default on some of our self-hosted runners. To work around tests failing on these, we [set the GitHub env var to override this check](https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/). Other relevant links: actions/checkout#1474 https://github.com/easybuilders/easybuild-framework/pull/4574/files actions/checkout#1809 actions/runner#3373 actions/checkout#1809

- Ref: actions/checkout#1809 (comment)

friedrichroell · 2024-07-11T06:06:18Z

has anyone tried to run the runner inside a docker container (which is easier to run node20 with)?

Adds an option to deal with the current glibc errors when running the tests. Issue was reported actions/checkout#1809 and solution comes from actions/checkout#1809 (comment)

Kobzol · 2024-07-11T06:14:50Z

I'm (and I suspect others are) running the checkout action inside a Docker container that is specifically using an old glibc version. The problem is not how to use Node 20, but rather how not to use it.

friedrichroell · 2024-07-11T07:36:27Z

its the other way around for me, i have an old system that i cant update, but my actions are running directly on the host. could you point me to a guide or something for running the actions on a docker container? 🙏
Cheers!

Kobzol · 2024-07-11T07:37:46Z

https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container

friedrichroell · 2024-07-11T08:25:17Z

i meant running the self hosted runner itself in a container

- Ref: actions/checkout#1809 (comment)

The glibc version on ubuntu1804 and centos7 is lower than the node20 required, so we need to force the node version to 16 when running CI. See more details: actions/checkout#1809

We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809

sawwn23 · 2024-07-16T08:11:06Z

Fixed by renaming the node20 folder and symlinking node16 to node20:
cd $RUNNER_HOME/actions-runner/externals
mv node20 node20.bak
ln -s node16 node20

Thanks for your workaround solution

jcooke-impinj changed the title ~~checkout@v3 using node20~~ checkout@v3 using node20 instead of node16 Jul 3, 2024

tautschnig mentioned this issue Jul 3, 2024

Auto-upload macOS ARM binaries to draft release model-checking/kani#3320

Merged

artemagvanian mentioned this issue Jul 3, 2024

Improve performance and language support of memory initialization checks model-checking/kani#3313

Merged

adpaco-aws mentioned this issue Jul 3, 2024

BuildBundle-Linux CI jobs are failing with version GLIBC_2.28 not found model-checking/kani#3321

Closed

thresheek mentioned this issue Jul 4, 2024

njs-related changes nginx/ci-self-hosted#2

Merged

GuyAv46 mentioned this issue Jul 4, 2024

GHA better node16/20 handle RediSearch/RediSearch#4843

Merged

2 tasks

zhulinJulia24 mentioned this issue Jul 4, 2024

/lib64/libm.so.6: version 'GLIBC_2.27' not found in my self-hosted runners (Amazon Linux 2) dawidd6/action-download-artifact#262

Closed

This was referenced Jul 4, 2024

stop using CentOS 7.9 container in CI, no longer works now GitHub Actions is using node.js 20 easybuilders/easybuild-framework#4573

Closed

CI: Allow using Node 16 actions easybuilders/easybuild-framework#4574

Merged

firewave mentioned this issue Jul 4, 2024

Workaround for docker CI failure danmar/cppcheck#6572

Merged

firewave pushed a commit to danmar/cppcheck that referenced this issue Jul 4, 2024

Workaround for docker CI failure (#6572)

ec8e27e

actions/checkout#1590 actions/checkout#1809

dalboris mentioned this issue Jul 4, 2024

Linux build fails with: version `GLIBC_2.28' not found required by /__e/node20/bin/node vgc/vgc#1804

Closed

endgame mentioned this issue Jul 4, 2024

GHA: Checkouts are failing due to node too new for glibc haskell-CI/haskell-ci#738

Closed

timbrunette mentioned this issue Jul 5, 2024

"ERR_INVALID_THIS" on "pnpm up" in Node 20 pnpm/pnpm#6424

Closed

hasufell added a commit to stable-haskell/haskell-language-server that referenced this issue Jul 5, 2024

Remove distro support for things that need glibc <2.27

947ef54

See actions/checkout#1809

hasufell added a commit to haskell/ghcup-metadata that referenced this issue Jul 5, 2024

Fix bindist action after GitHub broke half of our CI

cf07b8f

actions/checkout#1809

badumbatish mentioned this issue Jul 10, 2024

Pin node16 by allowing old version Rust-GCC/gccrs#3081

Merged

Claudio-Chies mentioned this issue Jul 10, 2024

GH Actions - MAVROS fix PX4/PX4-Autopilot#23377

Merged

0x400080 added a commit to duckduckgo/duckduckgo-locales that referenced this issue Jul 10, 2024

Update run-tests.yml

cf23e44

Now adding it to the correct place. (issue actions/checkout#1809)

cwperks mentioned this issue Jul 10, 2024

Fix checkout action failure opensearch-project/job-scheduler#650

Merged

6 tasks

0x400080 mentioned this issue Jul 10, 2024

Update run-tests.yml duckduckgo/duckduckgo-locales#718

Merged

denis-maiorov-brightsec mentioned this issue Jul 10, 2024

fix: allow node 16 in actions to build linux binary NeuraLegion/bright-cli#558

Merged

chchwy added a commit to chchwy/pencil2d that referenced this issue Jul 11, 2024

Fix linux the git checkout failure in GitHub Actions

8abcbb4

- Ref: actions/checkout#1809 (comment)

chchwy added a commit to chchwy/pencil2d that referenced this issue Jul 11, 2024

Fix linux-Qt5 git checkout failure in GitHub Actions

85d3f2d

- Ref: actions/checkout#1809 (comment)

chchwy added a commit to pencil2d/pencil that referenced this issue Jul 11, 2024

Fix linux-Qt5 git checkout failure in GitHub Actions

ab2e202

- Ref: actions/checkout#1809 (comment)

ysangkok mentioned this issue Jul 11, 2024

Provide motivation and clarity on Ubuntu version selection haskell-CI/haskell-ci#741

Open

Castaglia mentioned this issue Jul 11, 2024

Don't include <openssl/engine.h> if ENGINE support not detected proftpd/proftpd#1816

Merged

acelyc111 mentioned this issue Jul 12, 2024

chore(CI): Force CI to run on node16 apache/incubator-pegasus#2070

Merged

ggtakec mentioned this issue Jul 12, 2024

1.95 release s3fs-fuse/s3fs-fuse#2496

Open

anth-volk mentioned this issue Jul 14, 2024

Fix GitHub Actions to allow earlier glibc versions PolicyEngine/policyengine-household-api#146

Closed

tobni pushed a commit to pantsbuild/pants that referenced this issue Jul 15, 2024

Bump workflow actions for node20 (#21133)

9a70a83

We started to run into issues with node16 no longer being functional on the runner, with symptoms like actions/checkout#1809

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

checkout@v3 using node20 instead of node16 #1809

checkout@v3 using node20 instead of node16 #1809

jcooke-impinj commented Jul 3, 2024

jcooke-impinj commented Jul 3, 2024

adpaco-aws commented Jul 3, 2024

biswalc commented Jul 4, 2024 •

edited

Loading

junghyun-apr commented Jul 4, 2024

GuyAv46 commented Jul 4, 2024

Flamefire commented Jul 4, 2024 •

edited

Loading

david-batranu commented Jul 4, 2024

eshogo commented Jul 4, 2024 •

edited

Loading

david-batranu commented Jul 4, 2024

n1ngu commented Jul 4, 2024 •

edited

Loading

biswalc commented Jul 4, 2024

ayizhi commented Jul 5, 2024

FelixSchwarz commented Jul 5, 2024 •

edited

Loading

hasufell commented Jul 5, 2024

matthinea commented Jul 10, 2024

daiyam commented Jul 10, 2024

friedrichroell commented Jul 11, 2024

Kobzol commented Jul 11, 2024

friedrichroell commented Jul 11, 2024

Kobzol commented Jul 11, 2024

friedrichroell commented Jul 11, 2024

sawwn23 commented Jul 16, 2024

checkout@v3 using node20 instead of node16 #1809

checkout@v3 using node20 instead of node16 #1809

Comments

jcooke-impinj commented Jul 3, 2024

jcooke-impinj commented Jul 3, 2024

adpaco-aws commented Jul 3, 2024

biswalc commented Jul 4, 2024 • edited Loading

junghyun-apr commented Jul 4, 2024

GuyAv46 commented Jul 4, 2024

Flamefire commented Jul 4, 2024 • edited Loading

david-batranu commented Jul 4, 2024

eshogo commented Jul 4, 2024 • edited Loading

david-batranu commented Jul 4, 2024

n1ngu commented Jul 4, 2024 • edited Loading

biswalc commented Jul 4, 2024

ayizhi commented Jul 5, 2024

FelixSchwarz commented Jul 5, 2024 • edited Loading

hasufell commented Jul 5, 2024

matthinea commented Jul 10, 2024

daiyam commented Jul 10, 2024

friedrichroell commented Jul 11, 2024

Kobzol commented Jul 11, 2024

friedrichroell commented Jul 11, 2024

Kobzol commented Jul 11, 2024

friedrichroell commented Jul 11, 2024

sawwn23 commented Jul 16, 2024

biswalc commented Jul 4, 2024 •

edited

Loading

Flamefire commented Jul 4, 2024 •

edited

Loading

eshogo commented Jul 4, 2024 •

edited

Loading

n1ngu commented Jul 4, 2024 •

edited

Loading

FelixSchwarz commented Jul 5, 2024 •

edited

Loading