-
Notifications
You must be signed in to change notification settings - Fork 920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Actions Runner fails on IPv6 only host #3138
Comments
Possible duplicate of #2840. Is there any news on progress? |
I am interested in this, too. |
fetch
with node
, failing with IPv6 DNS error UND_ERR_CONNECT_TIMEOUT
#3213
## Flags Changes This adds the proxy-init flag `--iptables-mode` (with possible values `legacy` and `nft`), which supersedes `--firewal-bin-path` and `firewall-save-bin-path` (which still remain supported). Also the `--ipv6` flag has been added (default `true`). After the set of rules run via iptables are processed, if `--ipv6` is true (which is the default), the same set of rules will be run via ip6tables. Analog changes were applied to linkerd-cni as well. ## Backwards-Compatibility This is backwards-compatible with older control planes and upcoming control planes. If `--ipv6` is not passed (and thus defaults to true), this doesn't impact operation even if the cluster doesn't support IPv6; the ip6tables rules are applied but they're innocuous. OTOH if there's no kernel support for IPv6 (which is the case for github runners*) then the ip6tables command will fail but we'll just log the failure and not fail the linkerd-init container (nor the `add` command for linkerd-cni). This avoids having to explicitly set `--ipv6=false`, but it can be set if the user is aware of such limitations and wants to get rid of the errors. ## Testing Improvements The cni-plugin-integration workflow has been simplified by using a matrix strategy, and enhanced by parameterizing the iptables-mode config. ## Linkerd IPv6 Support This allows routing IPv6 traffic to the proxy, but is just the first step towards IPv6/dual-stack support. Control plane and proxy changes will come up next. ## (*) Github Runners IPv6 Support Even though `modinfo` signals support for IPv6, `ip6tables` commands throw modprobe errors. Indeed, according to actions/runner-images#668 support is not there yet. Also, according to actions/runner#3138 there are issues with hosted runners as well, but that might not affect us if we still expose an IPv4 interface to interact with github. Something to take into account when we get to IPv6 integration testing.
This commit introduces the `force-ipv4` GitHub action to address connectivity issues caused by the lack of IPv6 support in GitHub runners. Details: - actions/runner#3138 - actions/runner-images#668 This change solves connection problems when Node's `fetch` API fails due to `UND_ERR_CONNECT_TIMEOUT` errors. Details: - actions/runner-images#9540 - actions/runner#3213 This action disables IPv6 at the system level, ensuring all outging requests use IPv4. Resolving connectivity issues when running external URL checks and Docker build checks. This solution is a temporary workaround until GitHub runners support IPv6 or Node `fetch` API has a working solution such as Happy Eyeball. Detais: - nodejs/node#41625 - nodejs/undici#1531
Here's my workaround (open-source and documented) that I hope that can help you too:
After days of research and trial/error, this is how I got this working:
Related commit introducing this fix: undergroundwires/privacy.sexy@52fadcd |
This commit upgrades Node.js version to v20.x in CI/CD environment. Previously used Node 18.x is moving towards end-of-life, with a planned date of 2025-04-30. In contrast, Node 20.x has been offering long-term support (LTS) since 2023-10-24. This makes Node 20.x a stable and recommended version for production environments. This commit also configures `actions/setup-node` with the `check-latest` flag to always use the latest Node 20.x version, keeping CI/CD setup up-to-date with minimal maintenance. Details: - actions/setup-node#165 - actions/setup-node#160 Using Node 20.x in CI/CD environments provides better compatibility with Electron v29.0 which moves to Node 20.x. Details: - electron/electron#40343 This upgrade improves network connection handling in CI/CD pipelines (where issues occur due to GitHub runners not supporting IPv6). Details: - actions/runner#3138 - actions/runner-images#668 - actions/runner#3213 - actions/runner-images#9540 Node 20.x adopts the Happy Eyeballs algorithm for improved IPv6 connectivity. - nodejs/node#40702 - nodejs/node#41625 - nodejs/node#44731 This mitigates issues like `UND_ERR_CONNECT_TIMEOUT` and localhost DNS resolution in CI/CD environments: Details: - nodejs/node#40537 - actions/runner#3213 - actions/runner-images#9540 Node 20 introduces `setDefaultAutoSelectFamily`, a global function from Node 19.4.0, enabling better IPv4 support, especially in environments with limited or problematic IPv6 support. Details: - nodejs/node#45777 Node 20.x defaults to the new `autoSelectFamily`, improving network connection reliability in GitHub runners lacking full IPv6 support. Details: - nodejs/node#46790
This commit upgrades Node.js version to v20.x in CI/CD environment. Previously used Node 18.x is moving towards end-of-life, with a planned date of 2025-04-30. In contrast, Node 20.x has been offering long-term support (LTS) since 2023-10-24. This makes Node 20.x a stable and recommended version for production environments. This commit also configures `actions/setup-node` with the `check-latest` flag to always use the latest Node 20.x version, keeping CI/CD setup up-to-date with minimal maintenance. Details: - actions/setup-node#165 - actions/setup-node#160 Using Node 20.x in CI/CD environments provides better compatibility with Electron v29.0 which moves to Node 20.x. Details: - electron/electron#40343 This upgrade improves network connection handling in CI/CD pipelines (where issues occur due to GitHub runners not supporting IPv6). Details: - actions/runner#3138 - actions/runner-images#668 - actions/runner#3213 - actions/runner-images#9540 Node 20.x adopts the Happy Eyeballs algorithm for improved IPv6 connectivity. - nodejs/node#40702 - nodejs/node#41625 - nodejs/node#44731 This mitigates issues like `UND_ERR_CONNECT_TIMEOUT` and localhost DNS resolution in CI/CD environments: Details: - nodejs/node#40537 - actions/runner#3213 - actions/runner-images#9540 Node 20 introduces `setDefaultAutoSelectFamily`, a global function from Node 19.4.0, enabling better IPv4 support, especially in environments with limited or problematic IPv6 support. Details: - nodejs/node#45777 Node 20.x defaults to the new `autoSelectFamily`, improving network connection reliability in GitHub runners lacking full IPv6 support. Details: - nodejs/node#46790
We faced the same issue when migrating our self-hosted runners to IPv6-only VPC on AWS. |
The runner needs a configurable switch to use IPv4 only in dual-stack systems until GitHub actually supports IPv6. |
Describe the bug
On an IPv6 only host a self-hosted action runner fails to register itself as online due to a failed network request to a URL with an IPv4 only address.
Context: Many hosting companies are now charging for IPv4 addresses (most recently AWS), so having IPv6-only self-hosted runners is desirable from an operational cost perspective.
To Reproduce
Expected behavior
I expected the runner to start and register as online.
Runner Version and Platform
Version of your runner?
2.312.0
OS of the machine running the runner? OSX/Windows/Linux/...
Linux (Debian 12)
What's not working?
Start-up and registration flow requires IPv4 to connect to a URL (
pipelinesghubeus4.actions.githubusercontent.com
)nslookup for host indicates no IPv6 address;
Job Log Output
Runner_20240208-075819-utc.log
, potential token redacted;Runner and Worker's Diagnostic Logs
Logs given above
The text was updated successfully, but these errors were encountered: