Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Equinix Metal tests are failling on all channels because of emergency shell #1536

Closed
tormath1 opened this issue Sep 5, 2024 · 1 comment
Closed
Assignees

Comments

@tormath1
Copy link
Contributor

tormath1 commented Sep 5, 2024

In the last builds (on all channels), we noticed this:

     --- PASS: kubeadm.v1.29.2.flannel.base/node_readiness (10.52s)
     --- PASS: kubeadm.v1.29.2.flannel.base/nginx_deployment (5.47s)
     --- PASS: kubeadm.v1.29.2.flannel.base/NFS_deployment (17.22s)
         harness.go:598: Found emergency shell on machine 704a8550-4d86-4a00-9fe4-70c5a6e00cc8 console
         harness.go:598: Found emergency shell on machine 35dedd13-6dd9-4795-9d74-3e09126a3045 console
         harness.go:598: Found emergency shell on machine 66848575-9be0-4680-923b-8bd52b4f65ed console

Tests are passing but there is an emergency shell in the logs, so the test is considered as a failure.

Impact

Tests are failing on Equinix Metal and they continue to run after 14h sometimes as the CI is trying to rerun the tests + provisioning time.

Environment and steps to reproduce

  1. Run Equinix Metal tests in the CI or repro locally with an Equinix Metal account

Expected behavior

Tests are passing

Additional information

systemd-networkd is not running when ignition-mount is started but Ignition tries, at each stage, to request Equinix Metal metadata service, so it fails. It's not a systemd issue as LTS is affected and the breakage, on LTS, occurs between the last release (that was working) and the current nightlies: flatcar/scripts@lts-3510.3.5...lts-3510.3.5-nightly-20240830-2100 - could be a kernel issue?

@tormath1
Copy link
Contributor Author

Backported to all channels (alpha, beta, stable and LTS)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

1 participant