You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem:
After joining (kubeadm join) a new node (with no pre-installed images), I get only pause:3.5.
However, the /var/lib/kubelet/kubeadm-flags.env file contains the same as on the main node, that is:
KUBELET_KUBEADM_ARGS="--container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --pod-infra-container-image=k8s.gcr.io/pause:3.7"
In /var/log/messages, on the new node, we get:
Jul 10 03:28:48 localhost crio[1248]: time="2022-07-10 03:28:48.288405920-04:00" level=info msg="Checking image status: k8s.gcr.io/pause:3.7" id=c5f090a6-0c1f-4221-9bcb-716b879ca4cb name=/runtime.v1.ImageService/ImageStatus
Jul 10 03:28:48 localhost crio[1248]: time="2022-07-10 03:28:48.288573150-04:00" level=info msg="Image k8s.gcr.io/pause:3.7 not found" id=c5f090a6-0c1f-4221-9bcb-716b879ca4cb name=/runtime.v1.ImageService/ImageStatus
A workaround is to do "podman pull k8s.gcr.io/pause:3.7" directly on the new node.
What you expected to happen?
Correct versions of kubernetes infrastructure be installed without manual intervention.
How to reproduce it (as minimally and precisely as possible)?
Starting system from scratch, with no pre-installed images (podman image rm -a).
On main node (kubeadm init), I get two versions of the pause container: 3.5 and 3.7.
The reference to version 3.7 seems to come from /var/lib/kubelet/kubeadm-flags.env:
KUBELET_KUBEADM_ARGS="--container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --pod-infra-container-image=k8s.gcr.io/pause:3.7"
The reference to version 3.5 seems to come from cri-o, crio-status config yields:
[crio.image]
pause_image = "k8s.gcr.io/pause:3.5"
(Are these references to two different versions a problem?)
Now, in /var/log/messages, on the main node, we get:
Jul 16 16:45:19 localhost crio[1311004]: time="2022-07-16 16:45:19.570935505-04:00" level=info msg="Checking image status: k8s.gcr.io/pause:3.7" id=8d3456d7-6a7a-4cb1-a1f8-df9b61e8e9bb name=/runtime.v1.ImageService/ImageStatus
Jul 16 16:45:19 localhost crio[1311004]: time="2022-07-16 16:45:19.571631830-04:00" level=info msg="Image status: &{0xc00034a070 map[]}" id=8d3456d7-6a7a-4cb1-a1f8-df9b61e8e9bb name=/runtime.v1.ImageService/ImageStatus
And after joining a new node, on the new node, we get a "not found" error in /var/log/messages about pause:3.7.
Anything else we need to know?
The text was updated successfully, but these errors were encountered:
Thanks for pointing to this config of crio.
I've therefore done kubeadm config images list to get the correct version of the pause container (since the reference from the page above listed 3.6 and my setup seems to want 3.7).
I've added this config to crio.conf, sent HUP to crio, checked with systemctl status crio.
Then started a fresh install with kubeadm init and kubeadm join and now I only get pause:3.7.
No more "not found" messages in /var/log/messages.
Thanks.
What keywords did you search in kubeadm issues before filing this one?
pause version problem join
Is this a BUG REPORT or FEATURE REQUEST?
BUG REPORT
Versions
kubeadm version (use
kubeadm version
):kubeadm version: &version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.1", GitCommit:"3ddd0f45aa91e2f30c70734b175631bec5b5825a", GitTreeState:"clean", BuildDate:"2022-05-24T12:24:38Z", GoVersion:"go1.18.2", Compiler:"gc", Platform:"linux/amd64"}
Environment:
Kubernetes version (use
kubectl version
):Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.1", GitCommit:"3ddd0f45aa91e2f30c70734b175631bec5b5825a", GitTreeState:"clean", BuildDate:"2022-05-24T12:26:19Z", GoVersion:"go1.18.2", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.3", GitCommit:"aef86a93758dc3cb2c658dd9657ab4ad4afc21cb", GitTreeState:"clean", BuildDate:"2022-07-13T14:23:26Z", GoVersion:"go1.18.3", Compiler:"gc", Platform:"linux/amd64"}
Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release):
NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Stream 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
Kernel (e.g.
uname -a
):Linux XXX 4.18.0-394.el8.x86_64 kubeadm join on slave node fails preflight checks #1 SMP Tue May 31 16:19:11 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Container runtime (CRI) (e.g. containerd, cri-o):
cri-o-1.21.3-1.module_el8+12969+ebff6d46.x86_64
Container networking plugin (CNI) (e.g. Calico, Cilium):
calico v3.23
Others:
What happened?
Problem:
After joining (kubeadm join) a new node (with no pre-installed images), I get only pause:3.5.
However, the /var/lib/kubelet/kubeadm-flags.env file contains the same as on the main node, that is:
KUBELET_KUBEADM_ARGS="--container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --pod-infra-container-image=k8s.gcr.io/pause:3.7"
In /var/log/messages, on the new node, we get:
Jul 10 03:28:48 localhost crio[1248]: time="2022-07-10 03:28:48.288405920-04:00" level=info msg="Checking image status: k8s.gcr.io/pause:3.7" id=c5f090a6-0c1f-4221-9bcb-716b879ca4cb name=/runtime.v1.ImageService/ImageStatus
Jul 10 03:28:48 localhost crio[1248]: time="2022-07-10 03:28:48.288573150-04:00" level=info msg="Image k8s.gcr.io/pause:3.7 not found" id=c5f090a6-0c1f-4221-9bcb-716b879ca4cb name=/runtime.v1.ImageService/ImageStatus
A workaround is to do "podman pull k8s.gcr.io/pause:3.7" directly on the new node.
What you expected to happen?
Correct versions of kubernetes infrastructure be installed without manual intervention.
How to reproduce it (as minimally and precisely as possible)?
Starting system from scratch, with no pre-installed images (podman image rm -a).
On main node (kubeadm init), I get two versions of the pause container: 3.5 and 3.7.
The reference to version 3.7 seems to come from /var/lib/kubelet/kubeadm-flags.env:
KUBELET_KUBEADM_ARGS="--container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --pod-infra-container-image=k8s.gcr.io/pause:3.7"
The reference to version 3.5 seems to come from cri-o, crio-status config yields:
[crio.image]
pause_image = "k8s.gcr.io/pause:3.5"
(Are these references to two different versions a problem?)
Now, in /var/log/messages, on the main node, we get:
Jul 16 16:45:19 localhost crio[1311004]: time="2022-07-16 16:45:19.570935505-04:00" level=info msg="Checking image status: k8s.gcr.io/pause:3.7" id=8d3456d7-6a7a-4cb1-a1f8-df9b61e8e9bb name=/runtime.v1.ImageService/ImageStatus
Jul 16 16:45:19 localhost crio[1311004]: time="2022-07-16 16:45:19.571631830-04:00" level=info msg="Image status: &{0xc00034a070 map[]}" id=8d3456d7-6a7a-4cb1-a1f8-df9b61e8e9bb name=/runtime.v1.ImageService/ImageStatus
And after joining a new node, on the new node, we get a "not found" error in /var/log/messages about pause:3.7.
Anything else we need to know?
The text was updated successfully, but these errors were encountered: