-
Notifications
You must be signed in to change notification settings - Fork 706
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrading Cluster from 1.23.9 to 1.24.10 not possible #2819
Comments
/retitle Upgrading Cluster from 1.23.9 to 1.24.10 not possible |
Hi @michaelschefczyk
It appears that you are still using docker as the container runtime, and it does not support the runtime.v1 api. What's the |
/support |
Hello, @michaelschefczyk 🤖 👋 You seem to have troubles using Kubernetes and kubeadm. Please see: |
yes, if using docker try moving to containerd |
No need to worry about me, but please consider working on more telling error messages! I did resolve my case by seeting up a new cluster. Of course, this is a matrix of n-dimensional complexity: Debian bullseye does not provide a containerd-package compatible with Kubernetes > 1.25, so one needs to move to another source (in my case docker repository). Then, if one wants to use a private repository (outside docker, so that is is available even when the cluster is off), one does need tls where the certificates are trusted and providing the name via SAN (unclear if this is imposed by the particular containerd-version or by kubelet - there is documentation to skip certificate verification but it does not seem to match the latest version of containerd and/or kubelet). Hence, I made my respository available via a reverse proxy (haproxy). |
I have just experienced this problem and I believe the image pull errors that you experienced @michaelschefczyk are because your installed version of You can remedy the issue by downgrading apt install cri-tools=1.23.17-00 Now that I've upgraded to |
i am surprised cri-dockerd is as maintained. note that the original estimate was that the project will not be able to keep up. in any case, k8s recommends containerd or crio. |
Versions
kubeadm version: 1.23.9
Environment:
Kubernetes version: 1.23.9
Cloud provider or hardware configuration: Self-hosted virtual machine
OS: Debian GNU/Linux 11 (bullseye)
Kernel: 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
Container runtime: containerd github.com/containerd/containerd 1.4.13
ds1 1.4.13ds1-1~deb11u3Container networking plugin (CNI): rancher/mirrored-flannelcni-flannel v0.19.1
What happened?
After six months, I did intend to upgrade the cluster from 1.23.9 to 1.24.10
based on: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/
Upon kubeadm upgrade apply v1.24.10, Errors:
[preflight] Some fatal errors occurred:
[ERROR ImagePull]: failed to pull image registry.k8s.io/kube-apiserver:v1.24.10: output: time="2023-02-03T18:17:55+01:00" level=fatal msg="validate service connection: CRI v1 image API is not implemented for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/kube-controller-manager:v1.24.10: output: time="2023-02-03T18:17:55+01:00" level=fatal msg="validate service connection: CRI v1 image API is not implemented for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/kube-scheduler:v1.24.10: output: time="2023-02-03T18:17:55+01:00" level=fatal msg="validate service connection: CRI v1 image API is not implemented for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/kube-proxy:v1.24.10: output: time="2023-02-03T18:17:55+01:00" level=fatal msg="validate service connection: CRI v1 image API is not implemented for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/pause:3.7: output: time="2023-02-03T18:17:56+01:00" level=fatal msg="validate service connection: CRI v1 image API is not implemented for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/etcd:3.5.6-0: output: time="2023-02-03T18:17:56+01:00" level=fatal msg="validate service connection: CRI v1 image API is not implemented for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService"
, error: exit status 1
[ERROR ImagePull]: failed to pull image registry.k8s.io/coredns/coredns:v1.8.6: output: time="2023-02-03T18:17:56+01:00" level=fatal msg="validate service connection: CRI v1 image API is not implemented for endpoint "unix:///var/run/cri-dockerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService"
, error: exit status 1
editing kubectl -n kube-system edit cm kubeadm-config
imageRepository back to k8s.gcr.io does not work -> overwritten by registry.k8s.io each time
updating containerd to newer version from docker repo does not work, because then the API server is no longer available.
P.S. Support question: Is there a way out of this or do I kill my cluster and start over from scratch?
What you expected to happen?
Clean upgrade, as before.
How to reproduce it (as minimally and precisely as possible)?
Uncertain. It does happen every time I try it from a reverted version.
Anything else we need to know?
The text was updated successfully, but these errors were encountered: