Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kind create cluster fails on MacOS + docker desktop #3061

Closed
ep4sh opened this issue Jan 11, 2023 · 22 comments
Closed

kind create cluster fails on MacOS + docker desktop #3061

ep4sh opened this issue Jan 11, 2023 · 22 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.

Comments

@ep4sh
Copy link

ep4sh commented Jan 11, 2023 •
edited

What happened:

 ~ kind create cluster
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.25.3) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✗ Starting control-plane 🕹️
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged kind-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I0111 12:08:48.281449     132 initconfiguration.go:254] loading configuration from "/kind/kubeadm.conf"
W0111 12:08:48.282858     132 initconfiguration.go:331] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
[init] Using Kubernetes version: v1.25.3
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0111 12:08:48.289153     132 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0111 12:08:48.393822     132 certs.go:522] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.23.0.2 127.0.0.1]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0111 12:08:48.827200     132 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0111 12:08:49.033611     132 certs.go:522] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0111 12:08:49.219289     132 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0111 12:08:49.319645     132 certs.go:522] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.23.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.23.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0111 12:08:49.742862     132 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
I0111 12:08:49.876848     132 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
I0111 12:08:50.089378     132 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0111 12:08:50.161976     132 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0111 12:08:50.598000     132 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0111 12:08:50.653232     132 kubelet.go:66] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0111 12:08:50.770349     132 manifests.go:99] [control-plane] getting StaticPodSpecs
I0111 12:08:50.770602     132 certs.go:522] validating certificate period for CA certificate
I0111 12:08:50.770699     132 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0111 12:08:50.770705     132 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0111 12:08:50.770709     132 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0111 12:08:50.770712     132 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0111 12:08:50.770716     132 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
I0111 12:08:50.773904     132 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I0111 12:08:50.773964     132 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0111 12:08:50.774644     132 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0111 12:08:50.774687     132 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0111 12:08:50.774693     132 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0111 12:08:50.774697     132 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0111 12:08:50.774701     132 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0111 12:08:50.774705     132 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0111 12:08:50.774709     132 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0111 12:08:50.776035     132 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I0111 12:08:50.776106     132 manifests.go:99] [control-plane] getting StaticPodSpecs
I0111 12:08:50.776262     132 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0111 12:08:50.776771     132 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0111 12:08:50.777457     132 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0111 12:08:50.777499     132 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0111 12:08:50.777977     132 loader.go:374] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0111 12:08:50.781500     132 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0111 12:08:51.282917     132 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0111 12:08:51.783460     132 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0111 12:08:52.283225     132 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
.....
........
...............
I0111 12:12:50.788384     132 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
couldn't initialize a Kubernetes cluster
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/init.runWaitControlPlanePhase
	cmd/kubeadm/app/cmd/phases/init/waitcontrolplane.go:108
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:234
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:421
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:207
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:154
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:974
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:902
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594
error execution phase wait-control-plane
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:235
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:421
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:207
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:154
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:974
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:902
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594

Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'

What you expected to happen:
A cluster has been created

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Additional logs attached to the issue as .zip archive.
kind-control-plane.zip

Environment:

  • kind version: (use kind version):
 ~ kind --version
kind version 0.17.0
  • Runtime info: (use docker info or podman info):
  • OS (e.g. from /etc/os-release):
    MacOSx + docker desktop (Docker Desktop 4.15.0 (93002) is currently the newest version available.)
  • Kubernetes version: (use kubectl version):
 ~ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.2", GitCommit:"5835544ca568b757a8ecae5c153f317e5736700e", GitTreeState:"clean", BuildDate:"2022-09-21T14:33:49Z", GoVersion:"go1.19.1", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.6", GitCommit:"ad3338546da947756e8a88aa6822e9c11e7eac22", GitTreeState:"clean", BuildDate:"2022-04-14T08:43:11Z", GoVersion:"go1.17.9", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.25) and server (1.23) exceeds the supported minor version skew of +/-1
 ~ docker info
Client:
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.13.0)
  dev: Docker Dev Environments (Docker Inc., v0.0.5)
  extension: Manages Docker extensions (Docker Inc., v0.2.16)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.22.0)

Server:
 Containers: 10
  Running: 1
  Paused: 0
  Stopped: 9
 Images: 26
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 7.675GiB
 Name: docker-desktop
 ID: ERYV:IPQQ:OQAQ:GX7W:XYED:ICON:W4GJ:A3V2:45F5:GTRB:OY3H:IFZZ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false
  • Any proxies or other special environment settings?: nope
@ep4sh ep4sh added the kind/bug Categorizes issue or PR as related to a bug. label Jan 11, 2023
@stmcginnis
Copy link
Contributor

Can you fill in the rest of the details at the end? The docker info and kubectl version details may be useful.

Looking at the logs, it looks like your Docker install might not be healthy. This is in the log:

OCI runtime create failed: runc create failed: unable to start container process

Can you verify you are able to run containers outside of kind. Something like docker run hello-world is a decent sanity check.

@ep4sh
Copy link
Author

ep4sh commented Jan 11, 2023 •
edited

@stmcginnis yep, sure, sorry for missing it (I've just forgotten to add it =\
kubectl and docker info added above,
I was able to run hello-world container:

 ~ docker run  hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Btw, recentrly I was able to run kind cluster, before the upgrade.

@stmcginnis
Copy link
Contributor

Thanks! I don't see anything obvious from the output that would cause an issue. A common Docker Desktop thing I've seen is needing to allocate more CPU and memory, but you have 6 CPU and ~8 GB memory, so that should be plenty for running a cluster.

It's quite possible the new Docker Desktop release changed something that is causing this, but what that could be I don't know yet. Let's see if someone else can come along with any ideas.

In the meantime, and if it's possible, it might be best to downgrade your Docker Desktop to a working version until someone can sort out what's causing an issue here. If you downgrade and it works again, that would be a very good indicator that it is indeed a problem with the newer DD release.

@ep4sh
Copy link
Author

ep4sh commented Jan 11, 2023

Got it, thank you so much.
I'll try it!

@BenTheElder
Copy link
Member

Jan 11 12:55:26 kind-control-plane containerd[99]: time="2023-01-11T12:55:26.777135184Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-controller-manager-kind-control-plane,Uid:aaff90ec64f346d418f0a93d766752c5,Namespace:kube-system,Attempt:0,} failed, error" error="failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error reopening /dev/null inside container: open /dev/null: operation not permitted: unknown"

This is the containerd instance inside the docker container "node". The permissions issue is inside the nested container but could be coming from the node filesystem.

Can we check what /dev/null permission is at the "node" level by docker exec kind-control-plane ls -l /dev/null?

Docker desktop has previously had a umask issue with /dev/null docker/for-mac#5527

@ep4sh
Copy link
Author

ep4sh commented Jan 12, 2023

Hey @BenTheElder, thank you for reply!

yep, sure:

 ~ docker exec kind-control-plane ls -l /dev/null
crw-rw-rw- 1 root root 1, 3 Jan 11 12:55 /dev/null

@BenTheElder
Copy link
Member

Hmm looks like /dev/null is correct on the node.

@BenTheElder BenTheElder added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Jan 19, 2023
@alex-richter
Copy link

I've just stumbled across the same issue (also using my daily driver mac with docker desktop) and did some debugging. One thing that immediately caught my eye is that my journal.log was full of error messages like this:

Jan 20 16:19:32 kind-control-plane kubelet[184]: E0120 16:19:32.302684 184 pod_workers.go:965] "Error syncing pod, skipping" err="failed to \"CreatePodS andbox\" for \"kube-scheduler-kind-control-plane_kube-system(6d3dda2cad9846e0d48dbd5d5b9f59fc)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"kube-scheduler-kind-control-plane_kube-system(6d3dda2cad9846e0d48dbd5d5b9f59fc)\\\": rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format \\\"slice:prefix:name\\\" for systemd cgroups, got \\\"/kubelet/kubepods/burstable/pod6d3dda2cad9846e0d48dbd5d5b9f59fc/5bc1337ac55d891c743783740d686e714686e063bb37969ac965f44f2ab091de\\\" instead: unknown\"" pod="kube-system/kube-scheduler-kind-control-plane" podUID=6d3dda2cad9846e0d48dbd5d5b9f59fc

While doing some research on that error, I found this thread which explained the cause nicely: containerd/containerd#4857 (comment). So next, I did what was suggested in that post - I created a custom KubeletConfiguration that explicitly sets systemd as the cgroupDriver and put that into my kind config file. And now it actually works for me again. Here is the relevant part of my kind config for reference:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: KubeletConfiguration
    cgroupDriver: systemd

Hope this helps you out!

@BenTheElder
Copy link
Member

BenTheElder commented Jan 20, 2023 •
edited

kind v0.17 should be using systemd for the cgroupDriver for kubernetes 1.24+ 🤔

kind/pkg/cluster/internal/kubeadm/config.go

Lines 508 to 512 in 41b3503

// before 1.24 kind uses cgroupfs
// after 1.24 kind uses systemd starting in kind v0.13.0
// before kind v0.13.0 kubernetes 1.24 wasn't released yet
if ver.LessThan(version.MustParseSemantic("v1.24.0")) {
data.CgroupDriver = "cgroupfs"

@fengqi2010
Copy link

fengqi2010 commented Feb 22, 2023 •
edited

@alex-richter thanks for your advise, it works fine for me.

@BenTheElder
Copy link
Member

Still need to understand why it's not doing this without that config file, something is wrong :/

@alex-richter can you share your exact invocation? were you setting the node --image version or anything? what kind version?

@matrus2
Copy link

matrus2 commented Mar 16, 2023

I had the same issue. Thanks @alex-richter you solved that.

@BenTheElder
Copy link
Member

If anyone encountering this and seeing @alex-richter's config fix it could please share some information like:

  • kind version
  • docker info
  • exact kind command line / flags / config used

This solution is very confusing because kind v0.17.0 should default to:

  • kubernetes 1.25.3
  • KubeletConfiguration setting cgroupDriver: systemd for nodes with version 1.24.0+, cgroupfs for <1.24.0

@matrus2
Copy link

matrus2 commented Mar 17, 2023

kind version
kind v0.17.0 go1.19.2 darwin/amd64

docker info
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., v0.7.1)
compose: Docker Compose (Docker Inc., v2.2.3)
scan: Docker Scan (Docker Inc., v0.17.0)

Server:
Containers: 4
Running: 4
Paused: 0
Stopped: 0
Images: 1
Server Version: 20.10.12
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc version: v1.0.2-0-g52b36a2
init version: de40ad0
Security Options:
seccomp
Profile: default
cgroupns
Kernel Version: 5.10.76-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.64GiB
Name: docker-desktop
ID: TRH6:VDPL:PKKK:BZ25:YYFY:V3ZD:W4S5:42LU:IDYQ:P3XH:7VR5:GP3G
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5000
127.0.0.0/8
Live Restore Enabled: false

exact kind command line / flags / config used
The issue exists whenever I try to create cluster with multi node setup. I tried with multiple different networking configuration as I though that my custom setup may have some impact, but no.

@henrjk henrjk mentioned this issue Mar 17, 2023
Closed
@BenTheElder
Copy link
Member

BenTheElder commented Mar 17, 2023 •
edited

Thank you @matrus2

Are you able to confirm that this issue happens with kind create cluster -v 9 (no options, just cranking verbosity so we can see the kubeadm/kubelet config used), and is fixed with kind create cluster --config=<path to alex-richter's config>?

I'll try to get a mac test environment again soon (can't run docker desktop at work ...) and reproduce, been a hectic period for Kubernetes / KIND 😅

When I run kind create cluster -v 9 on my linux box with kind v0.17.0 I see the following kubelet config in the logs:

apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd
cgroupRoot: /kubelet
evictionHard:
  imagefs.available: 0%
  nodefs.available: 0%
  nodefs.inodesFree: 0%
failSwapOn: false
imageGCHighThresholdPercent: 100
kind: KubeletConfiguration

With the default 1.25.3 image.

If any of you are setting a custom node image, then please see the release notes for v0.13.0 / v0.14.0 noting the incompatible change with cgroups going forward:
https://github.com/kubernetes-sigs/kind/releases/tag/v0.13.0
https://github.com/kubernetes-sigs/kind/releases/tag/v0.14.0

And the more general note about images at https://kind.sigs.k8s.io/docs/user/quick-start/#creating-a-cluster

Unfortunately this is one of those tricky changes that we couldn't fix without requiring kind <> image interaction changes.

So far none of the comments here confirm manually configuring systemd works with any particular image, and my remainig suspicion is that it's a problem with the particular node image(s) being used. (Original comment in this thread involves 1.25.3, but also isn't clearly due to the cgroup driver and may be a different issue)

@BenTheElder
Copy link
Member

/dev/null issues may be #2999 (comment)

Please try again with the latest docker and KIND.

For the systemd cgroups workaround, see above, I still need more confirmation, I can't reproduce this.

@vallpaper
Copy link

Hi, running into this issue. On my M1 Mac I started using DOCKER_DEFAULT_PLATFORM set to linux/amd64 because not everything I work is compatible and can be compiled directly using arm64 images. With this set I am not able to run Kind with default config. Once removed running kind works.

❯ kind version
kind v0.18.0 go1.20.2 darwin/arm64
❯ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.10.4)
  compose: Docker Compose (Docker Inc., v2.17.2)
  dev: Docker Dev Environments (Docker Inc., v0.1.0)
  extension: Manages Docker extensions (Docker Inc., v0.2.19)
  init: Creates Docker-related starter files for your project (Docker Inc., v0.1.0-beta.2)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.25.0)
  scout: Command line tool for Docker Scout (Docker Inc., v0.9.0)

Server:
 Containers: 112
  Running: 0
  Paused: 0
  Stopped: 112
 Images: 167
 Server Version: 20.10.24
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 9.718GiB
 Name: docker-desktop
 ID: BQVW:3QUO:YAJJ:2WH5:6NW3:7Q4F:ULN5:A67K:SDAZ:RVOA:UGOD:R6IH
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false
❯ echo $DOCKER_DEFAULT_PLATFORM
linux/amd64
❯ kind create cluster -v 9
Creating cluster "kind" ...
DEBUG: docker/images.go:58] Image: kindest/node:v1.26.3@sha256:61b92f38dff6ccc29969e7aa154d34e38b89443af1a2c14e6cfbd2df6419c66f present locally
 ✓ Ensuring node image (kindest/node:v1.26.3) 🖼
 ✓ Preparing nodes 📦
DEBUG: config/config.go:96] Using the following kubeadm config for node kind-control-plane:
apiServer:
  certSANs:
  - localhost
  - 127.0.0.1
  extraArgs:
    runtime-config: ""
apiVersion: kubeadm.k8s.io/v1beta3
clusterName: kind
controlPlaneEndpoint: kind-control-plane:6443
controllerManager:
  extraArgs:
    enable-hostpath-provisioner: "true"
kind: ClusterConfiguration
kubernetesVersion: v1.26.3
networking:
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/16
scheduler:
  extraArgs: null
---
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- token: abcdef.0123456789abcdef
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 172.19.0.2
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
  kubeletExtraArgs:
    node-ip: 172.19.0.2
    node-labels: ""
    provider-id: kind://docker/kind/kind-control-plane
---
apiVersion: kubeadm.k8s.io/v1beta3
controlPlane:
  localAPIEndpoint:
    advertiseAddress: 172.19.0.2
    bindPort: 6443
discovery:
  bootstrapToken:
    apiServerEndpoint: kind-control-plane:6443
    token: abcdef.0123456789abcdef
    unsafeSkipCAVerification: true
kind: JoinConfiguration
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
  kubeletExtraArgs:
    node-ip: 172.19.0.2
    node-labels: ""
    provider-id: kind://docker/kind/kind-control-plane
---
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd
cgroupRoot: /kubelet
evictionHard:
  imagefs.available: 0%
  nodefs.available: 0%
  nodefs.inodesFree: 0%
failSwapOn: false
imageGCHighThresholdPercent: 100
kind: KubeletConfiguration
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
conntrack:
  maxPerCore: 0
iptables:
  minSyncPeriod: 1s
kind: KubeProxyConfiguration
mode: iptables
 ✓ Writing configuration 📜
DEBUG: kubeadminit/init.go:82] I0427 11:07:01.428364     131 initconfiguration.go:254] loading configuration from "/kind/kubeadm.conf"
W0427 11:07:01.438621     131 initconfiguration.go:331] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
[init] Using Kubernetes version: v1.26.3
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0427 11:07:01.476951     131 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0427 11:07:01.599293     131 certs.go:519] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.19.0.2 127.0.0.1]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0427 11:07:01.903918     131 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0427 11:07:02.114696     131 certs.go:519] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0427 11:07:02.783216     131 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0427 11:07:02.887968     131 certs.go:519] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.19.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.19.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0427 11:07:03.942267     131 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0427 11:07:04.595401     131 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I0427 11:07:04.722579     131 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0427 11:07:04.866722     131 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0427 11:07:05.002956     131 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0427 11:07:05.197347     131 kubelet.go:67] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0427 11:07:05.786067     131 manifests.go:99] [control-plane] getting StaticPodSpecs
I0427 11:07:05.787990     131 certs.go:519] validating certificate period for CA certificate
I0427 11:07:05.788277     131 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0427 11:07:05.788304     131 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0427 11:07:05.788314     131 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0427 11:07:05.788318     131 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0427 11:07:05.788321     131 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0427 11:07:05.790717     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I0427 11:07:05.790735     131 manifests.go:99] [control-plane] getting StaticPodSpecs
I0427 11:07:05.790963     131 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0427 11:07:05.790978     131 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0427 11:07:05.790982     131 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0427 11:07:05.790985     131 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0427 11:07:05.790989     131 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0427 11:07:05.790992     131 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0427 11:07:05.790995     131 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I0427 11:07:05.791509     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I0427 11:07:05.791527     131 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0427 11:07:05.791659     131 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0427 11:07:05.792010     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0427 11:07:05.793709     131 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0427 11:07:05.793811     131 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0427 11:07:05.794819     131 loader.go:373] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0427 11:07:05.813744     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 9 milliseconds
I0427 11:07:06.318485     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:06.824833     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:07.319677     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:07.817968     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:08.321718     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:08.823148     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:09.317155     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:09.822250     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:10.325020     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:10.822645     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:11.317208     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:11.824738     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:12.321052     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:12.819015     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:13.317286     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:13.820167     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:14.321543     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:14.817135     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:15.323890     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:15.825193     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:16.319811     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:16.817207     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:17.324115     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:17.825933     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:18.318976     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:18.824423     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:19.320235     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:19.817786     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:20.322200     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:20.823830     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:21.320609     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:21.817680     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:22.322037     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:22.819323     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:23.320725     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:23.818120     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:24.322866     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:24.820421     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:25.318654     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:25.821047     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:26.321285     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:26.817184     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:27.322339     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:27.821691     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:28.325375     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:28.819509     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:29.322559     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:29.824204     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:30.319689     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:30.821246     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:31.321032     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:31.824114     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:32.318539     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:32.822866     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:33.324692     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:33.817703     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:34.322007     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:34.825144     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:35.323927     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:35.817839     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:36.324706     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:36.824061     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:37.317605     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:37.817211     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:38.324592     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:38.820843     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:39.318031     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:39.823483     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:40.323962     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:40.816693     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:41.321072     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:41.822924     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:42.320033     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:42.819184     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:43.319430     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:43.825305     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:44.318112     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:44.817067     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:45.324856     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0427 11:07:45.818422     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:46.318472     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:46.822264     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:47.322314     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:47.818332     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:48.317476     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:48.823233     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:49.323384     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:49.817349     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:50.323027     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0427 11:07:50.818251     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:51.316912     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:51.821880     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:52.321882     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:52.821838     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:53.316678     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:53.824586     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:54.324069     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:54.817176     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:55.317022     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:55.822469     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:56.319542     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:56.817783     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:57.323958     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:57.820361     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:58.317110     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:58.818922     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:59.319032     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:59.821054     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:00.316623     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:00.819378     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0427 11:08:01.322042     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:01.818946     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:02.319903     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:02.824312     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:03.320795     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:03.816675     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:04.323297     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:04.823039     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:05.317623     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:05.816430     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:06.324785     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:06.819457     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:07.316295     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:07.822199     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:08.319605     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:08.816257     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:09.316964     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:09.819168     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:10.323450     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:10.816550     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:11.316910     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:11.818859     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:12.317833     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:12.817145     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:13.318528     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:13.819137     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:14.316196     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:14.819404     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:15.319943     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:15.817061     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:16.316797     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:16.823159     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:17.321248     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:17.816118     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:18.322050     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:18.818836     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:19.316595     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:19.816461     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:20.319940     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:20.822393     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0427 11:08:21.317224     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:21.818454     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:22.323473     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:22.816267     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:23.317433     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:23.824284     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:24.323284     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:08:24.816252     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:25.325616     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:08:25.819574     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:26.319043     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:26.817216     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:27.323054     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:27.819741     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:28.316234     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:28.819735     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:29.319493     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:29.816379     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:30.317462     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:30.821716     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:31.321720     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:31.816607     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:32.319402     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:32.818253     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:33.316259     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:33.816365     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:34.323332     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:34.826241     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 4 milliseconds
I0427 11:08:35.317415     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:35.823157     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:36.319711     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:36.816448     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:37.315950     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:37.822555     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:38.322985     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:38.815900     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:39.317285     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:39.823722     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:40.316641     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:40.818462     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:41.319838     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:41.820097     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:42.315632     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:42.819643     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:43.318534     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:43.815949     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:44.315735     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:44.816657     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:45.323428     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:45.816728     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:46.318623     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:46.822952     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:47.316109     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:47.815855     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:48.318422     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:48.821683     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:49.316536     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:49.819204     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:50.319998     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:50.816082     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:51.316201     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:51.817027     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:52.321288     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:52.817069     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:53.319695     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:53.822404     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:54.316198     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:54.818565     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:55.319625     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:55.818995     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:56.318037     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:56.821023     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:57.320723     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:57.816630     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:58.315970     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:58.818277     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:59.321324     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:59.816871     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:09:00.323486     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:09:00.819294     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.

Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
couldn't initialize a Kubernetes cluster
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/init.runWaitControlPlanePhase
	cmd/kubeadm/app/cmd/phases/init/waitcontrolplane.go:108
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:259
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:112
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594
error execution phase wait-control-plane
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:260
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:112
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594
 ✗ Starting control-plane 🕹️
Deleted nodes: ["kind-control-plane"]
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged kind-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I0427 11:07:01.428364     131 initconfiguration.go:254] loading configuration from "/kind/kubeadm.conf"
W0427 11:07:01.438621     131 initconfiguration.go:331] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
[init] Using Kubernetes version: v1.26.3
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0427 11:07:01.476951     131 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0427 11:07:01.599293     131 certs.go:519] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.19.0.2 127.0.0.1]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0427 11:07:01.903918     131 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0427 11:07:02.114696     131 certs.go:519] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0427 11:07:02.783216     131 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0427 11:07:02.887968     131 certs.go:519] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.19.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.19.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0427 11:07:03.942267     131 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0427 11:07:04.595401     131 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I0427 11:07:04.722579     131 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0427 11:07:04.866722     131 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0427 11:07:05.002956     131 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0427 11:07:05.197347     131 kubelet.go:67] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0427 11:07:05.786067     131 manifests.go:99] [control-plane] getting StaticPodSpecs
I0427 11:07:05.787990     131 certs.go:519] validating certificate period for CA certificate
I0427 11:07:05.788277     131 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0427 11:07:05.788304     131 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0427 11:07:05.788314     131 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0427 11:07:05.788318     131 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0427 11:07:05.788321     131 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0427 11:07:05.790717     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I0427 11:07:05.790735     131 manifests.go:99] [control-plane] getting StaticPodSpecs
I0427 11:07:05.790963     131 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0427 11:07:05.790978     131 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0427 11:07:05.790982     131 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0427 11:07:05.790985     131 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0427 11:07:05.790989     131 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0427 11:07:05.790992     131 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0427 11:07:05.790995     131 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I0427 11:07:05.791509     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I0427 11:07:05.791527     131 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0427 11:07:05.791659     131 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0427 11:07:05.792010     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0427 11:07:05.793709     131 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0427 11:07:05.793811     131 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0427 11:07:05.794819     131 loader.go:373] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0427 11:07:05.813744     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 9 milliseconds
I0427 11:07:06.318485     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:06.824833     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:07.319677     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:07.817968     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:08.321718     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:08.823148     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:09.317155     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:09.822250     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:10.325020     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:10.822645     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:11.317208     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:11.824738     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:12.321052     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:12.819015     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:13.317286     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:13.820167     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:14.321543     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:14.817135     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:15.323890     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:15.825193     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:16.319811     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:16.817207     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:17.324115     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:17.825933     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:18.318976     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:18.824423     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:19.320235     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:19.817786     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:20.322200     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:20.823830     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:21.320609     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:21.817680     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:22.322037     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:22.819323     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:23.320725     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:23.818120     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:24.322866     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:24.820421     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:25.318654     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:25.821047     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:26.321285     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:26.817184     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:27.322339     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:27.821691     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:28.325375     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:28.819509     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:29.322559     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:29.824204     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:30.319689     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:30.821246     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:31.321032     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:31.824114     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:32.318539     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:32.822866     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:33.324692     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:33.817703     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:34.322007     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:34.825144     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:35.323927     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:35.817839     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:36.324706     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:36.824061     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:37.317605     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:37.817211     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:38.324592     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:38.820843     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:39.318031     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:39.823483     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:40.323962     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:40.816693     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:41.321072     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:41.822924     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:42.320033     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:42.819184     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:43.319430     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:43.825305     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:44.318112     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:44.817067     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:45.324856     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0427 11:07:45.818422     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:46.318472     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:46.822264     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:47.322314     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:47.818332     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:48.317476     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:48.823233     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:49.323384     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:49.817349     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:50.323027     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0427 11:07:50.818251     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:51.316912     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:51.821880     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:52.321882     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:52.821838     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:07:53.316678     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:53.824586     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:54.324069     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:07:54.817176     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:55.317022     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:55.822469     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:56.319542     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:56.817783     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:57.323958     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:57.820361     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:58.317110     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:58.818922     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:07:59.319032     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:07:59.821054     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:00.316623     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:00.819378     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0427 11:08:01.322042     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:01.818946     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:02.319903     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:02.824312     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:03.320795     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:03.816675     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:04.323297     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:04.823039     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:05.317623     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:05.816430     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:06.324785     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:06.819457     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:07.316295     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:07.822199     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:08.319605     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:08.816257     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:09.316964     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:09.819168     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:10.323450     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:10.816550     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:11.316910     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:11.818859     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:12.317833     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:12.817145     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:13.318528     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:13.819137     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:14.316196     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:14.819404     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:15.319943     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:15.817061     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:16.316797     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:16.823159     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:17.321248     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:17.816118     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:18.322050     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:18.818836     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:19.316595     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:19.816461     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:20.319940     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:20.822393     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0427 11:08:21.317224     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:21.818454     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:22.323473     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:22.816267     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:23.317433     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:23.824284     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:24.323284     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:08:24.816252     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:25.325616     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0427 11:08:25.819574     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:26.319043     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:26.817216     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:27.323054     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:27.819741     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:28.316234     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:28.819735     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:29.319493     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:29.816379     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:30.317462     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:30.821716     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:31.321720     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:31.816607     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:32.319402     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:32.818253     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:33.316259     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:33.816365     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:34.323332     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:34.826241     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 4 milliseconds
I0427 11:08:35.317415     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:35.823157     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:36.319711     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:36.816448     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:37.315950     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:37.822555     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:38.322985     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:38.815900     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:39.317285     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:39.823722     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:40.316641     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:40.818462     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:41.319838     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:41.820097     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:42.315632     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:42.819643     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:43.318534     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:43.815949     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:44.315735     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:44.816657     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:45.323428     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:45.816728     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:46.318623     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:46.822952     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:47.316109     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:47.815855     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:48.318422     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:48.821683     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:49.316536     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:49.819204     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:50.319998     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:50.816082     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:51.316201     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:51.817027     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:52.321288     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:52.817069     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:53.319695     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:53.822404     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:54.316198     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:54.818565     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:55.319625     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:55.818995     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:56.318037     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:56.821023     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:08:57.320723     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:57.816630     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:58.315970     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:58.818277     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:08:59.321324     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:08:59.816871     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:09:00.323486     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0427 11:09:00.819294     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.

Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
couldn't initialize a Kubernetes cluster
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/init.runWaitControlPlanePhase
	cmd/kubeadm/app/cmd/phases/init/waitcontrolplane.go:108
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:259
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:112
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594
error execution phase wait-control-plane
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:260
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:112
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594
Stack Trace:
sigs.k8s.io/kind/pkg/errors.WithStack
	sigs.k8s.io/kind/pkg/errors/errors.go:59
sigs.k8s.io/kind/pkg/exec.(*LocalCmd).Run
	sigs.k8s.io/kind/pkg/exec/local.go:124
sigs.k8s.io/kind/pkg/cluster/internal/providers/docker.(*nodeCmd).Run
	sigs.k8s.io/kind/pkg/cluster/internal/providers/docker/node.go:146
sigs.k8s.io/kind/pkg/exec.CombinedOutputLines
	sigs.k8s.io/kind/pkg/exec/helpers.go:67
sigs.k8s.io/kind/pkg/cluster/internal/create/actions/kubeadminit.(*action).Execute
	sigs.k8s.io/kind/pkg/cluster/internal/create/actions/kubeadminit/init.go:81
sigs.k8s.io/kind/pkg/cluster/internal/create.Cluster
	sigs.k8s.io/kind/pkg/cluster/internal/create/create.go:135
sigs.k8s.io/kind/pkg/cluster.(*Provider).Create
	sigs.k8s.io/kind/pkg/cluster/provider.go:182
sigs.k8s.io/kind/pkg/cmd/kind/create/cluster.runE
	sigs.k8s.io/kind/pkg/cmd/kind/create/cluster/createcluster.go:111
sigs.k8s.io/kind/pkg/cmd/kind/create/cluster.NewCommand.func1
	sigs.k8s.io/kind/pkg/cmd/kind/create/cluster/createcluster.go:55
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/cobra@v1.4.0/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/cobra@v1.4.0/command.go:974
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/cobra@v1.4.0/command.go:902
sigs.k8s.io/kind/cmd/kind/app.Run
	sigs.k8s.io/kind/cmd/kind/app/main.go:53
sigs.k8s.io/kind/cmd/kind/app.Main
	sigs.k8s.io/kind/cmd/kind/app/main.go:35
main.main
	sigs.k8s.io/kind/main.go:25
runtime.main
	runtime/proc.go:250
runtime.goexit
	runtime/asm_arm64.s:1172

After removing linux/amd64 config

❯ echo $DOCKER_DEFAULT_PLATFORM

❯ kind create cluster -v 9
Creating cluster "kind" ...
DEBUG: docker/images.go:67] Pulling image: kindest/node:v1.26.3@sha256:61b92f38dff6ccc29969e7aa154d34e38b89443af1a2c14e6cfbd2df6419c66f ...
 ✓ Ensuring node image (kindest/node:v1.26.3) 🖼
 ✓ Preparing nodes 📦
DEBUG: config/config.go:96] Using the following kubeadm config for node kind-control-plane:
apiServer:
  certSANs:
  - localhost
  - 127.0.0.1
  extraArgs:
    runtime-config: ""
apiVersion: kubeadm.k8s.io/v1beta3
clusterName: kind
controlPlaneEndpoint: kind-control-plane:6443
controllerManager:
  extraArgs:
    enable-hostpath-provisioner: "true"
kind: ClusterConfiguration
kubernetesVersion: v1.26.3
networking:
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/16
scheduler:
  extraArgs: null
---
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- token: abcdef.0123456789abcdef
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 172.19.0.2
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
  kubeletExtraArgs:
    node-ip: 172.19.0.2
    node-labels: ""
    provider-id: kind://docker/kind/kind-control-plane
---
apiVersion: kubeadm.k8s.io/v1beta3
controlPlane:
  localAPIEndpoint:
    advertiseAddress: 172.19.0.2
    bindPort: 6443
discovery:
  bootstrapToken:
    apiServerEndpoint: kind-control-plane:6443
    token: abcdef.0123456789abcdef
    unsafeSkipCAVerification: true
kind: JoinConfiguration
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
  kubeletExtraArgs:
    node-ip: 172.19.0.2
    node-labels: ""
    provider-id: kind://docker/kind/kind-control-plane
---
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: systemd
cgroupRoot: /kubelet
evictionHard:
  imagefs.available: 0%
  nodefs.available: 0%
  nodefs.inodesFree: 0%
failSwapOn: false
imageGCHighThresholdPercent: 100
kind: KubeletConfiguration
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
conntrack:
  maxPerCore: 0
iptables:
  minSyncPeriod: 1s
kind: KubeProxyConfiguration
mode: iptables
 ✓ Writing configuration 📜
DEBUG: kubeadminit/init.go:82] I0427 11:50:19.812566     131 initconfiguration.go:254] loading configuration from "/kind/kubeadm.conf"
W0427 11:50:19.813678     131 initconfiguration.go:331] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
[init] Using Kubernetes version: v1.26.3
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0427 11:50:19.819623     131 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0427 11:50:20.006285     131 certs.go:519] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.19.0.2 127.0.0.1]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0427 11:50:20.282039     131 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0427 11:50:20.402786     131 certs.go:519] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0427 11:50:20.681686     131 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0427 11:50:20.981755     131 certs.go:519] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.19.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.19.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0427 11:50:21.720164     131 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
I0427 11:50:21.861357     131 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
I0427 11:50:22.303289     131 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0427 11:50:22.358430     131 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0427 11:50:22.525076     131 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0427 11:50:22.605981     131 kubelet.go:67] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0427 11:50:22.916477     131 manifests.go:99] [control-plane] getting StaticPodSpecs
I0427 11:50:22.916803     131 certs.go:519] validating certificate period for CA certificate
I0427 11:50:22.916849     131 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0427 11:50:22.916853     131 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0427 11:50:22.916856     131 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0427 11:50:22.916858     131 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0427 11:50:22.916860     131 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0427 11:50:22.918717     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I0427 11:50:22.918738     131 manifests.go:99] [control-plane] getting StaticPodSpecs
I0427 11:50:22.918886     131 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0427 11:50:22.918948     131 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0427 11:50:22.918959     131 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0427 11:50:22.918969     131 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0427 11:50:22.918978     131 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0427 11:50:22.918989     131 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0427 11:50:22.918999     131 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I0427 11:50:22.919425     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I0427 11:50:22.919498     131 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0427 11:50:22.919682     131 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0427 11:50:22.920194     131 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0427 11:50:22.921087     131 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0427 11:50:22.921104     131 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0427 11:50:22.921856     131 loader.go:373] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0427 11:50:22.924940     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0427 11:50:23.425826     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:50:23.925834     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:50:24.426488     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:50:24.925592     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:50:25.425897     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:50:25.926825     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:50:26.426406     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0427 11:50:29.469463     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 500 Internal Server Error in 2543 milliseconds
I0427 11:50:29.929031     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 500 Internal Server Error in 2 milliseconds
I0427 11:50:30.428918     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 500 Internal Server Error in 2 milliseconds
I0427 11:50:30.927591     131 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s 200 OK in 0 milliseconds
[apiclient] All control plane components are healthy after 8.004318 seconds
I0427 11:50:30.927840     131 uploadconfig.go:111] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0427 11:50:30.930908     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:30.932659     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:30.935395     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 2 milliseconds
I0427 11:50:30.935639     131 uploadconfig.go:125] [upload-config] Uploading the kubelet component config to a ConfigMap
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0427 11:50:30.938306     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:30.939569     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:30.940553     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 0 milliseconds
I0427 11:50:30.940615     131 uploadconfig.go:130] [upload-config] Preserving the CRISocket information for the control-plane node
I0427 11:50:30.940694     131 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///run/containerd/containerd.sock" to the Node API object "kind-control-plane" as an annotation
I0427 11:50:31.447107     131 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s 200 OK in 4 milliseconds
I0427 11:50:31.462493     131 round_trippers.go:553] PATCH https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s 200 OK in 11 milliseconds
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node kind-control-plane as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node kind-control-plane as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
I0427 11:50:31.970927     131 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s 200 OK in 6 milliseconds
I0427 11:50:31.983486     131 round_trippers.go:553] PATCH https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s 200 OK in 9 milliseconds
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0427 11:50:31.988123     131 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/secrets/bootstrap-token-abcdef?timeout=10s 404 Not Found in 3 milliseconds
I0427 11:50:31.993085     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/secrets?timeout=10s 201 Created in 4 milliseconds
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0427 11:50:31.995822     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 2 milliseconds
I0427 11:50:31.997807     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0427 11:50:31.999767     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0427 11:50:32.001760     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0427 11:50:32.003200     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0427 11:50:32.003297     131 clusterinfo.go:47] [bootstrap-token] loading admin kubeconfig
I0427 11:50:32.003892     131 loader.go:373] Config loaded from file:  /etc/kubernetes/admin.conf
I0427 11:50:32.003909     131 clusterinfo.go:58] [bootstrap-token] copying the cluster from admin.conf to the bootstrap kubeconfig
I0427 11:50:32.004115     131 clusterinfo.go:70] [bootstrap-token] creating/updating ConfigMap in kube-public namespace
I0427 11:50:32.006817     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps?timeout=10s 201 Created in 2 milliseconds
I0427 11:50:32.007004     131 clusterinfo.go:84] creating the RBAC rules for exposing the cluster-info ConfigMap in the kube-public namespace
I0427 11:50:32.009587     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s 201 Created in 2 milliseconds
I0427 11:50:32.011629     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s 201 Created in 1 milliseconds
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0427 11:50:32.011789     131 kubeletfinalize.go:90] [kubelet-finalize] Assuming that kubelet client certificate rotation is enabled: found "/var/lib/kubelet/pki/kubelet-client-current.pem"
I0427 11:50:32.012391     131 loader.go:373] Config loaded from file:  /etc/kubernetes/kubelet.conf
I0427 11:50:32.012911     131 kubeletfinalize.go:134] [kubelet-finalize] Restarting the kubelet to enable client certificate rotation
I0427 11:50:32.166002     131 round_trippers.go:553] GET https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=k8s-app%3Dkube-dns 200 OK in 2 milliseconds
I0427 11:50:32.168386     131 round_trippers.go:553] GET https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/coredns?timeout=10s 404 Not Found in 0 milliseconds
I0427 11:50:32.170031     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:32.171776     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:32.173195     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:32.175392     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:32.179288     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/deployments?timeout=10s 201 Created in 3 milliseconds
I0427 11:50:32.183045     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/services?timeout=10s 201 Created in 3 milliseconds
[addons] Applied essential addon: CoreDNS
I0427 11:50:32.185238     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:32.188411     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/daemonsets?timeout=10s 201 Created in 2 milliseconds
I0427 11:50:32.189977     131 round_trippers.go:553] POST https://kind-control-plane:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:32.191279     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:32.195667     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 1 milliseconds
I0427 11:50:32.394408     131 request.go:622] Waited for 198.536083ms due to client-side throttling, not priority and fairness, request: POST:https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s
I0427 11:50:32.395923     131 round_trippers.go:553] POST https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 1 milliseconds
[addons] Applied essential addon: kube-proxy
I0427 11:50:32.396416     131 loader.go:373] Config loaded from file:  /etc/kubernetes/admin.conf
I0427 11:50:32.396811     131 loader.go:373] Config loaded from file:  /etc/kubernetes/admin.conf

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join kind-control-plane:6443 --token <value withheld> \
	--discovery-token-ca-cert-hash sha256:b881aea971d08c51dbdfab282da5a22ed106a25f00040bfb3cf9c9bb837b3eb2 \
	--control-plane

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join kind-control-plane:6443 --token <value withheld> \
	--discovery-token-ca-cert-hash sha256:b881aea971d08c51dbdfab282da5a22ed106a25f00040bfb3cf9c9bb837b3eb2
 ✓ Starting control-plane 🕹️
DEBUG: installcni/cni.go:121] Using the following Kindnetd config:
null
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kindnet
rules:
- apiGroups:
  - policy
  resourceNames:
  - kindnet
  resources:
  - podsecuritypolicies
  verbs:
  - use
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kindnet
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kindnet
subjects:
- kind: ServiceAccount
  name: kindnet
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kindnet
  namespace: kube-system
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    app: kindnet
    k8s-app: kindnet
    tier: node
  name: kindnet
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: kindnet
  template:
    metadata:
      labels:
        app: kindnet
        k8s-app: kindnet
        tier: node
    spec:
      containers:
      - env:
        - name: HOST_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: POD_SUBNET
          value: 10.244.0.0/16
        - name: CONTROL_PLANE_ENDPOINT
          value: kind-control-plane:6443
        image: docker.io/kindest/kindnetd:v20230330-48f316cd@sha256:c19d6362a6a928139820761475a38c24c0cf84d507b9ddf414a078cf627497af
        name: kindnet-cni
        resources:
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 100m
            memory: 50Mi
        securityContext:
          capabilities:
            add:
            - NET_RAW
            - NET_ADMIN
          privileged: false
        volumeMounts:
        - mountPath: /etc/cni/net.d
          name: cni-cfg
        - mountPath: /run/xtables.lock
          name: xtables-lock
          readOnly: false
        - mountPath: /lib/modules
          name: lib-modules
          readOnly: true
      hostNetwork: true
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: kindnet
      tolerations:
      - operator: Exists
      volumes:
      - hostPath:
          path: /etc/cni/net.d
        name: cni-cfg
      - hostPath:
          path: /run/xtables.lock
          type: FileOrCreate
        name: xtables-lock
      - hostPath:
          path: /lib/modules
        name: lib-modules
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂

@BenTheElder
Copy link
Member

@vallpaper that would be #2718

@SharangC96
Copy link

SharangC96 commented May 16, 2023 •
edited

@BenTheElder
I am running into the same issue

docker info

Client:
Context:    default
Debug Mode: false
Plugins:
 buildx: Docker Buildx (Docker Inc.)
   Version:  v0.10.4
   Path:     /Users/sharangc/.docker/cli-plugins/docker-buildx
 compose: Docker Compose (Docker Inc.)
   Version:  v2.17.3
   Path:     /Users/sharangc/.docker/cli-plugins/docker-compose
 dev: Docker Dev Environments (Docker Inc.)
   Version:  v0.1.0
   Path:     /Users/sharangc/.docker/cli-plugins/docker-dev
 extension: Manages Docker extensions (Docker Inc.)
   Version:  v0.2.19
   Path:     /Users/sharangc/.docker/cli-plugins/docker-extension
 init: Creates Docker-related starter files for your project (Docker Inc.)
   Version:  v0.1.0-beta.4
   Path:     /Users/sharangc/.docker/cli-plugins/docker-init
 sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
   Version:  0.6.0
   Path:     /Users/sharangc/.docker/cli-plugins/docker-sbom
 scan: Docker Scan (Docker Inc.)
   Version:  v0.26.0
   Path:     /Users/sharangc/.docker/cli-plugins/docker-scan
 scout: Command line tool for Docker Scout (Docker Inc.)
   Version:  v0.10.0
   Path:     /Users/sharangc/.docker/cli-plugins/docker-scout

Server:
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 32
Server Version: 23.0.5
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Using metacopy: false
 Native Overlay Diff: true
 userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
runc version: v1.1.5-0-gf19387a
init version: de40ad0
Security Options:
 seccomp
  Profile: builtin
 cgroupns
Kernel Version: 5.15.49-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 6
Total Memory: 11.68GiB
Name: docker-desktop
ID: b74a879f-8fec-4315-9793-3b0a6a6e7fe9
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 hubproxy.docker.internal:5555
 127.0.0.0/8
Live Restore Enabled: false

kind --version
kind version 0.18.0

kubectl version

Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.9", GitCommit:"a1a87a0a2bcd605820920c6b0e618a8ab7d117d4", GitTreeState:"clean", BuildDate:"2023-04-12T12:16:51Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7```


cat kind-example-config.yaml 
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: KubeletConfiguration
    cgroupDriver: systemd


kind create cluster --config kind-example-config.yaml  
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.26.3) 🖼
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✗ Starting control-plane 🕹️ 
Deleted nodes: ["kind-control-plane"]
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged kind-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I0516 10:23:18.829132     134 initconfiguration.go:254] loading configuration from "/kind/kubeadm.conf"
W0516 10:23:18.839290     134 initconfiguration.go:331] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
[init] Using Kubernetes version: v1.26.3
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0516 10:23:18.877071     134 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I0516 10:23:19.008849     134 certs.go:519] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.18.0.2 127.0.0.1]
[certs] Generating "apiserver-kubelet-client" certificate and key
I0516 10:23:19.332211     134 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I0516 10:23:19.537270     134 certs.go:519] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I0516 10:23:19.657280     134 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I0516 10:23:19.774009     134 certs.go:519] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.18.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.18.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I0516 10:23:20.457838     134 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0516 10:23:20.704162     134 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I0516 10:23:20.888711     134 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I0516 10:23:21.030578     134 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0516 10:23:21.179077     134 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I0516 10:23:21.387066     134 kubelet.go:67] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0516 10:23:21.944153     134 manifests.go:99] [control-plane] getting StaticPodSpecs
I0516 10:23:21.946386     134 certs.go:519] validating certificate period for CA certificate
I0516 10:23:21.946717     134 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0516 10:23:21.946747     134 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0516 10:23:21.946752     134 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0516 10:23:21.946755     134 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0516 10:23:21.946759     134 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
I0516 10:23:21.949669     134 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I0516 10:23:21.949688     134 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0516 10:23:21.949932     134 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0516 10:23:21.949942     134 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0516 10:23:21.949945     134 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0516 10:23:21.949949     134 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0516 10:23:21.949952     134 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0516 10:23:21.949955     134 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0516 10:23:21.949958     134 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I0516 10:23:21.950572     134 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I0516 10:23:21.950587     134 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0516 10:23:21.950740     134 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I0516 10:23:21.951143     134 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0516 10:23:21.969972     134 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0516 10:23:21.970084     134 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I0516 10:23:21.971163     134 loader.go:373] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0516 10:23:21.987137     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 7 milliseconds
I0516 10:23:22.491754     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:22.993934     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:23.495966     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:23.990646     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:24.495003     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:24.995580     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:23:25.491272     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:25.992611     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:23:26.497639     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:26.991089     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:27.491418     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:27.993337     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:28.492575     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:28.990535     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:29.494626     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0516 10:23:29.992821     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:30.490087     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:30.995983     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:31.496238     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:31.992743     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:32.492922     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:32.994387     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:33.491071     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:33.990365     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:34.494657     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:34.990165     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:35.491873     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:35.993185     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:36.496482     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:23:36.990067     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:37.491105     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:37.993238     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:38.492511     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:38.991538     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:23:39.493418     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:39.994286     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0516 10:23:40.490083     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:40.990759     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:41.495445     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:41.990770     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:42.490906     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:42.995089     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:43.489957     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:43.995080     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:44.497967     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:44.992023     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:45.490399     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:45.995465     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:46.490561     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:46.990621     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:47.493700     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:47.990970     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:48.495401     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:48.995220     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:23:49.490523     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:49.994018     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:50.496200     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:50.995104     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:51.490498     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:51.992524     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:52.490838     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:52.991843     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:53.492769     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:53.994864     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:54.489897     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:54.993433     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:23:55.496117     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:23:55.991383     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:56.494737     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:56.995393     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:57.491858     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:57.990901     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:58.495390     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:58.993469     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:59.494592     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:23:59.996786     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0516 10:24:00.489674     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:00.994469     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:01.496309     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0516 10:24:01.990411     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:02.494842     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:02.992457     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:03.496111     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:03.992350     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:04.491533     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:04.991390     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:05.490734     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:05.993592     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:06.496065     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0516 10:24:06.990295     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:07.492548     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:07.993230     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:08.490314     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:08.996079     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:09.496653     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0516 10:24:09.989698     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:10.493286     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:10.997239     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 3 milliseconds
I0516 10:24:11.492653     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:11.997636     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:12.496979     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:12.989810     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:13.496692     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:13.994384     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:14.490048     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:14.995046     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:15.494363     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:15.989608     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:16.496659     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:16.989903     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0516 10:24:17.490162     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:17.992933     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:18.491054     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:18.989815     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:19.492260     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:19.992070     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:20.494347     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:20.992762     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:21.497600     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:21.989844     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:22.494616     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:22.992666     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:23.490944     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:23.992119     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:24.492203     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:24.992132     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:25.495671     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:25.995623     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:26.494719     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:26.990082     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:27.493976     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:27.990362     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:28.495865     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:28.992071     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:29.491314     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:29.991154     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:30.496549     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:30.990359     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:31.495322     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:31.996466     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:32.494720     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:32.995044     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:33.492560     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:33.992949     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:34.490567     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:34.994975     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:35.490813     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:35.995216     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:36.496772     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0516 10:24:36.992040     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:37.491212     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:37.997425     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:38.490173     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:38.991893     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:39.492412     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:39.996408     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:40.490243     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:40.995888     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:41.497670     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:41.990918     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:42.492942     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:42.992445     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:43.496078     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:43.991132     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:44.492533     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:44.993907     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:45.490135     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:45.996496     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:46.497189     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:46.990219     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:47.497170     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:47.996056     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:48.490058     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:48.997235     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:49.497492     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:49.990936     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:50.495898     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:50.992780     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:24:51.491586     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:51.994596     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:52.493016     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:52.991558     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:53.495224     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:53.997283     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:54.491372     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:54.994954     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:55.493317     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:55.993180     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:56.491077     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:56.997235     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:57.496063     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:57.991289     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:58.494748     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:58.996396     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:24:59.491130     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:24:59.997960     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:00.497637     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:00.991189     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:01.492733     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:25:01.995336     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:02.491915     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:02.997453     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:03.494060     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:03.991948     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:04.495345     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:04.993453     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:05.491049     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:05.991245     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:06.493950     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:25:06.990522     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:07.491340     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:07.994853     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:08.492097     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:25:08.991058     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:09.492882     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:25:09.995967     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:10.490260     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:10.995659     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:11.494992     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:25:11.993886     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:12.494624     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:12.992576     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 1 milliseconds
I0516 10:25:13.491388     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:13.994602     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:14.495271     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:14.991833     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:15.491540     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I0516 10:25:15.997048     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:16.492438     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 2 milliseconds
I0516 10:25:16.990296     134 round_trippers.go:553] GET https://kind-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.

Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
couldn't initialize a Kubernetes cluster
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/init.runWaitControlPlanePhase
	cmd/kubeadm/app/cmd/phases/init/waitcontrolplane.go:108
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:259
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:112
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594
error execution phase wait-control-plane
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:260
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:112
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1594

@BenTheElder
Copy link
Member

@SharangC96 can you please share the output of kind export logs after kind create cluster --retain, then you can clean up with kind delete cluster. You can zip or tar the exported log folder and drag and drop it to a github comment.

re:

cat kind-example-config.yaml 
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: KubeletConfiguration
    cgroupDriver: systemd

You should not need to set this. See: #3061 (comment), this is the default.

Are you seeing that this is not the case?

Also, if you're setting DOCKER_DEFAULT_PLATFORM, this is not the bug. See #2718

@hadrabap hadrabap mentioned this issue May 17, 2023
Open
@codyjroberts
Copy link

@BenTheElder can report that I also started running into this same problem where the health check fails when setting up the control plane.

Recently returned to a project, after not touching it for a few months. It was utilizing docker desktop and kind without issue on macos ARM. Docker desktop logs were reporting issues with privileged ports (which were enabled).

Tried:

  • podman, same issue
  • cgroup patch, same issue
  • upgrading kind 17.x to 18.x, same issue (old I know, just what nixpkgs supplies on the latest channels)

Then I went back to docker desktop (v4.21.1) and it worked. I suspected updating the nix channel in my project did the trick, it bumped a few deps.

  • kubectl client 1.25.4 -> 1.27.1
  • kubectl server 1.25.3 -> 1.26.3
  • kind 0.17.0 go1.19.9 darwin/arm64 -> 0.18.0 go1.20.6 darwin/arm64
  • kindest/node:1.25.3 -> kindest/node:v1.26.3

So I dropped back down and it still works. Removed the cgroup patch, still works. So I'm a bit clueless as to what resolved it. 🤷

Hopefully this is useful info. I'm unstuck for now, so no worries here.

@YTGhost YTGhost mentioned this issue Sep 8, 2023
Closed
@BenTheElder
Copy link
Member

I think this has gone stale, if you see something like this with current releases: Please file a new bug with all the details requested in the bug template and we'll investigate. This thread has diverged into one unclear issue and other unrelated issues.

If we think future bugs filings are related we can / will cross link back here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@BenTheElder @vallpaper @stmcginnis @codyjroberts @matrus2 @fengqi2010 @ep4sh @SharangC96 @alex-richter