pivot control from bootstrap cluster to the management cluster #8214

wilderk · 2023-03-01T21:27:01Z

wilderk
Mar 1, 2023

I am attempting to pivot control from bootstrap cluster to the management cluster following the move documentation . Using latest clusterctl v1.3.4. When I attempt to init the management cluster it hangs waiting for cert-manager.

clusterctl init --infrastructure aws
Fetching providers
Installing cert-manager Version="v1.11.0"
Waiting for cert-manager to be available...
Error: timed out waiting for the condition

I know that cert-manager is running and ready

kubectl get pods -A
NAMESPACE      NAME                                                                 READY   STATUS    RESTARTS   AGE
cert-manager   cert-manager-99bb69456-j9f7k                                         1/1     Running   0          31m
cert-manager   cert-manager-cainjector-ffb4747bb-4zc8t                              1/1     Running   0          31m
cert-manager   cert-manager-webhook-545bd5d7d8-6qg6c                                1/1     Running   0          31m
kube-flannel   kube-flannel-ds-6jmxk                                                1/1     Running   0          34m
kube-flannel   kube-flannel-ds-msnxw                                                1/1     Running   0          34m
kube-flannel   kube-flannel-ds-pj5k6                                                1/1     Running   0          34m
kube-system    coredns-565d847f94-ngp94                                             1/1     Running   0          37m
kube-system    coredns-565d847f94-nnc88                                             1/1     Running   0          37m
kube-system    etcd-ip-XX-X-XXX-XXX.us-east-2.compute.internal                      1/1     Running   0          37m
kube-system    kube-apiserver-ip-XX-X-XXX-XXX.us-east-2.compute.internal            1/1     Running   0          37m
kube-system    kube-controller-manager-ip-XX-X-XXX-XXX.us-east-2.compute.internal   1/1     Running   0          37m
kube-system    kube-proxy-prrb4                                                     1/1     Running   0          37m
kube-system    kube-proxy-rsgtf                                                     1/1     Running   0          37m
kube-system    kube-proxy-sr976                                                     1/1     Running   0          37m
kube-system    kube-scheduler-ip-XX-X-XXX-XXX.us-east-2.compute.internal            1/1     Running   0          37m

The management cluster was created with this command and config and I did not modify the generated manifest

clusterctl generate cluster wilder-capi-kubeadm --config clusterconfig-aws-kubeadm.yaml > wilder-capi-kubeadm.yaml

# Required Variables
AWS_CONTROL_PLANE_MACHINE_TYPE: t2.2xlarge
AWS_NODE_MACHINE_TYPE: t2.2xlarge
AWS_REGION: us-east-2
AWS_SSH_KEY_NAME: wilder-capi-kubeadm
KUBERNETES_VERSION: v1.25.6
# Optional Variables
CONTROL_PLANE_MACHINE_COUNT: 1
WORKER_MACHINE_COUNT: 2

Does any one have any ideas about why the clusterctl init thinks cert-manager is not ready?
I did not have this problem when I did the bootstrap cluster init.

Answered by wilderk

Mar 11, 2023

The cause of the problem is the node security group. I opened an issue in cluster-api-provider-aws
kubernetes-sigs/cluster-api-provider-aws#4136

View full answer

wilderk · 2023-03-01T23:01:14Z

wilderk
Mar 1, 2023
Author

logs from bootstrap init

clusterctl init --infrastructure aws -v5

No default config file available
Installing the clusterctl inventory CRD
Creating CustomResourceDefinition="providers.clusterctl.cluster.x-k8s.io"
Fetching providers
Fetching File="core-components.yaml" Provider="cluster-api" Type="CoreProvider" Version="v1.3.4"
Fetching File="bootstrap-components.yaml" Provider="kubeadm" Type="BootstrapProvider" Version="v1.3.4"
Fetching File="control-plane-components.yaml" Provider="kubeadm" Type="ControlPlaneProvider" Version="v1.3.4"
Fetching File="infrastructure-components.yaml" Provider="aws" Type="InfrastructureProvider" Version="v2.0.2"
Fetching File="metadata.yaml" Provider="cluster-api" Type="CoreProvider" Version="v1.3.4"
Fetching File="metadata.yaml" Provider="kubeadm" Type="BootstrapProvider" Version="v1.3.4"
Fetching File="metadata.yaml" Provider="kubeadm" Type="ControlPlaneProvider" Version="v1.3.4"
Fetching File="metadata.yaml" Provider="aws" Type="InfrastructureProvider" Version="v2.0.2"
Creating Namespace="cert-manager-test"
Installing cert-manager Version="v1.11.0"
Fetching File="cert-manager.yaml" Provider="cert-manager" Type="" Version="v1.11.0"
Creating Namespace="cert-manager"
Creating CustomResourceDefinition="clusterissuers.cert-manager.io"
Creating CustomResourceDefinition="challenges.acme.cert-manager.io"
Creating CustomResourceDefinition="certificaterequests.cert-manager.io"
Creating CustomResourceDefinition="issuers.cert-manager.io"
Creating CustomResourceDefinition="certificates.cert-manager.io"
Creating CustomResourceDefinition="orders.acme.cert-manager.io"
Creating ConfigMap="cert-manager-webhook" Namespace="cert-manager"
Creating ServiceAccount="cert-manager-cainjector" Namespace="cert-manager"
Creating ServiceAccount="cert-manager" Namespace="cert-manager"
Creating ServiceAccount="cert-manager-webhook" Namespace="cert-manager"
Creating ClusterRole="cert-manager-cainjector"
Creating ClusterRole="cert-manager-controller-issuers"
Creating ClusterRole="cert-manager-controller-clusterissuers"
Creating ClusterRole="cert-manager-controller-certificates"
Creating ClusterRole="cert-manager-controller-orders"
Creating ClusterRole="cert-manager-controller-challenges"
Creating ClusterRole="cert-manager-controller-ingress-shim"
Creating ClusterRole="cert-manager-view"
Creating ClusterRole="cert-manager-edit"
Creating ClusterRole="cert-manager-controller-approve:cert-manager-io"
Creating ClusterRole="cert-manager-controller-certificatesigningrequests"
Creating ClusterRole="cert-manager-webhook:subjectaccessreviews"
Creating ClusterRoleBinding="cert-manager-cainjector"
Creating ClusterRoleBinding="cert-manager-controller-issuers"
Creating ClusterRoleBinding="cert-manager-controller-clusterissuers"
Creating ClusterRoleBinding="cert-manager-controller-certificates"
Creating ClusterRoleBinding="cert-manager-controller-orders"
Creating ClusterRoleBinding="cert-manager-controller-challenges"
Creating ClusterRoleBinding="cert-manager-controller-ingress-shim"
Creating ClusterRoleBinding="cert-manager-controller-approve:cert-manager-io"
Creating ClusterRoleBinding="cert-manager-controller-certificatesigningrequests"
Creating ClusterRoleBinding="cert-manager-webhook:subjectaccessreviews"
Creating Role="cert-manager-cainjector:leaderelection" Namespace="kube-system"
Creating Role="cert-manager:leaderelection" Namespace="kube-system"
Creating Role="cert-manager-webhook:dynamic-serving" Namespace="cert-manager"
Creating RoleBinding="cert-manager-cainjector:leaderelection" Namespace="kube-system"
Creating RoleBinding="cert-manager:leaderelection" Namespace="kube-system"
Creating RoleBinding="cert-manager-webhook:dynamic-serving" Namespace="cert-manager"
Creating Service="cert-manager" Namespace="cert-manager"
Creating Service="cert-manager-webhook" Namespace="cert-manager"
Creating Deployment="cert-manager-cainjector" Namespace="cert-manager"
Creating Deployment="cert-manager" Namespace="cert-manager"
Creating Deployment="cert-manager-webhook" Namespace="cert-manager"
Creating MutatingWebhookConfiguration="cert-manager-webhook"
Creating ValidatingWebhookConfiguration="cert-manager-webhook"
Waiting for cert-manager to be available...
Updating Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Certificate="selfsigned-cert" Namespace="cert-manager-test"
Deleting Namespace="cert-manager-test"
Deleting Issuer="test-selfsigned" Namespace="cert-manager-test"
Deleting Certificate="selfsigned-cert" Namespace="cert-manager-test"
Installing Provider="cluster-api" Version="v1.3.4" TargetNamespace="capi-system"
Creating objects Provider="cluster-api" Version="v1.3.4" TargetNamespace="capi-system"
Creating Namespace="capi-system"
Creating CustomResourceDefinition="clusterclasses.cluster.x-k8s.io"
Creating CustomResourceDefinition="clusterresourcesetbindings.addons.cluster.x-k8s.io"
Creating CustomResourceDefinition="clusterresourcesets.addons.cluster.x-k8s.io"
Creating CustomResourceDefinition="clusters.cluster.x-k8s.io"
Creating CustomResourceDefinition="extensionconfigs.runtime.cluster.x-k8s.io"
Creating CustomResourceDefinition="ipaddressclaims.ipam.cluster.x-k8s.io"
Creating CustomResourceDefinition="ipaddresses.ipam.cluster.x-k8s.io"
Creating CustomResourceDefinition="machinedeployments.cluster.x-k8s.io"
Creating CustomResourceDefinition="machinehealthchecks.cluster.x-k8s.io"
Creating CustomResourceDefinition="machinepools.cluster.x-k8s.io"
Creating CustomResourceDefinition="machines.cluster.x-k8s.io"
Creating CustomResourceDefinition="machinesets.cluster.x-k8s.io"
Creating ServiceAccount="capi-manager" Namespace="capi-system"
Creating Role="capi-leader-election-role" Namespace="capi-system"
Creating ClusterRole="capi-system-capi-aggregated-manager-role"
Creating ClusterRole="capi-system-capi-manager-role"
Creating RoleBinding="capi-leader-election-rolebinding" Namespace="capi-system"
Creating ClusterRoleBinding="capi-system-capi-manager-rolebinding"
Creating Service="capi-webhook-service" Namespace="capi-system"
Creating Deployment="capi-controller-manager" Namespace="capi-system"
Creating Certificate="capi-serving-cert" Namespace="capi-system"
Creating Issuer="capi-selfsigned-issuer" Namespace="capi-system"
Creating MutatingWebhookConfiguration="capi-mutating-webhook-configuration"
Creating ValidatingWebhookConfiguration="capi-validating-webhook-configuration"
Creating inventory entry Provider="cluster-api" Version="v1.3.4" TargetNamespace="capi-system"
Installing Provider="bootstrap-kubeadm" Version="v1.3.4" TargetNamespace="capi-kubeadm-bootstrap-system"
Creating objects Provider="bootstrap-kubeadm" Version="v1.3.4" TargetNamespace="capi-kubeadm-bootstrap-system"
Creating Namespace="capi-kubeadm-bootstrap-system"
Creating CustomResourceDefinition="kubeadmconfigs.bootstrap.cluster.x-k8s.io"
Creating CustomResourceDefinition="kubeadmconfigtemplates.bootstrap.cluster.x-k8s.io"
Creating ServiceAccount="capi-kubeadm-bootstrap-manager" Namespace="capi-kubeadm-bootstrap-system"
Creating Role="capi-kubeadm-bootstrap-leader-election-role" Namespace="capi-kubeadm-bootstrap-system"
Creating ClusterRole="capi-kubeadm-bootstrap-system-capi-kubeadm-bootstrap-manager-role"
Creating RoleBinding="capi-kubeadm-bootstrap-leader-election-rolebinding" Namespace="capi-kubeadm-bootstrap-system"
Creating ClusterRoleBinding="capi-kubeadm-bootstrap-system-capi-kubeadm-bootstrap-manager-rolebinding"
Creating Service="capi-kubeadm-bootstrap-webhook-service" Namespace="capi-kubeadm-bootstrap-system"
Creating Deployment="capi-kubeadm-bootstrap-controller-manager" Namespace="capi-kubeadm-bootstrap-system"
Creating Certificate="capi-kubeadm-bootstrap-serving-cert" Namespace="capi-kubeadm-bootstrap-system"
Creating Issuer="capi-kubeadm-bootstrap-selfsigned-issuer" Namespace="capi-kubeadm-bootstrap-system"
Creating MutatingWebhookConfiguration="capi-kubeadm-bootstrap-mutating-webhook-configuration"
Creating ValidatingWebhookConfiguration="capi-kubeadm-bootstrap-validating-webhook-configuration"
Creating inventory entry Provider="bootstrap-kubeadm" Version="v1.3.4" TargetNamespace="capi-kubeadm-bootstrap-system"
Installing Provider="control-plane-kubeadm" Version="v1.3.4" TargetNamespace="capi-kubeadm-control-plane-system"
Creating objects Provider="control-plane-kubeadm" Version="v1.3.4" TargetNamespace="capi-kubeadm-control-plane-system"
Creating Namespace="capi-kubeadm-control-plane-system"
Creating CustomResourceDefinition="kubeadmcontrolplanes.controlplane.cluster.x-k8s.io"
Creating CustomResourceDefinition="kubeadmcontrolplanetemplates.controlplane.cluster.x-k8s.io"
Creating ServiceAccount="capi-kubeadm-control-plane-manager" Namespace="capi-kubeadm-control-plane-system"
Creating Role="capi-kubeadm-control-plane-leader-election-role" Namespace="capi-kubeadm-control-plane-system"
Creating ClusterRole="capi-kubeadm-control-plane-system-capi-kubeadm-control-plane-aggregated-manager-role"
Creating ClusterRole="capi-kubeadm-control-plane-system-capi-kubeadm-control-plane-manager-role"
Creating RoleBinding="capi-kubeadm-control-plane-leader-election-rolebinding" Namespace="capi-kubeadm-control-plane-system"
Creating ClusterRoleBinding="capi-kubeadm-control-plane-system-capi-kubeadm-control-plane-manager-rolebinding"
Creating Service="capi-kubeadm-control-plane-webhook-service" Namespace="capi-kubeadm-control-plane-system"
Creating Deployment="capi-kubeadm-control-plane-controller-manager" Namespace="capi-kubeadm-control-plane-system"
Creating Certificate="capi-kubeadm-control-plane-serving-cert" Namespace="capi-kubeadm-control-plane-system"
Creating Issuer="capi-kubeadm-control-plane-selfsigned-issuer" Namespace="capi-kubeadm-control-plane-system"
Creating MutatingWebhookConfiguration="capi-kubeadm-control-plane-mutating-webhook-configuration"
Creating ValidatingWebhookConfiguration="capi-kubeadm-control-plane-validating-webhook-configuration"
Creating inventory entry Provider="control-plane-kubeadm" Version="v1.3.4" TargetNamespace="capi-kubeadm-control-plane-system"
Installing Provider="infrastructure-aws" Version="v2.0.2" TargetNamespace="capa-system"
Creating objects Provider="infrastructure-aws" Version="v2.0.2" TargetNamespace="capa-system"
Creating Namespace="capa-system"
Creating CustomResourceDefinition="awsclustercontrolleridentities.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsclusterroleidentities.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsclusters.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsclusterstaticidentities.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsclustertemplates.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsfargateprofiles.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsmachinepools.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsmachines.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsmachinetemplates.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsmanagedclusters.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsmanagedcontrolplanes.controlplane.cluster.x-k8s.io"
Creating CustomResourceDefinition="awsmanagedmachinepools.infrastructure.cluster.x-k8s.io"
Creating CustomResourceDefinition="eksconfigs.bootstrap.cluster.x-k8s.io"
Creating CustomResourceDefinition="eksconfigtemplates.bootstrap.cluster.x-k8s.io"
Creating ServiceAccount="capa-controller-manager" Namespace="capa-system"
Creating Role="capa-leader-elect-role" Namespace="capa-system"
Creating ClusterRole="capa-system-capa-manager-role"
Creating RoleBinding="capa-leader-elect-rolebinding" Namespace="capa-system"
Creating ClusterRoleBinding="capa-system-capa-manager-rolebinding"
Creating Secret="capa-manager-bootstrap-credentials" Namespace="capa-system"
Creating Service="capa-webhook-service" Namespace="capa-system"
Creating Deployment="capa-controller-manager" Namespace="capa-system"
Creating Certificate="capa-serving-cert" Namespace="capa-system"
Creating Issuer="capa-selfsigned-issuer" Namespace="capa-system"
Creating MutatingWebhookConfiguration="capa-mutating-webhook-configuration"
Creating ValidatingWebhookConfiguration="capa-validating-webhook-configuration"
Creating inventory entry Provider="infrastructure-aws" Version="v2.0.2" TargetNamespace="capa-system"

Your management cluster has been initialized successfully!

You can now create your first workload cluster by running the following:

  clusterctl generate cluster [name] --kubernetes-version [version] | kubectl apply -f -

No default config file available

0 replies

wilderk · 2023-03-01T23:03:45Z

wilderk
Mar 1, 2023
Author

logs from management cluster init.

clusterctl init --infrastructure aws -v5
No default config file available
Installing the clusterctl inventory CRD
Creating CustomResourceDefinition="providers.clusterctl.cluster.x-k8s.io"
Fetching providers
Fetching File="core-components.yaml" Provider="cluster-api" Type="CoreProvider" Version="v1.3.4"
Fetching File="bootstrap-components.yaml" Provider="kubeadm" Type="BootstrapProvider" Version="v1.3.4"
Fetching File="control-plane-components.yaml" Provider="kubeadm" Type="ControlPlaneProvider" Version="v1.3.4"
Fetching File="infrastructure-components.yaml" Provider="aws" Type="InfrastructureProvider" Version="v2.0.2"
Fetching File="metadata.yaml" Provider="cluster-api" Type="CoreProvider" Version="v1.3.4"
Fetching File="metadata.yaml" Provider="kubeadm" Type="BootstrapProvider" Version="v1.3.4"
Fetching File="metadata.yaml" Provider="kubeadm" Type="ControlPlaneProvider" Version="v1.3.4"
Fetching File="metadata.yaml" Provider="aws" Type="InfrastructureProvider" Version="v2.0.2"
Creating Namespace="cert-manager-test"
Installing cert-manager Version="v1.11.0"
Fetching File="cert-manager.yaml" Provider="cert-manager" Type="" Version="v1.11.0"
Creating Namespace="cert-manager"
Creating CustomResourceDefinition="clusterissuers.cert-manager.io"
Creating CustomResourceDefinition="challenges.acme.cert-manager.io"
Creating CustomResourceDefinition="certificaterequests.cert-manager.io"
Creating CustomResourceDefinition="issuers.cert-manager.io"
Creating CustomResourceDefinition="certificates.cert-manager.io"
Creating CustomResourceDefinition="orders.acme.cert-manager.io"
Creating ConfigMap="cert-manager-webhook" Namespace="cert-manager"
Creating ServiceAccount="cert-manager-cainjector" Namespace="cert-manager"
Creating ServiceAccount="cert-manager" Namespace="cert-manager"
Creating ServiceAccount="cert-manager-webhook" Namespace="cert-manager"
Creating ClusterRole="cert-manager-cainjector"
Creating ClusterRole="cert-manager-controller-issuers"
Creating ClusterRole="cert-manager-controller-clusterissuers"
Creating ClusterRole="cert-manager-controller-certificates"
Creating ClusterRole="cert-manager-controller-orders"
Creating ClusterRole="cert-manager-controller-challenges"
Creating ClusterRole="cert-manager-controller-ingress-shim"
Creating ClusterRole="cert-manager-view"
Creating ClusterRole="cert-manager-edit"
Creating ClusterRole="cert-manager-controller-approve:cert-manager-io"
Creating ClusterRole="cert-manager-controller-certificatesigningrequests"
Creating ClusterRole="cert-manager-webhook:subjectaccessreviews"
Creating ClusterRoleBinding="cert-manager-cainjector"
Creating ClusterRoleBinding="cert-manager-controller-issuers"
Creating ClusterRoleBinding="cert-manager-controller-clusterissuers"
Creating ClusterRoleBinding="cert-manager-controller-certificates"
Creating ClusterRoleBinding="cert-manager-controller-orders"
Creating ClusterRoleBinding="cert-manager-controller-challenges"
Creating ClusterRoleBinding="cert-manager-controller-ingress-shim"
Creating ClusterRoleBinding="cert-manager-controller-approve:cert-manager-io"
Creating ClusterRoleBinding="cert-manager-controller-certificatesigningrequests"
Creating ClusterRoleBinding="cert-manager-webhook:subjectaccessreviews"
Creating Role="cert-manager-cainjector:leaderelection" Namespace="kube-system"
Creating Role="cert-manager:leaderelection" Namespace="kube-system"
Creating Role="cert-manager-webhook:dynamic-serving" Namespace="cert-manager"
Creating RoleBinding="cert-manager-cainjector:leaderelection" Namespace="kube-system"
Creating RoleBinding="cert-manager:leaderelection" Namespace="kube-system"
Creating RoleBinding="cert-manager-webhook:dynamic-serving" Namespace="cert-manager"
Creating Service="cert-manager" Namespace="cert-manager"
Creating Service="cert-manager-webhook" Namespace="cert-manager"
Creating Deployment="cert-manager-cainjector" Namespace="cert-manager"
Creating Deployment="cert-manager" Namespace="cert-manager"
Creating Deployment="cert-manager-webhook" Namespace="cert-manager"
Creating MutatingWebhookConfiguration="cert-manager-webhook"
Creating ValidatingWebhookConfiguration="cert-manager-webhook"
Waiting for cert-manager to be available...
Updating Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Error: timed out waiting for the condition

4 replies

chrischdi Mar 2, 2023
Collaborator

Could you try to do the following and post the result?

cat << EOF | kubectl apply --dry-run=server -f -
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: test-selfsigned
  namespace: default
spec:
  selfSigned: {}
EOF

If this does not show an error, then it seems like the error only exists during the installation. If so:

Could you do a kubectl get pods -n cert-manager during the reoccuring Creating Issuer="test-selfsigned" Namespace="cert-manager-test"? Are they all running?

wilderk Mar 2, 2023
Author

There is an error.

cat << EOF | kubectl apply --dry-run=server -f -
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: test-selfsigned
  namespace: default
spec:
  selfSigned: {}
EOF
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": context deadline exceeded

chrischdi Mar 3, 2023
Collaborator

So this very much looks like that there's something wrong at your cluster.

The kube-apiserver tries to reach out to the cert-manager webhook (via ValidatingWebhookConfiguration) and is not able to do so, so its somewhat unrelated to Cluster API, but the requirement that networking is fine in the management cluster is not valid.

If I have to make guesses: there is either something wrong in CNI (you use flannel according above output?), or kube-proxy, or infrastructure security group stuff, or network policies are blocking the requests, ... .

wilderk Mar 3, 2023
Author

I also suspect a problem with the workload cluster that the aws provider created. Flannel was the only CNI that I could get installed. Tried several others (Calico, Cilium, WeaveNet) but they all had problems installing. I suspect the root cause is the same thing that is causing cert-manager Issuer creation to fail.
What is troubling is that I did not make any changes to the cluster manifests created by clusterctl generate command. The aws provider should have created a cluster that is functional. I was using the latest aws provider v2.0.2 to init the bootstrap cluster.

fabriziopandini · 2023-03-06T10:43:11Z

fabriziopandini
Mar 6, 2023
Maintainer

please use issues for the request of support, there is a also kind/support label for identifying them.
Let's keep discussion for open topics only

1 reply

wilderk Mar 11, 2023
Author

Sorry. I didn't know the process. Will do better in the future.

wilderk · 2023-03-11T23:49:21Z

wilderk
Mar 11, 2023
Author

The cause of the problem is the node security group. I opened an issue in cluster-api-provider-aws
kubernetes-sigs/cluster-api-provider-aws#4136

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pivot control from bootstrap cluster to the management cluster #8214

{{title}}

Replies: 4 comments 5 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

pivot control from bootstrap cluster to the management cluster #8214

wilderk Mar 1, 2023

Replies: 4 comments · 5 replies

wilderk Mar 1, 2023 Author

wilderk Mar 1, 2023 Author

chrischdi Mar 2, 2023 Collaborator

wilderk Mar 2, 2023 Author

chrischdi Mar 3, 2023 Collaborator

wilderk Mar 3, 2023 Author

fabriziopandini Mar 6, 2023 Maintainer

wilderk Mar 11, 2023 Author

wilderk Mar 11, 2023 Author

wilderk
Mar 1, 2023

Replies: 4 comments 5 replies

wilderk
Mar 1, 2023
Author

wilderk
Mar 1, 2023
Author

chrischdi Mar 2, 2023
Collaborator

wilderk Mar 2, 2023
Author

chrischdi Mar 3, 2023
Collaborator

wilderk Mar 3, 2023
Author

fabriziopandini
Mar 6, 2023
Maintainer

wilderk Mar 11, 2023
Author

wilderk
Mar 11, 2023
Author