Skip to content

Commit

Permalink
Add a note about WMCO kubelet dependency
Browse files Browse the repository at this point in the history
  • Loading branch information
Danil-Grigorev committed Jul 12, 2021
1 parent edd7c29 commit 16d8d4d
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions enhancements/machine-api/out-of-tree-provider-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,8 @@ Without this information, the `scheduler` cannot schedule `Pods` that have any s
To ensure that cluster disaster recovery procedures can still operate smoothly, we will ensure that core control plane components and their operators tolerate the uninitialized taint, to prevent `CCM` blocking new control plane hosts being added if `CCM` is non-functional.
This will include, but is not limited to: Kube Controller Manager, Etcd, Kube API Server, Networking, Cluster Machine Approver.

*Note: Windows Machine Config Operator (WMCO) manages `kubelet` configuration for Windows nodes, for which described changes in this chapter would still apply.*

##### Example flag changes for kubelet

Current flag configuration for kubelet in AWS provider:
Expand Down Expand Up @@ -337,7 +339,8 @@ One of the responsibilities of the initialisation process for Kubelet is to set
A second part of the bootstrap process for a new `Node`, is to initialise the `CNI` (networking). Typically in an OpenShift cluster, this is handled once the Networking Operator starts.
The Networking operator will create the `CNI` pods (typically OpenShift SDN), which schedule on the `Node`, use the `Node` IP addresses to create a `HostSubnet` resource within Kubernetes and then mark then complete the initialisation process for the `CNI`, in doing so, marking the `Node` as ready and allowing the remaining workloads to start.

Before the `CNI` is initialized on a `Node`, in-cluster networking such as Service IPs, in particular the API server Service, will not work for any `Pod` on the `Node`. Additionally, any `Pod` that requires the Pod Networking implemented by `CNI`, cannot start. For this reason, `Pods` such as the Networking Operator must use host networking and the “API Int” load balancer to contact the Kube API Server.
Before the `CNI` is initialized on a `Node`, in-cluster networking such as Service IPs, in particular the API server Service, will not work for any `Pod` on the `Node`. Additionally, any `Pod` that requires the Pod Networking implemented by `CNI`, cannot start.
For this reason, `Pods` such as the Networking Operator must use host networking and the “API Int” load balancer to contact the Kube API Server.

Because the `CCM` is taking over the responsibility of setting the `Node` IP addresses, `CCM` will become a prerequisite for networking to become functional within any Cluster. Because the `CNI` is not initialised, we must ensure that the `CCCMO` and `CCM` Pods tolerate the scenario where `CNI` is non-functional.

Expand Down Expand Up @@ -553,7 +556,8 @@ This functionality should not be required as OpenShift handles certificate appro

Q: Does every node need a CCM?

- A: No. The cluster only needs one active `CCM` at any time. A `Deployment` will manage the `CCM` pod and will have 2 replicas which will use leader election to nominate an active leader.
- A: No. The cluster only needs one active `CCM` at any time. A `Deployment` will manage the `CCM` pod and will have 2 replicas which will use leader election to nominate an active leader and maintain HA by scheduling on control-plane nodes located in the different regions.
Only in some scenarios (depending on the cloud provider implementation) like Azure `cloud-node-manager` has to run on all the `Nodes` due to 1:1 relation betwen `Node` and a `CNM` replica in their case.
[Source](https://kubernetes.io/docs/concepts/overview/components/#cloud-controller-manager) This assumption may change in the future, as `CCM` may run in worker nodes to determine the state of the instance.

Q: How metrics are affected by the CCM migration?
Expand Down Expand Up @@ -708,4 +712,4 @@ Mandatory operator repository:
- [The Kubernetes Cloud Controller Manager](https://medium.com/@m.json/the-kubernetes-cloud-controller-manager-d440af0d2be5) article
https://hackmd.io/00IoVWBiSVm8mMByxerTPA#
- [CSI support](https://github.com/openshift/enhancements/blob/master/enhancements/storage/csi-driver-install.md#ocp-45-kubernetes-118)
- [CNI ]
- [CCM role in bootstrap process](https://docs.google.com/document/d/1yAczhHNJ4rDqVFFvyi7AZ27DEQdvx8DmLNbavIjrjn0)

0 comments on commit 16d8d4d

Please sign in to comment.