Skip to content

Commit

Permalink
initial draft of modified proposal including new CRD
Browse files Browse the repository at this point in the history
Signed-off-by: Jack Francis <jackfrancis@gmail.com>
  • Loading branch information
jackfrancis committed Oct 27, 2023
1 parent 6bb820a commit 1807fe7
Show file tree
Hide file tree
Showing 3 changed files with 196 additions and 79 deletions.
22 changes: 16 additions & 6 deletions docs/book/src/reference/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ A temporary cluster that is used to provision a Target Management cluster.
### Bootstrap provider

Refers to a [provider](#provider) that implements a solution for the [bootstrap](#bootstrap) process.
Bootstrap provider's interaction with Cluster API is based on what is defined in the [Cluster API contract](#contract).
Bootstrap provider's interaction with Cluster API is based on what is defined in the [Cluster API contract](#contract).

See [CABPK](#cabpk).
See [CABPK](#cabpk).

# C
---
Expand Down Expand Up @@ -132,6 +132,12 @@ See [core provider](#core-provider)

The Cluster API execution model, a set of controllers cooperating in managing the Kubernetes cluster lifecycle.

### Cluster Infrastructure

or __Kubernetes Cluster Infrastructure__

Defines the **infrastructure that supports a Kubernetes cluster**, like e.g. VPC, security groups, load balancers, etc. Please note that in the context of managed Kubernetes some of those components are going to be provided by the corresponding abstraction for a specific Cloud provider (EKS, OKE, AKS etc), and thus Cluster API should not take care of managing a subset or all those components.

### Contract

Or __Cluster API contract__
Expand All @@ -155,7 +161,7 @@ See [KCP](#kcp).

### Core provider

Refers to a [provider](#provider) that implements Cluster API core controllers; if you
Refers to a [provider](#provider) that implements Cluster API core controllers; if you
consider that the first project that must be deployed in a management Cluster is Cluster API itself, it should be clear why
the Cluster API project is also referred to as the core provider.

Expand Down Expand Up @@ -196,7 +202,7 @@ see [Server](#server)

### Infrastructure provider

Refers to a [provider](#provider) that implements provisioning of infrastructure/computational resources required by
Refers to a [provider](#provider) that implements provisioning of infrastructure/computational resources required by
the Cluster or by Machines (e.g. VMs, networking, etc.).
Infrastructure provider's interaction with Cluster API is based on what is defined in the [Cluster API contract](#contract).

Expand All @@ -205,7 +211,7 @@ When there is more than one way to obtain resources from the same infrastructure

For a complete list of providers see [Provider Implementations](providers.md).

### Inline patch
### Inline patch

A [patch](#patch) defined inline in a [ClusterClass](#clusterclass). An alternative to an [external patch](#external-patch).

Expand Down Expand Up @@ -269,6 +275,10 @@ See also: [Server](#server)

Perform create, scale, upgrade, or destroy operations on the cluster.

### Managed Kubernetes

Managed Kubernetes refers to any Kubernetes cluster provisioning and maintenance abstraction, usually exposed as an API, that is natively available in a Cloud provider. For example: [EKS](https://aws.amazon.com/eks/), [OKE](https://www.oracle.com/cloud/cloud-native/container-engine-kubernetes/), [AKS](https://azure.microsoft.com/en-us/products/kubernetes-service), [GKE](https://cloud.google.com/kubernetes-engine), [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service), [DOKS](https://www.digitalocean.com/products/kubernetes), and many more throughout the Kubernetes Cloud Native ecosystem.

### Managed Topology

See [Topology](#topology)
Expand Down Expand Up @@ -306,7 +316,7 @@ A generically understood combination of a kernel and system-level userspace inte
# P
---

### Patch
### Patch

A set of instructions describing modifications to a Kubernetes object. Examples include JSON Patch and JSON Merge Patch.

Expand Down
18 changes: 9 additions & 9 deletions docs/proposals/20220725-managed-kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ reviewers:
creation-date: 2022-07-25
last-updated: 2023-06-15
status: implementable
see-also: ./20230407-managed-k8s-capi-contract-changes.md
see-also: ./20230407-flexible-managed-k8s-endpoints.md
replaces:
superseded-by:
---
Expand Down Expand Up @@ -97,7 +97,7 @@ Some Cluster API Providers (i.e. Azure with AKS first and then AWS with EKS) hav

While working on supporting ClusterClass for EKS in Cluster API Provider AWS (CAPA), it was discovered that the current implementation of EKS within CAPA, where a single resource kind (AWSManagedControlPlane) is used for both ControlPlane and Infrastructure, is incompatible with other parts of CAPI assuming the two objects are different (Reference [issue here](https://github.com/kubernetes-sigs/cluster-api/issues/6126)).

Separation of ControlPlane and Infrastructure is expected for the ClusterClass implementation to work correctly. However, after the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-managed-k8s-capi-contract-changes) have been implemented there is the option to supply only the control plane, but you still cannot supply the same resource for both.
Separation of ControlPlane and Infrastructure is expected for the ClusterClass implementation to work correctly. However, after the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-flexible-managed-k8s-endpoints.md) have been implemented there is the option to supply only the control plane, but you still cannot supply the same resource for both.

The responsibilities between the CAPI control plane and infrastructure are blurred with a managed Kubernetes service like AKS or EKS. For example, when you create a EKS control plane in AWS it also creates infrastructure that CAPI would traditionally view as the responsibility of the cluster “infrastructure provider”.

Expand All @@ -118,7 +118,7 @@ A good example here is the API server load balancer:
- Enforce the Managed Kubernetes recommendations as a requirement for Cluster API providers when they implement Managed Kubernetes.
- If providers that have already implemented Managed Kubernetes and would like guidance on if/how they could move to be aligned with the recommendations of this proposal then discussions should be facilitated.
- Provide advice in this proposal on how to refactor the existing implementations of managed Kubernetes in CAPA & CAPZ.
- Propose a new architecture or API changes to CAPI for managed Kubernetes. This has been covered by the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-managed-k8s-capi-contract-changes).
- Propose a new architecture or API changes to CAPI for managed Kubernetes. This has been covered by the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-flexible-managed-k8s-endpoints.md).
- Be a concrete design for the GKE implementation in Cluster API Provider GCP (CAPG).
- Recommend how Managed Kubernetes services would leverage CAPI internally to run their offer.

Expand Down Expand Up @@ -247,7 +247,7 @@ The following section discusses different API implementation options along with

#### Option 1: Two kinds with a ControlPlane and a pass-through InfraCluster

**This option will be no longer needed when the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-managed-k8s-capi-contract-changes) have been implemented as option 2 can be used for a simpler solution**
**This option will be no longer needed when the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-flexible-managed-k8s-endpoints.md) have been implemented as option 2 can be used for a simpler solution**

This option introduces 2 new resource kinds:

Expand Down Expand Up @@ -304,7 +304,7 @@ type GCPManagedClusterSpec struct {

#### Option 2: Just a ControlPlane kind and no InfraCluster

**This option is enabled when the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-managed-k8s-capi-contract-changes) have been implemented.**
**This option is enabled when the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-flexible-managed-k8s-endpoints.md) have been implemented.**

This option introduces 1 new resource kind:

Expand Down Expand Up @@ -400,7 +400,7 @@ type GCPManagedClusterSpec struct {
}
```

When the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-managed-k8s-capi-contract-changes) have been implemented there is the option to return the control plane endpoint directly from the ControlPlane instead of passing it via the Infracluster.
When the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-flexible-managed-k8s-endpoints.md) have been implemented there is the option to return the control plane endpoint directly from the ControlPlane instead of passing it via the Infracluster.

**Pros**

Expand Down Expand Up @@ -429,7 +429,7 @@ The reasons for this recommendation are as follows:

If the managed Kubernetes services does not require any base infrastructure to be setup before creating the instance of the service then option 2 (Just a ControlPlane kind (and no InfraCluster) is the recommendation.

This recommendation assumes that the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-managed-k8s-capi-contract-changes) have been implemented. Until that point option 1 (Two kinds with a ControlPlane and a pass-through InfraCluster) will have to be used.
This recommendation assumes that the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-flexible-managed-k8s-endpoints.md) have been implemented. Until that point option 1 (Two kinds with a ControlPlane and a pass-through InfraCluster) will have to be used.

### Existing Managed Kubernetes Implementations

Expand Down Expand Up @@ -484,7 +484,7 @@ Some of the areas of change (this is not an exhaustive list):
- Update the [Provider contracts documentation](../book/src/developer/providers/contracts.md) to state that the same kind should not be used to satisfy 2 different provider contracts.
- Update the [Cluster Infrastructure documentation](../book/src/developer/providers/cluster-infrastructure.md) to provide guidance on how to populate the `controlPlaneEndpoint` in the scenario where the control plane creates the api server load balancer. We should include sample code.
- Update the [Control Plane Controller](../book/src/developer/architecture/controllers/control-plane.md) diagram for managed k8s services case. The Control Plane reconcile needs to start when `InfrastructureReady` is true.
- Updates based on the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-managed-k8s-capi-contract-changes).
- Updates based on the changes documented in the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-flexible-managed-k8s-endpoints.md).

## Other Considerations for CAPI

Expand Down Expand Up @@ -606,4 +606,4 @@ As mentioned in the goals section, it is up to providers with existing implement
- [x] 03/17/2022: Compile a Google Doc following the CAEP template ([link](https://docs.google.com/document/d/1dMN4-KppBkA51sxXPSQhYpqETp2AG_kHzByXTmznxFA/edit?usp=sharing))
- [x] 04/20/2022: Present proposal at a community meeting
- [x] 07/27/2022: Move the proposal to a PR in CAPI repo
- [x] 06/15/2023: Updates as a result of the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-managed-k8s-capi-contract-changes.md) and also updates as a result of the current state of managed k8s in CAPI.
- [x] 06/15/2023: Updates as a result of the [Contract Changes to Support Managed Kubernetes CAEP](./20230407-flexible-managed-k8s-endpoints.md) and also updates as a result of the current state of managed k8s in CAPI.
Loading

0 comments on commit 1807fe7

Please sign in to comment.