Add a guide describing ownerReference usage in CAPI

Signed-off-by: killianmuldoon <kmuldoon@vmware.com>
kubernetes-sigs · Aug 9, 2023 · 6bd6c6b · 6bd6c6b
1 parent 99009fe
commit 6bd6c6b
Show file tree

Hide file tree

Showing 3 changed files with 93 additions and 0 deletions.
diff --git a/docs/book/src/SUMMARY.md b/docs/book/src/SUMMARY.md
@@ -117,3 +117,4 @@
     - [Code Review in Cluster API](./REVIEWING.md)
     - [Version Support](./reference/versions.md)
     - [Supported Labels and Annotations](./reference/labels_and_annotations.md)
+    - [Owner References](./reference/owner_references.md)
diff --git a/docs/book/src/reference/owner_references.md b/docs/book/src/reference/owner_references.md
@@ -0,0 +1,80 @@
+# Owner References
+
+
+Cluster API uses [Kubernetes owner references](https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/) to track relationships between objects. These references are used for Kubernetes garbage collection, which is the basis of Cluster deletion in CAPI. They are also used places where the ownership hierarchy is important, for example when using `clusterctl move`.
+
+CAPI uses owner references in an opinionated way. The following guidelines should be considered:
+1. Objects should always be created with an owner reference to prevent leaking objects. Initial ownerReferences can be replaced later where another object is a more appropriate owner.
+2. Owner references should be re-reconciled if they are lost for an object. This is required as some tools - e.g. velero - may delete owner references on objects.
+3. Owner references should be kept to the most recent apiVersion.
+   - This ensures garbage collection still works after an old apiVersion is no longer served.
+4. Owner references should not be added unless required.
+   - Multiple owner references on a single object should be exceptional.
+
+
+
+
+## Owner reference relationships in Cluster API
+
+The below tables map out the a reference for ownership relationships for the objects in a Cluster API cluster.
+Providers may implement their own ownership relationships which may or may not map directly to the below tables. 
+These owner references are almost all tested in an [end-to-end test](https://github.com/kubernetes-sigs/cluster-api/blob/caaa74482b51fae777334cd7a29595da1c06481e/test/e2e/quick_start_test.go#L31). Lack of testing is noted where this is not the case. CAPI Providers can take advantage of the e2e test framework to ensure their owner references are predictable, documented and stable.
+
+ Kubernetes core types
+
+| type      | Owner               | Note                                     |
+|-----------|---------------------|------------------------------------------|
+| Secret    | KubeadmControlPlane | For cluster certificates                 |
+| Secret    | KubeadmConfig       | For bootstrap secrets                    |
+| Secret    | ClusterResourceSet  | When created by CRS. Not covered in e2e. |
+| ConfigMap | ClusterResourceSet  | When created by CRS                      |
+
+## Core types
+
+| type                | Owner               | Note                       |
+|---------------------|---------------------|----------------------------|
+| ExtensionConfig     | None                |                            |
+| ClusterClass        | None                |                            |
+| Cluster             | None                |                            |
+| MachineDeployments  | Cluster             |                            |
+| MachineSet          | MachineDeployment   |                            |
+| Machine             | MachineSet          | When created by MachineSet |
+| Machine             | KubeadmControlPlane | When created by KCP        |
+| MachineHealthChecks | Cluster             |                            |
+
+
+
+## Experimental types
+| type                       | Owner              | Note |
+|----------------------------|--------------------|------|
+| ClusterResourcesSet        | None               |      |
+| ClusterResourcesSetBinding | ClusterResourceSet |      |
+| MachinePool                | Cluster            |      |
+
+
+## KubeadmControlPlane types
+| type                        | Owner        | Note |
+|-----------------------------|--------------|------|
+| KubeadmControlPlane         | Cluster      |      |
+| KubeadmControlPlaneTemplate | ClusterClass |      |
+
+
+## Kubeadm bootstrap types
+| type                  | Owner        | Note                                      |
+|-----------------------|--------------|-------------------------------------------|
+| KubeadmConfig         | Machine      | When created for Machine                  |
+| KubeadmConfig         | MachinePool  | When created for MachinePool              |
+| KubeadmConfigTemplate | Cluster      | When referenced in MachineDeployment spec |
+| KubeadmConfigTemplate | ClusterClass | When referenced in ClusterClass           |
+
+## Infrastructure provider types
+| type                          | Owner        | Note                                        |
+|-------------------------------|--------------|---------------------------------------------|
+| InfrastructureMachine         | Machine      |                                             |
+| InfrastructureMachineTemplate | Cluster      | When created by cluster topology controller |
+| InfrastructureMachineTemplate | ClusterClass | When referenced in a ClusterClass           |
+| InfrastructureCluster         | Cluster      |                                             |
+| InfrastructureClusterTemplate | ClusterClass |                                             |
+| InfrastructureMachinePool     | MachinePool  |                                             |
+
+
diff --git a/test/framework/ownerreference_helpers.go b/test/framework/ownerreference_helpers.go
@@ -118,6 +118,8 @@ var (
 
 // CoreOwnerReferenceAssertion maps Cluster API core types to functions which return an error if the passed
 // OwnerReferences aren't as expected.
+// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
+// That document should be updated if these references change.
 var CoreOwnerReferenceAssertion = map[string]func([]metav1.OwnerReference) error{
 	extensionConfigKind: func(owners []metav1.OwnerReference) error {
 		// ExtensionConfig should have no owners.
@@ -161,6 +163,8 @@ var (
 
 // ExpOwnerReferenceAssertions maps experimental types to functions which return an error if the passed OwnerReferences
 // aren't as expected.
+// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
+// That document should be updated if these references change.
 var ExpOwnerReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
 	clusterResourceSetKind: func(owners []metav1.OwnerReference) error {
 		// ClusterResourcesSet doesn't have ownerReferences (it is a clusterctl move-hierarchy root).
@@ -184,6 +188,8 @@ var (
 
 // KubernetesReferenceAssertions maps Kubernetes types to functions which return an error if the passed OwnerReferences
 // aren't as expected.
+// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
+// That document should be updated if these references change.
 var KubernetesReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
 	secretKind: func(owners []metav1.OwnerReference) error {
 		// Secrets for cluster certificates must be owned by the KubeadmControlPlane. The bootstrap secret should be owned by a KubeadmControlPlane.
@@ -205,6 +211,8 @@ var (
 
 // KubeadmControlPlaneOwnerReferenceAssertions maps Kubeadm control plane types to functions which return an error if the passed
 // OwnerReferences aren't as expected.
+// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
+// That document should be updated if these references change.
 var KubeadmControlPlaneOwnerReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
 	kubeadmControlPlaneKind: func(owners []metav1.OwnerReference) error {
 		// The KubeadmControlPlane must be owned by a Cluster.
@@ -226,6 +234,8 @@ var (
 
 // KubeadmBootstrapOwnerReferenceAssertions maps KubeadmBootstrap types to functions which return an error if the passed OwnerReferences
 // aren't as expected.
+// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
+// That document should be updated if these references change.
 var KubeadmBootstrapOwnerReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
 	kubeadmConfigKind: func(owners []metav1.OwnerReference) error {
 		// The KubeadmConfig must be owned by a Cluster or by a MachinePool.
@@ -248,6 +258,8 @@ var (
 
 // DockerInfraOwnerReferenceAssertions maps Docker Infrastructure types to functions which return an error if the passed
 // OwnerReferences aren't as expected.
+// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
+// That document should be updated if these references change.
 var DockerInfraOwnerReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
 	dockerMachineKind: func(owners []metav1.OwnerReference) error {
 		// The DockerMachine must be owned by a Machine.