-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Revamp "Watching Resources" documentation for accuracy and clarity
- Fully revamped the "Watching Resources" section for improved clarity and accuracy. - Ensured terminology aligns with Kubernetes and controller-runtime standards. - Provided detailed examples for watching owned resources, non-owned resources, and applying predicates to refine watches.
- Loading branch information
1 parent
333170b
commit df188dd
Showing
12 changed files
with
500 additions
and
573 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,16 +1,115 @@ | ||
# Watching Resources | ||
|
||
Inside a `Reconcile()` control loop, you are looking to do a collection of operations until it has the desired state on the cluster. | ||
Therefore, it can be necessary to know when a resource that you care about is changed. | ||
In the case that there is an action (create, update, edit, delete, etc.) on a watched resource, `Reconcile()` should be called for the resources watching it. | ||
When extending the Kubernetes API, we aim to ensure that our solutions behave consistently with Kubernetes itself. | ||
For example, consider a `Deployment` resource, which is managed by a controller. This controller is responsible | ||
for responding to changes in the cluster—such as when a `Deployment` is created, updated, or deleted—by triggering | ||
reconciliation to ensure the resource’s state matches the desired state. | ||
|
||
[Controller Runtime libraries](https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/builder) provide many ways for resources to be managed and watched. | ||
This ranges from the easy and obvious use cases, such as watching the resources which were created and managed by the controller, to more unique and advanced use cases. | ||
Similarly, when developing our controllers, we want to watch for relevant changes in resources that are crucial | ||
to our solution. These changes—whether creations, updates, or deletions—should trigger the reconciliation | ||
loop to take appropriate actions and maintain consistency across the cluster. | ||
|
||
See each subsection for explanations and examples of the different ways in which your controller can _Watch_ the resources it cares about. | ||
The [controller-runtime][controller-runtime] library provides several ways to watch and manage resources. | ||
|
||
- [Watching Operator Managed Resources](watching-resources/operator-managed.md) - | ||
These resources are created and managed by the same operator as the resource watching them. | ||
This section covers both if they are managed by the same controller or separate controllers. | ||
- [Watching Externally Managed Resources](watching-resources/externally-managed.md) - | ||
These resources could be manually created, or managed by other operators/controllers or the Kubernetes control plane. | ||
## Primary Resources | ||
|
||
The **Primary Resource** is the resource that your controller is responsible | ||
for managing. For example, if you create a custom resource definition (CRD) for `MyApp`, | ||
the corresponding controller is responsible for managing instances of `MyApp`. | ||
|
||
In this case, `MyApp` is the **Primary Resource** for that controller, and your controller’s | ||
reconciliation loop focuses on ensuring the desired state of these primary resources is maintained. | ||
|
||
When you create a new API using Kubebuilder, the following default code is scaffolded, | ||
ensuring that the controller watches all relevant events—such as creations, updates, and | ||
deletions—for (`For()`) the new API. | ||
|
||
This setup guarantees that the reconciliation loop is triggered whenever an instance | ||
of the API is created, updated, or deleted: | ||
|
||
```go | ||
// Watches the primary resource (e.g., MyApp) for create, update, delete events | ||
if err := ctrl.NewControllerManagedBy(mgr). | ||
For(&<YourAPISpec>{}). <-- See there that the Controller is For this API | ||
Complete(r); err != nil { | ||
return err | ||
} | ||
``` | ||
|
||
## Secondary Resources | ||
|
||
Your controller will likely also need to manage **Secondary Resources**, | ||
which are the resources required on the cluster to support the **Primary Resource**. | ||
|
||
Changes to these **Secondary Resources** can directly impact the **Primary Resource**, | ||
so the controller must watch and reconcile these resources accordingly. | ||
|
||
### Which are Owned by the Controller | ||
|
||
These **Secondary Resources**, such as `Services`, `ConfigMaps`, or `Deployments`, | ||
when `Owned` by the controllers, are created and managed by the specific controller | ||
and are tied to the **Primary Resource** via [OwnerReferences][owner-ref-k8s-docs]. | ||
|
||
For example, if we have a controller to manage our CR(s) of the Kind `MyApp` | ||
on the cluster, which represents our application solution, all resources required | ||
to ensure that `MyApp` is up and running with the desired number of instances | ||
will be **Secondary Resources**. The code responsible for creating, deleting, | ||
and updating these resources will be part of the `MyApp` Controller. | ||
We would add the appropriate [OwnerReferences][owner-ref-k8s-docs] | ||
using the [controllerutil.SetControllerReference][cr-owner-ref-doc] | ||
function to indicate that these resources are owned by the same controller | ||
responsible for managing `MyApp` instances, which will be reconciled by the `MyAppReconciler`. | ||
|
||
Additionally, if the **Primary Resource** is deleted, Kubernetes' garbage collection mechanism | ||
ensures that all associated **Secondary Resources** are automatically deleted in a | ||
cascading manner. | ||
|
||
### Which are NOT `Owned` by the Controller | ||
|
||
Note that **Secondary Resources** can either be APIs/CRDs defined in your project or in other projects that are | ||
relevant to the **Primary Resources**, but which the specific controller is not responsible for creating or managing. | ||
|
||
For example, if we have a CRD that represents a backup solution (i.e. `MyBackup`) for our `MyApp`, | ||
it might need to watch changes in the `MyApp` resource to trigger reconciliation in `MyBackup` | ||
to ensure the desired state. Similarly, `MyApp`'s behavior might also be impacted by | ||
CRDs/APIs defined in other projects. | ||
|
||
In both scenarios, these resources are treated as **Secondary Resources**, even if they are not `Owned` | ||
(i.e., not created or managed) by the `MyAppController`. | ||
|
||
In Kubebuilder, resources that are not defined in the project itself and are not | ||
a **Core Type** (those not defined in the Kubernetes API) are called **External Types**. | ||
|
||
An **External Type** refers to a resource that is not defined in your | ||
project but one that you need to watch and respond to. | ||
For example, if **Operator A** manages a `MyApp` CRD for application deployment, | ||
and **Operator B** handles backups, **Operator B** can watch the `MyApp` CRD as an external type | ||
to trigger backup operations based on changes in `MyApp`. | ||
|
||
In this scenario, **Operator B** could define a `BackupConfig` CRD that relies on the state of `MyApp`. | ||
By treating `MyApp` as a **Secondary Resource**, **Operator B** can watch and reconcile changes in **Operator A**'s `MyApp`, | ||
ensuring that backup processes are initiated whenever `MyApp` is updated or scaled. | ||
|
||
## General Concept of Watching Resources | ||
|
||
Whether a resource is defined within your project or comes from an external project, the concept of **Primary** | ||
and **Secondary Resources** remains the same: | ||
- The **Primary Resource** is the resource the controller is primarily responsible for managing. | ||
- **Secondary Resources** are those that are required to ensure the primary resource works as desired. | ||
|
||
Therefore, regardless of whether the resource was defined by your project or by another project, | ||
your controller can watch, reconcile, and manage changes to these resources as needed. | ||
|
||
## Usage of Predicates | ||
|
||
For more complex use cases, [Predicates][cr-predicates] can be used to fine-tune | ||
when your controller should trigger reconciliation. Predicates allow you to filter | ||
events based on specific conditions, such as changes to particular fields, labels, or annotations, | ||
ensuring that your controller only responds to relevant events and operates efficiently. | ||
|
||
[controller-runtime]: https://github.com/kubernetes-sigs/controller-runtime | ||
[owner-ref-k8s-docs]: https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/ | ||
[cr-predicates]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/predicate | ||
[secondary-resources-doc]: watching-resources/secondary-owned-resources | ||
[predicates-with-external-type-doc]: watching-resources/predicates-with-watch | ||
[cr-owner-ref-doc]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller/controllerutil#SetOwnerReference |
31 changes: 0 additions & 31 deletions
31
docs/book/src/reference/watching-resources/externally-managed.md
This file was deleted.
Oops, something went wrong.
25 changes: 0 additions & 25 deletions
25
docs/book/src/reference/watching-resources/operator-managed.md
This file was deleted.
Oops, something went wrong.
113 changes: 113 additions & 0 deletions
113
docs/book/src/reference/watching-resources/predicates-with-watch.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
# Using Predicates to Refine Watches | ||
|
||
When working with controllers, it's often beneficial to use **Predicates** to | ||
filter events and control when the reconciliation loop should be triggered. | ||
|
||
[Predicates][predicates-doc] allow you to define conditions based on events (such as create, update, or delete) | ||
and resource fields (such as labels, annotations, or status fields). By using **[Predicates][predicates-doc]**, | ||
you can refine your controller’s behavior to respond only to specific changes in the resources | ||
it watches. | ||
|
||
This can be especially useful when you want to refine which | ||
changes in resources should trigger a reconciliation. By using predicates, | ||
you avoid unnecessary reconciliations and can ensure that the | ||
controller only reacts to relevant changes. | ||
|
||
## When to Use Predicates | ||
|
||
**Predicates are useful when:** | ||
|
||
- You want to ignore certain changes, such as updates that don't impact the fields your controller is concerned with. | ||
- You want to trigger reconciliation only for resources with specific labels or annotations. | ||
- You want to watch external resources and react only to specific changes. | ||
|
||
## Example: Using Predicates to Filter Update Events | ||
|
||
Let’s say that we only want our **`BackupBusybox`** controller to reconcile | ||
when certain fields of the **`Busybox`** resource change, for example, when | ||
the `spec.size` field changes, but we want to ignore all other changes (such as status updates). | ||
|
||
### Defining a Predicate | ||
|
||
In the following example, we define a predicate that only | ||
allows reconciliation when there’s a meaningful update | ||
to the **`Busybox`** resource: | ||
|
||
```go | ||
import ( | ||
"sigs.k8s.io/controller-runtime/pkg/predicate" | ||
"sigs.k8s.io/controller-runtime/pkg/event" | ||
) | ||
|
||
// Predicate to trigger reconciliation only on size changes in the Busybox spec | ||
updatePred := predicate.Funcs{ | ||
// Only allow updates when the spec.size of the Busybox resource changes | ||
UpdateFunc: func(e event.UpdateEvent) bool { | ||
oldObj := e.ObjectOld.(*examplecomv1alpha1.Busybox) | ||
newObj := e.ObjectNew.(*examplecomv1alpha1.Busybox) | ||
|
||
// Trigger reconciliation only if the spec.size field has changed | ||
return oldObj.Spec.Size != newObj.Spec.Size | ||
}, | ||
|
||
// Allow create events | ||
CreateFunc: func(e event.CreateEvent) bool { | ||
return true | ||
}, | ||
|
||
// Allow delete events | ||
DeleteFunc: func(e event.DeleteEvent) bool { | ||
return true | ||
}, | ||
|
||
// Allow generic events (e.g., external triggers) | ||
GenericFunc: func(e event.GenericEvent) bool { | ||
return true | ||
}, | ||
} | ||
``` | ||
|
||
### Explanation | ||
|
||
In this example: | ||
- The **`UpdateFunc`** returns `true` only if the **`spec.size`** field has changed between the old and new objects, meaning that all other changes in the `spec`, like annotations or other fields, will be ignored. | ||
- **`CreateFunc`**, **`DeleteFunc`**, and **`GenericFunc`** return `true`, meaning that create, delete, and generic events are still processed, allowing reconciliation to happen for these event types. | ||
|
||
This ensures that the controller reconciles only when the specific field **`spec.size`** is modified, while ignoring any other modifications in the `spec` that are irrelevant to your logic. | ||
|
||
### Example: Using Predicates in `Watches` | ||
|
||
Now, we apply this predicate in the **`Watches()`** method of | ||
the **`BackupBusyboxReconciler`** to trigger reconciliation only for relevant events: | ||
|
||
```go | ||
// SetupWithManager sets up the controller with the Manager. | ||
// The controller will watch both the BackupBusybox primary resource and the Busybox resource, using predicates. | ||
func (r *BackupBusyboxReconciler) SetupWithManager(mgr ctrl.Manager) error { | ||
return ctrl.NewControllerManagedBy(mgr). | ||
For(&examplecomv1alpha1.BackupBusybox{}). // Watch the primary resource (BackupBusybox) | ||
Watches( | ||
&source.Kind{Type: &examplecomv1alpha1.Busybox{}}, // Watch the Busybox CR | ||
handler.EnqueueRequestsFromMapFunc(func(obj client.Object) []reconcile.Request { | ||
return []reconcile.Request{ | ||
{ | ||
NamespacedName: types.NamespacedName{ | ||
Name: "backupbusybox", // Reconcile the associated BackupBusybox resource | ||
Namespace: obj.GetNamespace(), // Use the namespace of the changed Busybox | ||
}, | ||
}, | ||
} | ||
}), | ||
builder.WithPredicates(updatePred), // Apply the predicate | ||
). // Trigger reconciliation when the Busybox resource changes (if it meets predicate conditions) | ||
Complete(r) | ||
} | ||
``` | ||
|
||
### Explanation | ||
|
||
- **[`builder.WithPredicates(updatePred)`][predicates-doc]**: This method applies the predicate, ensuring that reconciliation only occurs | ||
when the **`spec.size`** field in **`Busybox`** changes. | ||
- **Other Events**: The controller will still trigger reconciliation on `Create`, `Delete`, and `Generic` events. | ||
|
||
[predicates-doc]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/source#WithPredicates |
Oops, something went wrong.