-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Chris Hein <me@chrishein.com>
- Loading branch information
1 parent
8507812
commit 041b120
Showing
2 changed files
with
244 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,244 @@ | ||
# ComponentConfig Controller Runtime Support | ||
|
||
Author: @christopherhein | ||
Last Updated on: 02/24/2020 | ||
|
||
## Table of Contents | ||
|
||
<!--ts--> | ||
* [ComponentConfig Controller Runtime Support](#componentconfig-controller-runtime-support) | ||
* [Table of Contents](#table-of-contents) | ||
* [Summary](#summary) | ||
* [Motivation](#motivation) | ||
* [Links to Open Issues](#links-to-open-issues) | ||
* [Goals](#goals) | ||
* [Non-Goals/Future Work](#non-goalsfuture-work) | ||
* [Proposal](#proposal) | ||
* [ComponentConfig Load Order](#componentconfig-load-order) | ||
* [User Stories](#user-stories) | ||
* [Controller Author with controller-runtime](#controller-author-with-controller-runtime) | ||
* [Controller Author with kubebuilder (tbd proposal for kubebuilder)](#controller-author-with-kubebuilder-tbd-proposal-for-kubebuilder) | ||
* [Controller User without modifications to config](#controller-user-without-modifications-to-config) | ||
* [Controller User with modifications to config](#controller-user-with-modifications-to-config) | ||
* [Risks and Mitigations](#risks-and-mitigations) | ||
* [Alternatives](#alternatives) | ||
* [Implementation History](#implementation-history) | ||
|
||
<!--te--> | ||
|
||
## Summary | ||
|
||
Currently controllers that use `controller-runtime` need to configure the `ctrl.Manager` by using flags or hard coding values into the initialization methods. Core Kubernetes has started to move away from using flags as a mechanism for configuring components and standardized along the pattern of [`ComponentConfig` or Versioned Component Configuration Files](https://docs.google.com/document/d/1FdaEJUEh091qf5B98HM6_8MS764iXrxxigNIdwHYW9c/edit). This proposal is to bring `ComponentConfig` patterns into `controller-runtime` to allow controller authors to make `go` types backed by apimachinery to unmarshal and configure the `ctrl.Manager` reducing the flags and allowing code based tools to easily configure controllers instead of requiring them to mutate CLI args. | ||
|
||
|
||
## Motivation | ||
|
||
This change is important because: | ||
- it will help make it easier for controllers to be configured by other machine processes | ||
- it will reduce the upfront flags required to start a controller | ||
- allow for more configuration types which flags don't natively support | ||
- allow using and upgrading older configurations avoiding breaking changes in flags | ||
|
||
### Links to Open Issues | ||
|
||
- [Provide a ComponentConfig to tweak the Manager](https://github.com/kubernetes-sigs/controller-runtime/issues/518) | ||
- [Reduce command line flag boilerplate](https://github.com/kubernetes-sigs/controller-runtime/issues/207) | ||
- [Implement ComponentConfig by default & stop using (most) flags](https://github.com/kubernetes-sigs/kubebuilder/issues/722) | ||
|
||
### Goals | ||
|
||
- Provide an interface for pulling configuration data out of exposed `ComponentConfig` types (see below for implementation) | ||
- Provide a new `ctrl.NewFromComponentConfig()` function for initializing a manager | ||
|
||
### Non-Goals/Future Work | ||
|
||
- `kubebuilder` implementation and design in another PR | ||
- Changing the default `controller-runtime` implementation | ||
- Dynamically reloading ComponentConfig object | ||
|
||
## Proposal | ||
|
||
The `ctrl.Manager` _SHOULD_ support loading configurations from `ComponentConfig` like objects. | ||
An interface for that object with getters for the specific configuration parameters is created to bridge existing patterns. | ||
|
||
Without breaking the current `ctrl.NewManager` which uses an exported `ctrl.Options{}` the `manager.go` can expose a new func, `NewFromComponentConfig()` this would be able to loop through the getters to hydrate an internal `ctrl.Options{}` and pass that into `New()`. | ||
|
||
File: _`pkg/manager/manager.go`_ | ||
```golang | ||
// ManagerConfiguration defines what the ComponentConfig object for ControllerRuntime needs to support | ||
type ManagerConfiguration interface { | ||
GetSyncPeriod() *time.Duration | ||
|
||
GetLeaderElection() bool | ||
GetLeaderElectionNamespace() string | ||
GetLeaderElectionID() string | ||
|
||
GetLeaseDuration() *time.Duration | ||
GetRenewDeadline() *time.Duration | ||
GetRetryPeriod() *time.Duration | ||
|
||
GetNamespace() string | ||
GetMetricsBindAddress() string | ||
GetHealthProbeBindAddress() string | ||
|
||
GetReadinessEndpointName() string | ||
GetLivenessEndpointName() string | ||
|
||
GetPort() int | ||
GetHost() string | ||
|
||
GetCertDir() string | ||
} | ||
|
||
func NewFromComponentConfig(config *rest.Config, scheme *runtime.Scheme, managerconfig ManagerConfiguration) (Manager, error) { | ||
options := Options{} | ||
|
||
if scheme != nil { | ||
options.Scheme = scheme | ||
} | ||
|
||
// Loop through getters | ||
if managerconfig.GetLeaderElection() { | ||
managerconfig.GetLeaderElection() | ||
} | ||
// ... | ||
|
||
return New(config, options) | ||
} | ||
``` | ||
|
||
#### ComponentConfig Load Order | ||
|
||
![ComponentConfig Load Order](/designs/images/component-config-load.png) | ||
|
||
Within a separate design (link once created) this will require controller authors to generate a type that implements the `ManagerConfiguration` interface. The following is a sample of what this looks like: | ||
|
||
```golang | ||
package config | ||
|
||
import ( | ||
"time" | ||
|
||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
) | ||
|
||
type ControllerNameConfigurationSpec struct { | ||
LeaderElection ControllerNameConfigurationLeaderElection `json:"leaderElection,omitempty"` | ||
|
||
LeaseDuration *time.Duration `json:"leaseDuration,omitempty"` | ||
RenewDeadline *time.Duration `json:"renewDeadline,omitempty"` | ||
RetryPeriod *time.Duration`json:"retryPeriod,omitempty"` | ||
|
||
Namespace string `json:"namespace,omitempty"` | ||
|
||
MetricsBindAddress string `json:"metricsBindAddress,omitempty"` | ||
|
||
Health ControllerNameConfigurationHealth | ||
|
||
Port int `json:"port,omitempty"` | ||
Host string `json:"host,omitempty"` | ||
|
||
CertDir string `json:"certDir,omitempty"` | ||
} | ||
|
||
type ControllerNameConfigurationLeaderElection struct { | ||
Enabled bool `json:"enabled,omitempty"` | ||
Namespace string `json:"namespace,omitempty"` | ||
ID string `json:"id,omitempty"` | ||
} | ||
|
||
type ControllerNameConfigurationHealth struct { | ||
HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"` | ||
|
||
ReadinessEndpointName string `json:"readinessEndpointName,omitempty"` | ||
LivenessEndpointName string `json:"livenessEndpointName,omitempty"` | ||
} | ||
|
||
type ControllerNameConfiguration struct { | ||
metav1.TypeMeta | ||
|
||
Spec ControllerNameConfigurationSpec `json:"spec"` | ||
} | ||
``` | ||
|
||
With the above `struct` we would need to implement the getters to satisfy the `ManagerConfiguration` interface, these could be automatically generated when you `init` or `create componentconfig` from the `kubebuilder` CLI. This could be something as simple as the following. | ||
|
||
```golang | ||
package config | ||
|
||
import ( | ||
"time" | ||
) | ||
|
||
func (in *ControllerNameConfiguration) GetSyncPeriod() *time.Duration {} | ||
func (in *ControllerNameConfiguration) GetLeaderElection() bool {} | ||
func (in *ControllerNameConfiguration) GetLeaderElectionNamespace() string {} | ||
func (in *ControllerNameConfiguration) GetLeaderElectionID() string {} | ||
|
||
func (in *ControllerNameConfiguration) GetLeaseDuration() *time.Duration {} | ||
func (in *ControllerNameConfiguration) GetRenewDeadline() *time.Duration {} | ||
func (in *ControllerNameConfiguration) GetRetryPeriod() *time.Duration {} | ||
|
||
func (in *ControllerNameConfiguration) GetNamespace() string {} | ||
func (in *ControllerNameConfiguration) GetMetricsBindAddress() string {} | ||
func (in *ControllerNameConfiguration) GetHealthProbeBindAddress() string {} | ||
|
||
func (in *ControllerNameConfiguration) GetReadinessEndpointName() string {} | ||
func (in *ControllerNameConfiguration) GetLivenessEndpointName() string {} | ||
|
||
func (in *ControllerNameConfiguration) GetPort() int {} | ||
func (in *ControllerNameConfiguration) GetHost() string {} | ||
|
||
func (in *ControllerNameConfiguration) GetCertDir() string {} | ||
``` | ||
|
||
Besides the implementation of the `ComponentConfig` The controller author as it stands would also need to implement the unmarshalling of the `ConfigMap` into the `ComponentConfig`, for this `controller-runtime` could expose helper methods to load a file from disk, unmarshal to the struct and pass the pointer into the `NewFromComponentConfig()` to return the `ctrl.Manager` | ||
|
||
### User Stories | ||
|
||
#### Controller Author with `controller-runtime` | ||
|
||
- Implement `ComponentConfig` type | ||
- Implement `ManagerConfiguration` interface for `ComponentConfig` object | ||
- Set up `ConfigMap` unmarshalling into `ComponentConfig` type | ||
- Initialize `ctrl.Manager` with `NewFromComponentConfig` | ||
- Build custom controller as usual | ||
|
||
#### Controller Author with `kubebuilder` (tbd proposal for `kubebuilder`) | ||
|
||
- Initialize `kubebuilder` project using `--component-config-name=XYZConfiguration` | ||
- Build custom controller as usual | ||
|
||
#### Controller User without modifications to config | ||
|
||
_Provided that the controller provides manifests_ | ||
|
||
- Apply the controller to the cluster | ||
- Deploy custom resources | ||
|
||
#### Controller User with modifications to config | ||
|
||
- _Following from previous example without changes_ | ||
- Create a new `ConfigMap` for changes | ||
- Modify the `controller-runtime` pod to use the new `ConfigMap` | ||
- Apply the controller to the cluster | ||
- Deploy custom resources | ||
|
||
|
||
### Risks and Mitigations | ||
|
||
- Given that this isn't changing the core Manager initialization for `controller-runtime` it's fairly low risk | ||
- If the underlaying `ctrl.Options{}` | ||
|
||
## Alternatives | ||
|
||
* `NewFromComponentConfig()` could load the object from disk and hydrate the `ComponentConfig` type | ||
|
||
## Implementation History | ||
|
||
- [x] 02/19/2020: Proposed idea in an issue or [community meeting] | ||
- [x] 02/24/2020: Proposal submitted to `controller-runtime` | ||
|
||
|
||
<!-- Links --> | ||
[community meeting]: https://docs.google.com/document/d/1Ih-2cgg1bUrLwLVTB9tADlPcVdgnuMNBGbUl4D-0TIk |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.