Skip to content

Commit

Permalink
Adding component config proposal
Browse files Browse the repository at this point in the history
Signed-off-by: Chris Hein <me@chrishein.com>
  • Loading branch information
christopherhein committed Feb 26, 2020
1 parent 8507812 commit 041b120
Show file tree
Hide file tree
Showing 2 changed files with 244 additions and 0 deletions.
244 changes: 244 additions & 0 deletions designs/component-config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,244 @@
# ComponentConfig Controller Runtime Support

Author: @christopherhein
Last Updated on: 02/24/2020

## Table of Contents

<!--ts-->
* [ComponentConfig Controller Runtime Support](#componentconfig-controller-runtime-support)
* [Table of Contents](#table-of-contents)
* [Summary](#summary)
* [Motivation](#motivation)
* [Links to Open Issues](#links-to-open-issues)
* [Goals](#goals)
* [Non-Goals/Future Work](#non-goalsfuture-work)
* [Proposal](#proposal)
* [ComponentConfig Load Order](#componentconfig-load-order)
* [User Stories](#user-stories)
* [Controller Author with controller-runtime](#controller-author-with-controller-runtime)
* [Controller Author with kubebuilder (tbd proposal for kubebuilder)](#controller-author-with-kubebuilder-tbd-proposal-for-kubebuilder)
* [Controller User without modifications to config](#controller-user-without-modifications-to-config)
* [Controller User with modifications to config](#controller-user-with-modifications-to-config)
* [Risks and Mitigations](#risks-and-mitigations)
* [Alternatives](#alternatives)
* [Implementation History](#implementation-history)

<!--te-->

## Summary

Currently controllers that use `controller-runtime` need to configure the `ctrl.Manager` by using flags or hard coding values into the initialization methods. Core Kubernetes has started to move away from using flags as a mechanism for configuring components and standardized along the pattern of [`ComponentConfig` or Versioned Component Configuration Files](https://docs.google.com/document/d/1FdaEJUEh091qf5B98HM6_8MS764iXrxxigNIdwHYW9c/edit). This proposal is to bring `ComponentConfig` patterns into `controller-runtime` to allow controller authors to make `go` types backed by apimachinery to unmarshal and configure the `ctrl.Manager` reducing the flags and allowing code based tools to easily configure controllers instead of requiring them to mutate CLI args.


## Motivation

This change is important because:
- it will help make it easier for controllers to be configured by other machine processes
- it will reduce the upfront flags required to start a controller
- allow for more configuration types which flags don't natively support
- allow using and upgrading older configurations avoiding breaking changes in flags

### Links to Open Issues

- [Provide a ComponentConfig to tweak the Manager](https://github.com/kubernetes-sigs/controller-runtime/issues/518)
- [Reduce command line flag boilerplate](https://github.com/kubernetes-sigs/controller-runtime/issues/207)
- [Implement ComponentConfig by default & stop using (most) flags](https://github.com/kubernetes-sigs/kubebuilder/issues/722)

### Goals

- Provide an interface for pulling configuration data out of exposed `ComponentConfig` types (see below for implementation)
- Provide a new `ctrl.NewFromComponentConfig()` function for initializing a manager

### Non-Goals/Future Work

- `kubebuilder` implementation and design in another PR
- Changing the default `controller-runtime` implementation
- Dynamically reloading ComponentConfig object

## Proposal

The `ctrl.Manager` _SHOULD_ support loading configurations from `ComponentConfig` like objects.
An interface for that object with getters for the specific configuration parameters is created to bridge existing patterns.

Without breaking the current `ctrl.NewManager` which uses an exported `ctrl.Options{}` the `manager.go` can expose a new func, `NewFromComponentConfig()` this would be able to loop through the getters to hydrate an internal `ctrl.Options{}` and pass that into `New()`.

File: _`pkg/manager/manager.go`_
```golang
// ManagerConfiguration defines what the ComponentConfig object for ControllerRuntime needs to support
type ManagerConfiguration interface {
GetSyncPeriod() *time.Duration

GetLeaderElection() bool
GetLeaderElectionNamespace() string
GetLeaderElectionID() string

GetLeaseDuration() *time.Duration
GetRenewDeadline() *time.Duration
GetRetryPeriod() *time.Duration

GetNamespace() string
GetMetricsBindAddress() string
GetHealthProbeBindAddress() string

GetReadinessEndpointName() string
GetLivenessEndpointName() string

GetPort() int
GetHost() string

GetCertDir() string
}

func NewFromComponentConfig(config *rest.Config, scheme *runtime.Scheme, managerconfig ManagerConfiguration) (Manager, error) {
options := Options{}

if scheme != nil {
options.Scheme = scheme
}

// Loop through getters
if managerconfig.GetLeaderElection() {
managerconfig.GetLeaderElection()
}
// ...

return New(config, options)
}
```

#### ComponentConfig Load Order

![ComponentConfig Load Order](/designs/images/component-config-load.png)

Within a separate design (link once created) this will require controller authors to generate a type that implements the `ManagerConfiguration` interface. The following is a sample of what this looks like:

```golang
package config

import (
"time"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

type ControllerNameConfigurationSpec struct {
LeaderElection ControllerNameConfigurationLeaderElection `json:"leaderElection,omitempty"`

LeaseDuration *time.Duration `json:"leaseDuration,omitempty"`
RenewDeadline *time.Duration `json:"renewDeadline,omitempty"`
RetryPeriod *time.Duration`json:"retryPeriod,omitempty"`

Namespace string `json:"namespace,omitempty"`

MetricsBindAddress string `json:"metricsBindAddress,omitempty"`

Health ControllerNameConfigurationHealth

Port int `json:"port,omitempty"`
Host string `json:"host,omitempty"`

CertDir string `json:"certDir,omitempty"`
}

type ControllerNameConfigurationLeaderElection struct {
Enabled bool `json:"enabled,omitempty"`
Namespace string `json:"namespace,omitempty"`
ID string `json:"id,omitempty"`
}

type ControllerNameConfigurationHealth struct {
HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`

ReadinessEndpointName string `json:"readinessEndpointName,omitempty"`
LivenessEndpointName string `json:"livenessEndpointName,omitempty"`
}

type ControllerNameConfiguration struct {
metav1.TypeMeta

Spec ControllerNameConfigurationSpec `json:"spec"`
}
```

With the above `struct` we would need to implement the getters to satisfy the `ManagerConfiguration` interface, these could be automatically generated when you `init` or `create componentconfig` from the `kubebuilder` CLI. This could be something as simple as the following.

```golang
package config

import (
"time"
)

func (in *ControllerNameConfiguration) GetSyncPeriod() *time.Duration {}
func (in *ControllerNameConfiguration) GetLeaderElection() bool {}
func (in *ControllerNameConfiguration) GetLeaderElectionNamespace() string {}
func (in *ControllerNameConfiguration) GetLeaderElectionID() string {}

func (in *ControllerNameConfiguration) GetLeaseDuration() *time.Duration {}
func (in *ControllerNameConfiguration) GetRenewDeadline() *time.Duration {}
func (in *ControllerNameConfiguration) GetRetryPeriod() *time.Duration {}

func (in *ControllerNameConfiguration) GetNamespace() string {}
func (in *ControllerNameConfiguration) GetMetricsBindAddress() string {}
func (in *ControllerNameConfiguration) GetHealthProbeBindAddress() string {}

func (in *ControllerNameConfiguration) GetReadinessEndpointName() string {}
func (in *ControllerNameConfiguration) GetLivenessEndpointName() string {}

func (in *ControllerNameConfiguration) GetPort() int {}
func (in *ControllerNameConfiguration) GetHost() string {}

func (in *ControllerNameConfiguration) GetCertDir() string {}
```

Besides the implementation of the `ComponentConfig` The controller author as it stands would also need to implement the unmarshalling of the `ConfigMap` into the `ComponentConfig`, for this `controller-runtime` could expose helper methods to load a file from disk, unmarshal to the struct and pass the pointer into the `NewFromComponentConfig()` to return the `ctrl.Manager`

### User Stories

#### Controller Author with `controller-runtime`

- Implement `ComponentConfig` type
- Implement `ManagerConfiguration` interface for `ComponentConfig` object
- Set up `ConfigMap` unmarshalling into `ComponentConfig` type
- Initialize `ctrl.Manager` with `NewFromComponentConfig`
- Build custom controller as usual

#### Controller Author with `kubebuilder` (tbd proposal for `kubebuilder`)

- Initialize `kubebuilder` project using `--component-config-name=XYZConfiguration`
- Build custom controller as usual

#### Controller User without modifications to config

_Provided that the controller provides manifests_

- Apply the controller to the cluster
- Deploy custom resources

#### Controller User with modifications to config

- _Following from previous example without changes_
- Create a new `ConfigMap` for changes
- Modify the `controller-runtime` pod to use the new `ConfigMap`
- Apply the controller to the cluster
- Deploy custom resources


### Risks and Mitigations

- Given that this isn't changing the core Manager initialization for `controller-runtime` it's fairly low risk
- If the underlaying `ctrl.Options{}`

## Alternatives

* `NewFromComponentConfig()` could load the object from disk and hydrate the `ComponentConfig` type

## Implementation History

- [x] 02/19/2020: Proposed idea in an issue or [community meeting]
- [x] 02/24/2020: Proposal submitted to `controller-runtime`


<!-- Links -->
[community meeting]: https://docs.google.com/document/d/1Ih-2cgg1bUrLwLVTB9tADlPcVdgnuMNBGbUl4D-0TIk
Binary file added designs/images/component-config-load.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 041b120

Please sign in to comment.