adding defaultcontrollercomponentconfig design

Signed-off-by: Chris Hein <me@chrishein.com>
kubernetes-sigs · Mar 3, 2020 · d74b31d · d74b31d
1 parent 4ec6da5
commit d74b31d
Showing 1 changed file with 122 additions and 44 deletions.
diff --git a/designs/component-config.md b/designs/component-config.md
@@ -1,7 +1,7 @@
 # ComponentConfig Controller Runtime Support
-
 Author: @christopherhein
-Last Updated on: 02/24/2020
+
+Last Updated on: 03/02/2020
 
 ## Table of Contents
 
@@ -15,46 +15,51 @@ Last Updated on: 02/24/2020
          * [Non-Goals/Future Work](#non-goalsfuture-work)
       * [Proposal](#proposal)
             * [ComponentConfig Load Order](#componentconfig-load-order)
-         * [User Stories](#user-stories)
-            * [Controller Author with controller-runtime](#controller-author-with-controller-runtime)
-            * [Controller Author with kubebuilder (tbd proposal for kubebuilder)](#controller-author-with-kubebuilder-tbd-proposal-for-kubebuilder)
-            * [Controller User without modifications to config](#controller-user-without-modifications-to-config)
-            * [Controller User with modifications to config](#controller-user-with-modifications-to-config)
-         * [Risks and Mitigations](#risks-and-mitigations)
+            * [Default ComponentConfig Type](#default-componentconfig-type)
+               * [Caveats](#caveats)
+            * [Kubebuilder Scaffolding Example](#kubebuilder-scaffolding-example)
+      * [User Stories](#user-stories)
+         * [Controller Author with controller-runtime](#controller-author-with-controller-runtime)
+         * [Controller Author with kubebuilder (tbd proposal for kubebuilder)](#controller-author-with-kubebuilder-tbd-proposal-for-kubebuilder)
+         * [Controller User without modifications to config](#controller-user-without-modifications-to-config)
+         * [Controller User with modifications to config](#controller-user-with-modifications-to-config)
+      * [Risks and Mitigations](#risks-and-mitigations)
       * [Alternatives](#alternatives)
       * [Implementation History](#implementation-history)
 
 <!--te-->
 
 ## Summary
 
-Currently controllers that use `controller-runtime` need to configure the `ctrl.Manager` by using flags or hard coding values into the initialization methods. Core Kubernetes has started to move away from using flags as a mechanism for configuring components and standardized along the pattern of [`ComponentConfig` or Versioned Component Configuration Files](https://docs.google.com/document/d/1FdaEJUEh091qf5B98HM6_8MS764iXrxxigNIdwHYW9c/edit). This proposal is to bring `ComponentConfig` patterns into `controller-runtime` to allow controller authors to make `go` types backed by apimachinery to unmarshal and configure the `ctrl.Manager` reducing the flags and allowing code based tools to easily configure controllers instead of requiring them to mutate CLI args.
+Currently controllers that use `controller-runtime` need to configure the `ctrl.Manager` by using flags or hardcoding values into the initialization methods. Core Kubernetes has started to move away from using flags as a mechanism for configuring components and standardized along the pattern of [`ComponentConfig` or Versioned Component Configuration Files](https://docs.google.com/document/d/1FdaEJUEh091qf5B98HM6_8MS764iXrxxigNIdwHYW9c/edit). This proposal is to bring `ComponentConfig` patterns into `controller-runtime` to allow controller authors to make `go` types backed by `apimachinery` to unmarshal and configure the `ctrl.Manager` reducing the flags and allowing code based tools to easily configure controllers instead of requiring them to mutate CLI args.
 
 
 ## Motivation
 
 This change is important because: 
 - it will help make it easier for controllers to be configured by other machine processes
-- it will reduce the upfront flags required to start a controller
+- it will reduce the required flags required to start a controller
 - allow for more configuration types which flags don't natively support
 - allow using and upgrading older configurations avoiding breaking changes in flags
 
 ### Links to Open Issues
 
-- [Provide a ComponentConfig to tweak the Manager](https://github.com/kubernetes-sigs/controller-runtime/issues/518)
-- [Reduce command line flag boilerplate](https://github.com/kubernetes-sigs/controller-runtime/issues/207)
-- [Implement ComponentConfig by default & stop using (most) flags](https://github.com/kubernetes-sigs/kubebuilder/issues/722)
+- [#518 Provide a ComponentConfig to tweak the Manager](https://github.com/kubernetes-sigs/controller-runtime/issues/518)
+- [#207 Reduce command line flag boilerplate](https://github.com/kubernetes-sigs/controller-runtime/issues/207)
+- [#722 Implement ComponentConfig by default & stop using (most) flags](https://github.com/kubernetes-sigs/kubebuilder/issues/722)
 
 ### Goals
 
 - Provide an interface for pulling configuration data out of exposed `ComponentConfig` types (see below for implementation)
 - Provide a new `ctrl.NewFromComponentConfig()` function for initializing a manager
+- Provide a `DefaultControllerConfig` to make the switch easier
 
 ### Non-Goals/Future Work
 
 - `kubebuilder` implementation and design in another PR
 - Changing the default `controller-runtime` implementation
 - Dynamically reloading ComponentConfig object
+- Providing `flags` interface and overrides
 
 ## Proposal
 
@@ -90,18 +95,22 @@ type ManagerConfiguration interface {
 	GetCertDir() string
 }
 
-func NewFromComponentConfig(config *rest.Config, scheme *runtime.Scheme, managerconfig ManagerConfiguration) (Manager, error) {
-        options := Options{}
+func NewFromComponentConfig(config *rest.Config, scheme *runtime.Scheme,filename string, managerconfig ManagerConfiguration) (Manager, error) {
+	codecs := serializer.NewCodecFactory(scheme)
+    if err := decodeComponentConfigFileInto(codecs, filename, componentconfig); err != nil {
+
+	}
+	options := Options{}
 
-		if scheme != nil {
-			options.Scheme = scheme
-		}
+	if scheme != nil {
+		options.Scheme = scheme
+	}
 
-		// Loop through getters
-        if managerconfig.GetLeaderElection() {
-                managerconfig.GetLeaderElection()
-        }
-        // ...
+	// Loop through getters
+	if managerconfig.GetLeaderElection() {
+			managerconfig.GetLeaderElection()
+	}
+	// ...
 
 	return New(config, options)
 }
@@ -111,6 +120,84 @@ func NewFromComponentConfig(config *rest.Config, scheme *runtime.Scheme, manager
 
 ![ComponentConfig Load Order](/designs/images/component-config-load.png)
 
+#### Default ComponentConfig Type
+
+To enable `controller-runtime` to have a default `ComponentConfig` struct which can be used instead of requiring each controller or extension to build it's own `ComponentConfig` type, we can create a `DefaultControllerConfiguration` type which can exist in `pkg/api/v1alpha1/types.go`. This will allow the controller authors to use this before needing to implement their own type with additional configs.
+
+```golang
+package v1alpha1
+
+import (
+	"time"
+
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	configv1alpha1 "sigs.k8s.io/component-base/config/v1alpha1"
+)
+
+// DefaultControllerConfigurationSpec defines the desired state of DefaultControllerConfiguration
+type DefaultControllerConfigurationSpec struct {
+	SyncPeriod *time.Duration `json:"syncPeriod,omitempty"`
+
+	LeaderElection configv1alpha1.LeaderElectionConfiguration `json:"leaderElection,omitempty"`
+
+	MetricsBindAddress string `json:"metricsBindAddress,omitempty"`
+
+	Health DefaultControllerConfigurationHealth `json:"health,omitempty"`
+
+	Port *int   `json:"port,omitempty"`
+	Host string `json:"host,omitempty"`
+
+	CertDir string `json:"certDir,omitempty"`
+}
+
+// DefaultControllerConfigurationHealth defines the health configs
+type DefaultControllerConfigurationHealth struct {
+	HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`
+
+	ReadinessEndpointName string `json:"readinessEndpointName,omitempty"`
+	LivenessEndpointName  string `json:"livenessEndpointName,omitempty"`
+}
+
+// DefaultControllerConfiguration is the Schema for the DefaultControllerConfigurations API
+type DefaultControllerConfiguration struct {
+	metav1.TypeMeta `json:",inline"`
+
+	Spec   DefaultControllerConfigurationSpec   `json:"spec,omitempty"`
+}
+```
+
+This would allow a controller author to use this struct with any config that supports the json/yaml structure. For example a controller author could define their `Kind` as `FoobarControllerConfiguration` and have it defined as the following.
+
+```yaml
+# config.yaml
+apiVersion: somedomain.io/v1alpha1
+kind: FoobarControllerConfiguration
+spec:
+  port: 9443
+  metricsBindAddress: ":8080"
+  leaderElection:
+    leaderElect: false
+```
+
+Given the following config and `DefaultControllerConfiguration` we'd be able to initialize the controller using the following.
+
+
+```golang
+	mgr, err := ctrl.NewManagerFromComponentConfig(ctrl.GetConfigOrDie(), scheme, configname, &defaultv1alpha1.DefaultControllerConfiguration{})
+	if err != nil {
+		// ...
+	}
+```
+
+The above example uses `configname` which is the name of the file to load the configuration from and uses `scheme` to get the specific serializer, eg `serializer.NewCodecFactory(scheme)`. This will allow the configuration to be unmarshalled into the `runtime.Object` type and passed into the 
+`ctrl.NewManagerFromComponentConfig()` as a `ManagerConfiguration` interface.
+
+##### Caveats
+
+> ⚠️ Using `DecodeComponentConfigFileInto` does not support the overrides for flags, this is something that is left up to the controller author since they could be using many different flagging interfaces. eg [`flag`](https://golang.org/pkg/flag/), [`pflag`](https://godoc.org/github.com/spf13/pflag), [`flagnum`](https://godoc.org/github.com/luci/luci-go/common/flag/flagenum) and `controller-runtime` should be agnostic to the CLI implementation.
+
+#### Kubebuilder Scaffolding Example
+
 Within a separate design (link once created) this will require controller authors to generate a type that implements the `ManagerConfiguration` interface. The following is a sample of what this looks like:
 
 ```golang
@@ -120,16 +207,13 @@ import (
   "time"
 
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	configv1alpha1 "sigs.k8s.io/component-base/config/v1alpha1"
 ) 
 
 type ControllerNameConfigurationSpec struct {
-	LeaderElection ControllerNameConfigurationLeaderElection `json:"leaderElection,omitempty"`
-
-	LeaseDuration *time.Duration `json:"leaseDuration,omitempty"`
-	RenewDeadline *time.Duration `json:"renewDeadline,omitempty"`
-	RetryPeriod *time.Duration`json:"retryPeriod,omitempty"`
-
-	Namespace string `json:"namespace,omitempty"`
+	SyncPeriod *time.Duration `json:"syncPeriod,omitempty"`
+
+	LeaderElection configv1alpha1.LeaderElectionConfiguration `json:"leaderElection,omitempty"`
 
 	MetricsBindAddress string `json:"metricsBindAddress,omitempty"`
 
@@ -141,12 +225,6 @@ type ControllerNameConfigurationSpec struct {
 	CertDir string `json:"certDir,omitempty"`
 }
 
-type ControllerNameConfigurationLeaderElection struct {
-	Enabled bool `json:"enabled,omitempty"`
-	Namespace string `json:"namespace,omitempty"`
-	ID string `json:"id,omitempty"`
-}
-
 type ControllerNameConfigurationHealth struct {
 	HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`
 
@@ -194,29 +272,29 @@ func (in *ControllerNameConfiguration) GetCertDir() string {}
 
 Besides the implementation of the `ComponentConfig` The controller author as it stands would also need to implement the unmarshalling of the `ConfigMap` into the `ComponentConfig`, for this `controller-runtime` could expose helper methods to load a file from disk, unmarshal to the struct and pass the pointer into the `NewFromComponentConfig()` to return the `ctrl.Manager`
 
-### User Stories
+## User Stories
 
-#### Controller Author with `controller-runtime`
+### Controller Author with `controller-runtime`
 
 - Implement `ComponentConfig` type
 - Implement `ManagerConfiguration` interface for `ComponentConfig` object
 - Set up `ConfigMap` unmarshalling into `ComponentConfig` type
 - Initialize `ctrl.Manager` with `NewFromComponentConfig`
 - Build custom controller as usual
 
-#### Controller Author with `kubebuilder` (tbd proposal for `kubebuilder`)
+### Controller Author with `kubebuilder` (tbd proposal for `kubebuilder`)
 
 - Initialize `kubebuilder` project using `--component-config-name=XYZConfiguration`
 - Build custom controller as usual
 
-#### Controller User without modifications to config
+### Controller User without modifications to config
 
 _Provided that the controller provides manifests_
 
 - Apply the controller to the cluster
 - Deploy custom resources
 
-#### Controller User with modifications to config
+### Controller User with modifications to config
 
 - _Following from previous example without changes_
 - Create a new `ConfigMap` for changes
@@ -225,19 +303,19 @@ _Provided that the controller provides manifests_
 - Deploy custom resources
 
 
-### Risks and Mitigations
+## Risks and Mitigations
 
 - Given that this isn't changing the core Manager initialization for `controller-runtime` it's fairly low risk
-- If the underlaying `ctrl.Options{}`
 
 ## Alternatives
 
-* `NewFromComponentConfig()` could load the object from disk and hydrate the `ComponentConfig` type
+* `NewFromComponentConfig()` could load the object from disk based on the file name and hydrate the `ComponentConfig` type.
 
 ## Implementation History
 
 - [x] 02/19/2020: Proposed idea in an issue or [community meeting]
 - [x] 02/24/2020: Proposal submitted to `controller-runtime`
+- [x] 03/02/2020: Updated with default `DefaultControllerConfiguration`
 
 
 <!-- Links -->