Add a design document to filter cache by field

The controller-runtime is caching all the resource instance even if not
all of them end up being reconciled, this design document describe a
proposal to fix part of that problem.

Signed-off-by: Quique Llorente <ellorent@redhat.com>

designs/filter-cache-by-field.md

@@ -0,0 +1,217 @@
+Use cache ListWatch selectors
+===================
+## Motivation
+
+The controller-runtime subscribe to changes using a cache mechanism that get
+updated using kubernetes informers, those informers implement a `ListWatch`
+that specify the namespace and type specified at controller builder time.
+
+So the only possible filtering of that cache is namespace and resource type,
+this means that if some user is interested only on Reconcile some resource with
+specific fields it will still cache all the instances on that type increasing
+the memory and CPU load. This also get worse in case controller-runtime is
+deployed as a daemonset instead of a deployment since all the nodes with
+daemonset's pods will expect to receive notification of all the instances and
+that can end with scalation problems depending on the type or resource it's
+watching.
+
+As side effect of this proposal the request that does not match the filtered
+will not go over the cache so they will increase inter cluster node traffic and
+master CPU and memory load, so users has to be warned that if this filtering
+is used they know it and they are careful with the request they are doing.
+
+The alternative that can be done with the current controller-runtme is
+to implement a custom cache that do the filtering since cache is pluggable
+into the controller-runtime [1]
+
+This proposal is related to the following issue [2]
+
+## Proposal 1
+
+Increase `cache.Options` as follow:
+
+```golang
+type Options struct {
+	Scheme                  *runtime.Scheme
+	Mapper                  meta.RESTMapper
+	Resync                  *time.Duration
+	Namespace               string
+	FieldSelectorByResource map[schema.GroupResource]fields.Selector
+}
+```
+
+Incrase `manager.Options` as follow:
+
+```golang
+type Options struct {
+	CacheFieldSelectorByResource map[schema.GroupResource]fields.Selector
+```
+
+Pass the new manager options field field cache options and this
+is passed to informer's ListWatch and add the filtering option:
+
+```golang
+
+# At pkg/cluster/cluster.go
+
+cache, err := options.NewCache(config, cache.Options{Scheme: options.Scheme, Mapper: mapper, Resync: options.SyncPeriod, Namespace: options.Namespace, FieldSelectorByResource: options.CacheFieldSelectorByResource})
+
+
+# At pkg/cache/internal/informers_map.go
+
+func (ip *specificInformersMap) findFieldSelectorByGVR(gvr schema.GroupVersionResource) string {
+	gr := schema.GroupResource{Group: gvr.Group, Resource: gvr.Resource}
+	selctr := ip.fieldSelectorByResource[gr]
+    if selctr == nil {
+        return nil
+    }
+    return selctr.String()
+}
+
+ListFunc: func(opts metav1.ListOptions) (runtime.Object, error) {
+			opts.FieldSelector = ip.findFieldSelectorByGVR(mapping.Resource)
+...
+```
+
+Here is a PR with the implementatin at the `pkg/cache` part [3]
+
+### Example
+
+Users will change the Manager options, similar to namespace to specify what
+Fields they want to limit the cache to by resource.
+
+```golang
+ ctrlOptions := ctrl.Options{
+     ...
+        CacheFieldSelectorByResource: map[schema.GroupResource]string{
+							{Group: "", Resource: "nodes"}: fiels.SelectorFromSet(fields.Set{"metadata.name": "node01"}),
+							{Group: "", Resource: "nodenetworkstate"}: fiels.SelectorFromSet(fields.Set{"metadata.name": "node01"}),
+						},
+		}
+     ...
+    }
+```
+
+
+## Proposal 1
+
+Create a small `pkg/selector` package to the struct to define label/field
+selector and some helper functions
+
+```golang
+package selector
+
+import (
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/fields"
+	"k8s.io/apimachinery/pkg/labels"
+)
+
+// Selector specify the label/field selector to fill in ListOptions
+type Selector struct {
+	Label labels.Selector
+	Field fields.Selector
+}
+
+// FillInListOpts fill in ListOptions LabelSelector and FieldSelector if needed
+func (s Selector) FillInListOpts(listOpts *metav1.ListOptions) {
+...
+}
+```
+
+Increase builder and add a `WithSelector` to `ForInput`
+
+```golang
+
+// {{{ Selector Options
+
+// WithSelector the given predicates list.
+func WithSelector(selector selector.Selector) Selector {
+	return Selector{
+		selector: selector,
+	}
+}
+
+// Selector filters events before enqueuing the keys.
+type Selector struct {
+	selector selector.Selector
+}
+
+// ApplyToFor applies this configuration to the given ForInput options.
+func (w Selector) ApplyToFor(opts *ForInput) {
+	opts.selector = w.selector
+}
+
+// ApplyToOwns applies this configuration to the given OwnsInput options.
+func (w Selector) ApplyToOwns(opts *OwnsInput) {
+	opts.selector = w.selector
+}
+
+// ApplyToWatches applies this configuration to the given WatchesInput options.
+func (w Selector) ApplyToWatches(opts *WatchesInput) {
+	opts.selector = w.selector
+}
+
+var _ ForOption = &Selector{}
+var _ OwnsOption = &Selector{}
+var _ WatchesOption = &Selector{}
+
+// }}}
+```
+
+Change `Informers` interface to add a `SetSelector` function that will pass
+the selector specified at builder to the `Kind` cache.
+
+```golang
+type Informers interface {
+	// SetSelector apply a selector at informer ListWatch to fillter in the
+	// the cache
+	SetSelector(obj client.Object, selector selector.Selector) error
+```
+
+After traversing the cache instruct ListWatch to use the Selector
+
+```golang
+return &cache.ListWatch{
+		ListFunc: func(opts metav1.ListOptions) (runtime.Object, error) {
+			ip.findSelectorByKind(gvk).FillInListOpts(&opts)
+            ...
+		},
+		// Setup the watch function
+		WatchFunc: func(opts metav1.ListOptions) (watch.Interface, error) {
+			ip.findSelectorByKind(gvk).FillInListOpts(&opts)
+            ...
+```
+
+A reference implementation of this proposal is done at [4]
+
+### Example
+
+Users will set the Selector inside the controllers at builder time
+
+
+```golang
+bldr := ControllerManagedBy(m).
+				For(&appsv1.Deployment{}, WithSelector(
+					selector.Selector{
+						Field: fields.SelectorFromSet(fields.Set{
+							"metadata.name": "deploy-name-6",
+						}),
+					},
+				)).
+				Owns(&appsv1.ReplicaSet{}, WithSelector(
+					selector.Selector{
+						Field: fields.SelectorFromSet(fields.Set{
+							"metadata.name": "rs-name-6",
+						}),
+					},
+				))
+
+```
+
+
+
+[1] https://github.com/nmstate/kubernetes-nmstate/pull/687
+[2] https://github.com/kubernetes-sigs/controller-runtime/issues/244
+[3] https://github.com/kubernetes-sigs/controller-runtime/pull/1404
+[4] https://github.com/kubernetes-sigs/controller-runtime/pull/1425