Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy packages over from kubebuilder #2

Merged
merged 88 commits into from
Jun 7, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
1f0f848
Fork [apiserver-builder](https://github.com/kubernetes-incubator/apis…
pwittrock Mar 15, 2018
94d0b96
Add informerer factory to default args
pwittrock Mar 15, 2018
a68b9e9
Add support for declaratively starting Kubernetes informers from the …
pwittrock Mar 15, 2018
2765af0
Refactor inject package so it will be possible to evolve in a backwar…
pwittrock Mar 15, 2018
8ae709a
Add eventrecorder
pwittrock Mar 16, 2018
7b682be
Add predicates for handling objects
pwittrock Mar 16, 2018
7590e2a
Add godocs for inject packages
pwittrock Mar 17, 2018
8f032c2
Update controller libraries
pwittrock Mar 19, 2018
055ff5b
Update example names
pwittrock Mar 19, 2018
fbee966
Move admission package to internal until it is ready
pwittrock Mar 19, 2018
9b27764
Address comments from out-of-band review
pwittrock Mar 21, 2018
e7bb2cb
Add string method for ReconcileKey
natronq Mar 23, 2018
af5684e
Change watching events to use interface instead of struct for more fl…
pwittrock Apr 2, 2018
9e9b328
Support mapping an object to multiple keys
pwittrock Apr 3, 2018
14e7d63
Support for ReconcileKeys in transformations
pwittrock Apr 3, 2018
170e572
update comments
pwittrock Apr 3, 2018
2166171
Merge pull request #43 from pwittrock/master
Apr 4, 2018
daf94d6
Fix some issues in types.ReconcileKey
briantkennedy Apr 3, 2018
bb6df84
Documentation for eventhandlers.Path
pwittrock Apr 5, 2018
747125c
standardize kubebuilder codegeneration labels to have +kubebuilder: p…
Liujingfang1 Apr 9, 2018
861678d
Merge pull request #66 from Liujingfang1/tagprefix
Apr 10, 2018
36a1bcf
fixed infinite loop in watchControllerOf API
droot Apr 17, 2018
cbc7871
Update inject args kubernetes Clientset to Interface.
briantkennedy Apr 19, 2018
0749bbb
Embbed type in the example function to improve printing
Apr 20, 2018
f8f1ce4
Embbed more example types for docs
Apr 20, 2018
33e7ce0
rebase 1.10 k8s
Apr 24, 2018
ea534eb
Refactor kubebuilder integration test start
Apr 27, 2018
02917e1
Add support for +kubebuilder:categories
pmorie May 1, 2018
415380c
Improve godocs for generated tags
pwittrock May 3, 2018
ea2ab7f
Don't use apiserver-builder to generate reference documentation.
pwittrock May 7, 2018
a97b49f
added dummy import
droot May 9, 2018
b7add45
Provide helpful error message if informer isn't found in manager.
pwittrock May 10, 2018
6c1a8cb
make rbac rules example consistent
pwittrock May 10, 2018
e8f22be
Add validation for informers and rbac annotations.
pwittrock May 10, 2018
9873559
Add support for CRD validaiton
May 14, 2018
8826636
pkg/controller: re-add to the queue failed reconcile keys
jhernandezb May 26, 2018
b3d0565
Controller Libraries V2
pwittrock May 19, 2018
9d4e449
Merge pull request #232 from pwittrock/interface
May 30, 2018
ff1155b
Controller Enqueue / Source implementation
pwittrock Jun 1, 2018
c656219
fix examples
pwittrock Jun 2, 2018
cfe05aa
Implement workerqueue
pwittrock Jun 2, 2018
100e513
Implement watch
pwittrock Jun 2, 2018
acaa901
[WIP] informer prototype and base main
DirectXMan12 Jun 2, 2018
6d5aed0
Update Source definition and implementation to work with InformersCache.
pwittrock Jun 3, 2018
09489a8
Unit tests for source package
pwittrock Jun 4, 2018
a6f9dcb
Additional test coverage for ctrl packagbe.
pwittrock Jun 4, 2018
80ea715
Logging with modified logr
DirectXMan12 Jun 4, 2018
57487d1
controller-platform tests for eventhandlers
pwittrock Jun 4, 2018
29c0fdd
Lazy logging initialization
DirectXMan12 Jun 4, 2018
240308e
controller-platform tests
pwittrock Jun 4, 2018
e35cd9f
controller-platform unit tests
pwittrock Jun 4, 2018
1dd99bf
Integration test for ctrl package
pwittrock Jun 5, 2018
dc100c4
Fix compile issues with sources tests
pwittrock Jun 5, 2018
fb8a309
controller-platform implement dependency injection for controller
pwittrock Jun 5, 2018
39bb0a6
Client Interface and Cache Access
DirectXMan12 Jun 5, 2018
7e82f29
controller-platform controller integration tests for update and delete
pwittrock Jun 5, 2018
1280e40
controller-platform refactor informers names
pwittrock Jun 5, 2018
ef0fa87
Continue informer refactor
pwittrock Jun 5, 2018
539308c
method to construct ObjectCache from informers
DirectXMan12 Jun 5, 2018
953965b
ammend me
pwittrock Jun 5, 2018
56a8778
Code review comments. Code is in broken state with TODOs
pwittrock Jun 5, 2018
be7d9ca
Fix tests
pwittrock Jun 5, 2018
8d54599
Ensure Controller has been injected before allowing Watch to be called.
pwittrock Jun 5, 2018
e98e57c
Fix non-namespaced field selector indexes.
DirectXMan12 Jun 5, 2018
46494e8
Inject the Client
pwittrock Jun 5, 2018
c8d2c90
Add integration test for EnqueueOwnerHandler
pwittrock Jun 5, 2018
d7b86e4
Make config look at the user's home directory before failing
pwittrock Jun 6, 2018
b46d1ae
Fix a bunch of todos
DirectXMan12 Jun 6, 2018
437d48f
Source integration test fix
pwittrock Jun 6, 2018
ec5086e
Address controller-platform TODO's
pwittrock Jun 6, 2018
7cd70ee
Field indexers
DirectXMan12 Jun 6, 2018
6490536
Fix main
pwittrock Jun 6, 2018
b2ed01e
controller-platform cleanup
pwittrock Jun 6, 2018
6b0bb41
controller-manager minor example updates
pwittrock Jun 6, 2018
1e02052
Example program for controller libraries.
pwittrock Jun 6, 2018
80cfb8e
Comment the controller-platform example.
pwittrock Jun 6, 2018
70ab3c9
client
DirectXMan12 Jun 6, 2018
08a971d
Add client to example
pwittrock Jun 6, 2018
eecd2ab
Initialize empty map in client.
pwittrock Jun 6, 2018
a36fd75
Support injecting both live and cache clients
pwittrock Jun 6, 2018
10202b7
Update controller interface
pwittrock Jun 7, 2018
54a73ee
Add FieldIndexers back
pwittrock Jun 7, 2018
643525b
Clean up injection so it can return errors. All objects should be fu…
pwittrock Jun 7, 2018
65a6b00
Minor code cleanup
pwittrock Jun 7, 2018
674f2d8
Make it so controllers can be created after the ControllerManager is …
pwittrock Jun 7, 2018
a8668a7
Update godocs
pwittrock Jun 7, 2018
1800846
Isolated content of pkg/
pwittrock Jun 7, 2018
404997d
Copying pkg/
pwittrock Jun 7, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
308 changes: 308 additions & 0 deletions pkg/client/cache.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,308 @@
package client

import (
"context"
"fmt"
"reflect"

"k8s.io/apimachinery/pkg/api/errors"
apimeta "k8s.io/apimachinery/pkg/api/meta"
"k8s.io/apimachinery/pkg/fields"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/selection"
"k8s.io/client-go/tools/cache"

"github.com/kubernetes-sigs/kubebuilder/pkg/informer"
logf "github.com/kubernetes-sigs/kubebuilder/pkg/log"
)

var log = logf.KBLog.WithName("object-cache")

// ObjectCache is a ReadInterface
var _ ReadInterface = &ObjectCache{}

type ObjectCache struct {
cachesByType map[reflect.Type]*SingleObjectCache
scheme *runtime.Scheme
}

func ObjectCacheFromInformers(informers map[schema.GroupVersionKind]cache.SharedIndexInformer, scheme *runtime.Scheme) *ObjectCache {
res := NewObjectCache(scheme)
res.AddInformers(informers)
return res
}

func (o *ObjectCache) AddInformers(informers map[schema.GroupVersionKind]cache.SharedIndexInformer) {
for gvk, informer := range informers {
o.AddInformer(gvk, informer)
}
}

func (o *ObjectCache) Call(gvk schema.GroupVersionKind, c cache.SharedIndexInformer) {
o.AddInformer(gvk, c)
}

func (o *ObjectCache) AddInformer(gvk schema.GroupVersionKind, c cache.SharedIndexInformer) {
obj, err := o.scheme.New(gvk)
if err != nil {
log.Error(err, "could not register informer in ObjectCache for GVK", "GroupVersionKind", gvk)
return
}
if _, found := o.CacheFor(obj); found {
return
}
o.RegisterCache(obj, gvk, c.GetIndexer())
}

func NewObjectCache(scheme *runtime.Scheme) *ObjectCache {
return &ObjectCache{
cachesByType: make(map[reflect.Type]*SingleObjectCache),
scheme: scheme,
}
}

func (c *ObjectCache) RegisterCache(obj runtime.Object, gvk schema.GroupVersionKind, store cache.Indexer) {
objType := reflect.TypeOf(obj)
c.cachesByType[objType] = &SingleObjectCache{
Indexer: store,
GroupVersionKind: gvk,
}
}

func (c *ObjectCache) CacheFor(obj runtime.Object) (*SingleObjectCache, bool) {
objType := reflect.TypeOf(obj)
cache, isKnown := c.cachesByType[objType]
return cache, isKnown
}

func (c *ObjectCache) Get(ctx context.Context, key ObjectKey, out runtime.Object) error {
cache, isKnown := c.CacheFor(out)
if !isKnown {
return fmt.Errorf("no cache for objects of type %T, must have asked for an watch/informer first", out)
}
return cache.Get(ctx, key, out)
}

func (c *ObjectCache) List(ctx context.Context, opts *ListOptions, out runtime.Object) error {
itemsPtr, err := apimeta.GetItemsPtr(out)
if err != nil {
return nil
}
// http://knowyourmeme.com/memes/this-is-fine
outType := reflect.Indirect(reflect.ValueOf(itemsPtr)).Type().Elem()
cache, isKnown := c.cachesByType[outType]
if !isKnown {
return fmt.Errorf("no cache for objects of type %T", out)
}
return cache.List(ctx, opts, out)
}

// SingleObjectCache is a ReadInterface
var _ ReadInterface = &SingleObjectCache{}

// SingleObjectCache is a ReadInterface that retrieves objects
// from a single local cache populated by a watch.
type SingleObjectCache struct {
// Indexer is the underlying indexer wrapped by this cache.
Indexer cache.Indexer
// GroupVersionKind is the group-version-kind of the resource.
GroupVersionKind schema.GroupVersionKind
}

func (c *SingleObjectCache) Get(_ context.Context, key ObjectKey, out runtime.Object) error {
storeKey := objectKeyToStoreKey(key)
obj, exists, err := c.Indexer.GetByKey(storeKey)
if err != nil {
return err
}
if !exists {
// Resource gets transformed into Kind in the error anyway, so this is fine
return errors.NewNotFound(schema.GroupResource{
Group: c.GroupVersionKind.Group,
Resource: c.GroupVersionKind.Kind,
}, key.Name)
}
if _, isObj := obj.(runtime.Object); !isObj {
return fmt.Errorf("cache contained %T, which is not an Object", obj)
}

// deep copy to avoid mutating cache
// TODO(directxman12): revisit the decision to always deepcopy
obj = obj.(runtime.Object).DeepCopyObject()

// TODO(directxman12): this is a terrible hack, pls fix
// (we should have deepcopyinto)
outVal := reflect.ValueOf(out)
objVal := reflect.ValueOf(obj)
if !objVal.Type().AssignableTo(outVal.Type()) {
return fmt.Errorf("cache had type %s, but %s was asked for", objVal.Type(), outVal.Type())
}
reflect.Indirect(outVal).Set(reflect.Indirect(objVal))
return nil
}

func (c *SingleObjectCache) List(ctx context.Context, opts *ListOptions, out runtime.Object) error {
var objs []interface{}
var err error

if opts != nil && opts.FieldSelector != nil {
// TODO(directxman12): support more complicated field selectors by
// combining multiple indicies, GetIndexers, etc
field, val, requiresExact := requiresExactMatch(opts.FieldSelector)
if !requiresExact {
return fmt.Errorf("non-exact field matches are not supported by the cache")
}
// list all objects by the field selector. If this is namespaced and we have one, ask for the
// namespaced index key. Otherwise, ask for the non-namespaced variant by using the fake "all namespaces"
// namespace.
objs, err = c.Indexer.ByIndex(fieldIndexName(field), keyToNamespacedKey(opts.Namespace, val))
} else if opts != nil && opts.Namespace != "" {
objs, err = c.Indexer.ByIndex(cache.NamespaceIndex, opts.Namespace)
} else {
objs = c.Indexer.List()
}
if err != nil {
return err
}
var labelSel labels.Selector
if opts != nil && opts.LabelSelector != nil {
labelSel = opts.LabelSelector
}

outItems := make([]runtime.Object, 0, len(objs))
for _, item := range objs {
obj, isObj := item.(runtime.Object)
if !isObj {
return fmt.Errorf("cache contained %T, which is not an Object", obj)
}
meta, err := apimeta.Accessor(obj)
if err != nil {
return err
}
if labelSel != nil {
lbls := labels.Set(meta.GetLabels())
if !labelSel.Matches(lbls) {
continue
}
}
outItems = append(outItems, obj.DeepCopyObject())
}
if err := apimeta.SetList(out, outItems); err != nil {
return err
}
return nil
}

// TODO: Make an interface with this function that has an Informers as an object on the struct
// that automatically calls InformerFor and passes in the Indexer into IndexByField

// noNamespaceNamespace is used as the "namespace" when we want to list across all namespaces
const allNamespacesNamespace = "__all_namespaces"

type InformerFieldIndexer struct {
Informers informer.Informers
}

func (i *InformerFieldIndexer) IndexField(obj runtime.Object, field string, extractValue IndexerFunc) error {
informer, err := i.Informers.InformerFor(obj)
if err != nil {
return err
}
return IndexByField(informer.GetIndexer(), field, extractValue)
}

// IndexByField adds an indexer to the underlying cache, using extraction function to get
// value(s) from the given field. This index can then be used by passing a field selector
// to List. For one-to-one compatibility with "normal" field selectors, only return one value.
// The values may be anything. They will automatically be prefixed with the namespace of the
// given object, if present. The objects passed are guaranteed to be objects of the correct type.
func IndexByField(indexer cache.Indexer, field string, extractor IndexerFunc) error {
indexFunc := func(objRaw interface{}) ([]string, error) {
// TODO(directxman12): check if this is the correct type?
obj, isObj := objRaw.(runtime.Object)
if !isObj {
return nil, fmt.Errorf("object of type %T is not an Object", objRaw)
}
meta, err := apimeta.Accessor(obj)
if err != nil {
return nil, err
}
ns := meta.GetNamespace()

rawVals := extractor(obj)
var vals []string
if ns == "" {
// if we're not doubling the keys for the namespaced case, just re-use what was returned to us
vals = rawVals
} else {
// if we need to add non-namespaced versions too, double the length
vals = make([]string, len(rawVals)*2)
}
for i, rawVal := range rawVals {
// save a namespaced variant, so that we can ask
// "what are all the object matching a given index *in a given namespace*"
vals[i] = keyToNamespacedKey(ns, rawVal)
if ns != "" {
// if we have a namespace, also inject a special index key for listing
// regardless of the object namespace
vals[i+len(rawVals)] = keyToNamespacedKey("", rawVal)
}
}

return vals, nil
}

if err := indexer.AddIndexers(cache.Indexers{fieldIndexName(field): indexFunc}); err != nil {
return err
}
return nil
}

// fieldIndexName constructs the name of the index over the given field,
// for use with an Indexer.
func fieldIndexName(field string) string {
return "field:" + field
}

// keyToNamespacedKey prefixes the given index key with a namespace
// for use in field selector indexes.
func keyToNamespacedKey(ns string, baseKey string) string {
if ns != "" {
return ns + "/" + baseKey
}
return allNamespacesNamespace + "/" + baseKey
}

// objectKeyToStorageKey converts an object key to store key.
// It's akin to MetaNamespaceKeyFunc. It's seperate from
// String to allow keeping the key format easily in sync with
// MetaNamespaceKeyFunc.
func objectKeyToStoreKey(k ObjectKey) string {
if k.Namespace == "" {
return k.Name
}
return k.Namespace + "/" + k.Name
}

// requiresExactMatch checks if the given field selector is of the form `k=v` or `k==v`.
func requiresExactMatch(sel fields.Selector) (field, val string, required bool) {
reqs := sel.Requirements()
if len(reqs) != 1 {
return "", "", false
}
req := reqs[0]
if req.Operator != selection.Equals && req.Operator != selection.DoubleEquals {
return "", "", false
}
return req.Field, req.Value, true
}

// SplitReaderWriter forms an interface Interface by composing separate
// read and write interfaces. This way, you can have an Interface that
// reads from a cache and writes to the API server.
type SplitReaderWriter struct {
ReadInterface
WriteInterface
}
Loading