Fix RBAC kube-dns issues

[fgimenez]

With the changes introduced in #70 for removing RBAC permissive-binding we were getting these errors on kube-dns:

E0503 07:32:42.846108       1 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:kube-system:default" cannot list endpoints at the cluster scope
E0503 07:32:42.846213       1 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list services at the cluster scope
E0503 07:32:42.846304       1 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.ConfigMap: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"

These changes allow kube-system services full access to the API and prevent the above failures, see:

• RBAC is broken kubernetes/minikube#1734
• Enable RBAC by default kubernetes/minikube#1722

[xh3b4sd]

Just for my interest, is this the reason why aws-operator CI fails all over the place?

[fgimenez]

I don't think so, the binary with the RBAC permissive-binding changes was in a pre-release state until properly tested, so it was not accessible in CI builds.

The issue resolved in this PR was about the name resolution of services (cert-operator could not access vault for instance, and the secrets would be consistently not created). I see in this aws-operator failed build https://circleci.com/gh/giantswarm/aws-operator/6064 that the problem is about the API not being up, so not related.

[xh3b4sd]

I see. Just found this in the logs.

TooManyBuckets: You have attempted to create more buckets than allowed

Will check back with Fernando.