Skip to content
This repository has been archived by the owner on Apr 4, 2023. It is now read-only.

Fix kube-dns RBAC issues #108

Merged
merged 2 commits into from
Nov 8, 2017
Merged

Fix kube-dns RBAC issues #108

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1884ae4
Fix kube-dns RBAC issues.
wallrj Nov 8, 2017
9641171
Add another expected error
wallrj Nov 8, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions hack/install-e2e-dependencies.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,12 @@ if ! retry TIMEOUT=300 kubectl get nodes; then
echo "ERROR: Timeout waiting for Minikube to be ready"
exit 1
fi

# Fix kube-dns RBAC issues.
# Allow kube-dns and other kube-system services full access to the API.
# See:
# * https://github.com/kubernetes/minikube/issues/1734
# * https://github.com/kubernetes/minikube/issues/1722
kubectl create clusterrolebinding cluster-admin:kube-system \
--clusterrole=cluster-admin \
--serviceaccount=kube-system:default
20 changes: 0 additions & 20 deletions hack/prepare-e2e.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,26 +11,6 @@ apiVersion: v1
kind: List
items:

### Fix kube-dns RBAC issues ###
# Create a ServiceAccount for kube-dns to use
- apiVersion: v1
kind: ServiceAccount
metadata:
name: kube-dns
namespace: kube-system
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: system:kube-dns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:kube-dns
subjects:
- kind: ServiceAccount
name: default
namespace: kube-system
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm - are you sure that this change actually fixes anything?

By the looks of what the rules previously were, I don't think it will. We previously granted kube-system:default the system:kube-dns role, which (I'd imagine) is sufficient for kube-dns to run properly.

The kube-dns service account that's been created above is actually not used from what I can see.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only difference between before and after, is that we've removed a service account that's unused ('kube-dns'), and granted kube-dns full access to the API instead of using the kube-dns ClusterRole (which in a default k8s cluster looks like the following):

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  creationTimestamp: 2017-10-23T15:15:15Z
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: system:kube-dns
  resourceVersion: "57"
  selfLink: /apis/rbac.authorization.k8s.io/v1beta1/clusterroles/system%3Akube-dns
  uid: f9194879-b804-11e7-a76b-42010a8401a1
rules:
- apiGroups:
  - ""
  resources:
  - endpoints
  - services
  verbs:
  - list
  - watch

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an extract from the logs before applying any RBAC fixes.

kubectl logs deployment/kube-dns --namespace kube-system kubedns
I1108 13:34:41.182340       1 dns.go:48] version: 1.14.4-2-g5584e04
I1108 13:34:41.183027       1 server.go:66] Using configuration read from ConfigMap: kube-system:kube-dns
I1108 13:34:41.183070       1 server.go:113] FLAG: --alsologtostderr="false"
I1108 13:34:41.183085       1 server.go:113] FLAG: --config-dir=""
I1108 13:34:41.183091       1 server.go:113] FLAG: --config-map="kube-dns"
I1108 13:34:41.183095       1 server.go:113] FLAG: --config-map-namespace="kube-system"
I1108 13:34:41.183098       1 server.go:113] FLAG: --config-period="10s"
I1108 13:34:41.183103       1 server.go:113] FLAG: --dns-bind-address="0.0.0.0"
I1108 13:34:41.183107       1 server.go:113] FLAG: --dns-port="10053"
I1108 13:34:41.183113       1 server.go:113] FLAG: --domain="cluster.local."
I1108 13:34:41.183124       1 server.go:113] FLAG: --federations=""
I1108 13:34:41.183129       1 server.go:113] FLAG: --healthz-port="8081"
I1108 13:34:41.183133       1 server.go:113] FLAG: --initial-sync-timeout="1m0s"
I1108 13:34:41.183136       1 server.go:113] FLAG: --kube-master-url=""
I1108 13:34:41.183147       1 server.go:113] FLAG: --kubecfg-file=""
I1108 13:34:41.183150       1 server.go:113] FLAG: --log-backtrace-at=":0"
I1108 13:34:41.183157       1 server.go:113] FLAG: --log-dir=""
I1108 13:34:41.183161       1 server.go:113] FLAG: --log-flush-frequency="5s"
I1108 13:34:41.183164       1 server.go:113] FLAG: --logtostderr="true"
I1108 13:34:41.183168       1 server.go:113] FLAG: --nameservers=""
I1108 13:34:41.183171       1 server.go:113] FLAG: --stderrthreshold="2"
I1108 13:34:41.183174       1 server.go:113] FLAG: --v="2"
I1108 13:34:41.183177       1 server.go:113] FLAG: --version="false"
I1108 13:34:41.183183       1 server.go:113] FLAG: --vmodule=""
I1108 13:34:41.183257       1 server.go:176] Starting SkyDNS server (0.0.0.0:10053)
I1108 13:34:41.183483       1 server.go:198] Skydns metrics enabled (/metrics:10055)
I1108 13:34:41.183491       1 dns.go:147] Starting endpointsController
I1108 13:34:41.183494       1 dns.go:150] Starting serviceController
I1108 13:34:41.183577       1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I1108 13:34:41.183588       1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
E1108 13:34:41.195392       1 sync_configmap.go:86] Error getting ConfigMap kube-system:kube-dns err: configmaps "kube-dns" is forbidden: User "system:serviceaccount:kube-system:default" cannot get configmaps in the namespace "kube-system"
E1108 13:34:41.195420       1 dns.go:183] Error getting initial ConfigMap: configmaps "kube-dns" is forbidden: User "system:serviceaccount:kube-system:default" cannot get configmaps in the namespace "kube-system", starting with default values
E1108 13:34:41.196072       1 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list services at the cluster scope
E1108 13:34:41.196147       1 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:kube-system:default" cannot list endpoints at the cluster scope
E1108 13:34:41.196762       1 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.ConfigMap: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"

It seemed to me that cannot list configmaps might be the problem and was missing from the original binding.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 cool, sounds about right then! I wonder why that wasn't in the Role already 🤔


### Tiller ###
# Create a ServiceAccount for tiller to use
- apiVersion: v1
Expand Down