Skip to content
This repository has been archived by the owner on May 6, 2022. It is now read-only.

RBAC setup behind the aggregator. #936

Merged
merged 2 commits into from
Jul 17, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions charts/catalog/templates/apiserver-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ spec:
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
spec:
serviceAccountName: "{{ .Values.apiserver.serviceAccount }}"
containers:
- name: apiserver
image: {{ .Values.apiserver.image }}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ spec:
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
spec:
serviceAccountName: "{{ .Values.controllerManager.serviceAccount }}"
containers:
- name: controller-manager
image: {{ .Values.controllerManager.image }}
Expand Down
141 changes: 141 additions & 0 deletions charts/catalog/templates/rbac.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
{{- if .Capabilities.APIVersions.Has "rbac.authorization.k8s.io/v1beta1" }}
apiVersion: v1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pmorie @arschles @luxas @liggitt should I make a helm chart and wrap this whole thing in it? Not sure if it's okay to unconditionally send rbac objects. Would helm fail?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MHBauer I believe we should be able to use the following to test whether the rbac API group is available:

{{- if .Capabilities.APIVersions.Has "rbac.k8s.io/v1beta1" }}
// stuff
{{- end }}

kind: List
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any specific reason to use List and not ---?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liggitt gave it to me like this. I dunno if there's any general preference or a behavioral difference between a list and a stream of objects.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it lets you tweak things live, then reexport with a kubectl get -o yaml of the objects you're interested in

items:

### API Server ###

# TODO: if this is just for namespace lifecycle admission, move to a generic role
# the role for the apiserver
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: "servicecatalog.k8s.io:apiserver"
# this rule defined on the role for specifically the
# namespace-lifecycle admission-controller
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get", "list", "watch"]
# API-server service-account gets its own role
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: "servicecatalog.k8s.io:apiserver"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: "servicecatalog.k8s.io:apiserver"
subjects:
- apiGroup: ""
kind: ServiceAccount
name: "{{ .Values.apiserver.serviceAccount }}"
namespace: "{{ .Release.Namespace }}"
# apiserver gets the auth-delegator role to delegate auth decisions to
# the core apiserver
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: "servicecatalog.k8s.io:apiserver-auth-delegator"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:auth-delegator
subjects:
- apiGroup: ""
kind: ServiceAccount
name: "{{ .Values.apiserver.serviceAccount }}"
namespace: "{{ .Release.Namespace }}"
# apiserver gets the ability to read authentication. This allows it to
# read the specific configmap that has the requestheader-* entries to
# enable api aggregation
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: "servicecatalog.k8s.io:apiserver-authentication-reader"
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: extension-apiserver-authentication-reader
subjects:
- apiGroup: ""
kind: ServiceAccount
name: "{{ .Values.apiserver.serviceAccount }}"
namespace: "{{ .Release.Namespace }}"

### Controller-Manager ###

# controller-manager role defines what access the service-catalog
# controller-manager needs to manage the resources of the
# service-catalog
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: "servicecatalog.k8s.io:controller-manager"
rules:
- apiGroups: [""]
resources: ["events"]
verbs: ["create","patch","update"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't have update here, just create/patch

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

match update and patch

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? It's essentially the same thing and should be coupled together or..?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's essentially the same thing and should be coupled together

Yes, if you grant patch, there's no reason not to include update as well

# TODO: do not grant global access, limit to particular secrets referenced from bindings
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get","create","delete"]
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get","list","watch"]
# access to our service-catalog types
- apiGroups: ["servicecatalog.k8s.io"]
resources: ["serviceclasses"]
verbs: ["get","list","watch","create","patch","update","delete"]
- apiGroups: ["servicecatalog.k8s.io"]
resources: ["bindings","brokers","instances"]
verbs: ["get","list","watch"]
- apiGroups: ["servicecatalog.k8s.io"]
resources: ["bindings/status","brokers/status","instances/status"]
verbs: ["update"]
# give the controller-manager service account access to whats defined in its role.
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: "servicecatalog.k8s.io:controller-manager"
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: "servicecatalog.k8s.io:controller-manager"
subjects:
- apiGroup: ""
kind: ServiceAccount
name: "{{ .Values.controllerManager.serviceAccount }}"
namespace: "{{ .Release.Namespace }}"

# This gives create/update access to an endpoint in kube-system for leader election
# TODO: use an object other than endpoints, and in the same namespace as the service catalog, not in kube-system
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: "servicecatalog.k8s.io::leader-locking-controller-manager"
namespace: kube-system
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["create"]
- apiGroups: [""]
resources: ["endpoints"]
resourceNames: ["service-catalog-controller-manager"]
verbs: ["get","update"]
- apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: service-catalog-controller-manager
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: service-catalog-controller-manager
subjects:
- apiGroup: ""
kind: ServiceAccount
name: "{{ .Values.controllerManager.serviceAccount }}"
namespace: "{{ .Release.Namespace }}"
{{ end }}
13 changes: 13 additions & 0 deletions charts/catalog/templates/serviceaccounts.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: v1
kind: List
items:
# The SA for the apiserver
- apiVersion: v1
kind: ServiceAccount
metadata:
name: "{{ .Values.apiserver.serviceAccount }}"
# The SA for the controller-manager
- apiVersion: v1
kind: ServiceAccount
metadata:
name: "{{ .Values.controllerManager.serviceAccount }}"
2 changes: 2 additions & 0 deletions charts/catalog/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ apiserver:
activated: false
# If specified, audit log goes to specified path.
logPath: "/tmp/service-catalog-apiserver-audit.log"
serviceAccount: service-catalog-apiserver
controllerManager:
# controller-manager image to use
image: quay.io/kubernetes-service-catalog/controller-manager:v0.0.13
Expand All @@ -97,4 +98,5 @@ controllerManager:
leaderElection:
# Whether the controller has option to set leader election namespace.
activated: false
serviceAccount: service-catalog-controller-manager
useAggregator: false
9 changes: 9 additions & 0 deletions contrib/jenkins/init_cluster.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
set -o nounset
set -o errexit

ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"

while [[ $# -ne 0 ]]; do
case "$1" in
--project) PROJECT="$2"; shift ;;
Expand Down Expand Up @@ -54,5 +56,12 @@ echo "Using cluster ${CLUSTERNAME}."
gcloud container clusters get-credentials "${CLUSTERNAME}" --project="${PROJECT}" --zone="${ZONE}" \
|| { echo 'Cannot get credentials for cluster.'; exit 1; }

# On GKE you need to give your user proper permissions in order to create new
# cluster roles. Needed for RBAC setup.
ACCOUNT_NAME="$(gcloud info | grep Account | sed 's/.*\[\(.*\)\]/\1/')"
kubectl create clusterrolebinding jenkins-cluster-admin-binding \
--clusterrole=cluster-admin --user="${ACCOUNT_NAME}" \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arschles @kibbles-n-bytes for default minikube I did
kubectl create clusterrolebinding minikube-cluster-admin-binding --clusterrole=cluster-admin --user=minikube

Copy link
Contributor Author

@MHBauer MHBauer Jul 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, did not help, same
Error: release catalog failed: clusterroles.rbac.authorization.k8s.io "servicecatalog.k8s.io:apiserver" is forbidden: attempt to grant extra privileges: [PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["get"]} PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["list"]} PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["watch"]}] user=&{system:serviceaccount:kube-system:default d6100657-632b-11e7-8c06-befc308e6091 [system:serviceaccounts system:serviceaccounts:kube-system system:authenticated] map[]} ownerrules=[] ruleResolutionErrors=[]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rbac wasn't on in minikube, oops.

add --extra-config=apiserver.Authorization.Mode=RBAC

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add this to the walkthrough docs in a follow-up

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracked in #1021

|| { echo 'Cannot not create cluster-admin role for service account.'; exit 1; }

helm init \
|| { echo 'Cannot initialize Helm.'; exit 1; }