Service Catalog walkthrough is broken for minikube

[nilebox]

After introducing RBAC, Service Catalog walkthrough doesn't work for minikube anymore:

Error: release catalog failed: clusterroles.rbac.authorization.k8s.io "servicecatalog.k8s.io:apiserver" is forbidden: attempt to grant extra privileges: [PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["get"]} PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["list"]} PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["watch"]}] user=&{system:serviceaccount:kube-system:default ff1057e8-721b-11e7-b7bb-a23c9ee63bc7 [system:serviceaccounts system:serviceaccounts:kube-system system:authenticated] map[]} ownerrules=[] ruleResolutionErrors=[]
while installing service catalog chart 

My understanding is that now there is a need for preconfiguring some roles in Kubernetes cluster before installing a Service Catalog helm chart.
It might be useful to add a script or separate documentation page describing this.

[nilebox]

@MHBauer can you help with this?
My current understanding is:

1. run kubectl create clusterrolebinding fixRBAC --clusterrole=cluster-admin --serviceaccount=kube-system:default
2. take a look at the other steps in local-up-cluster.sh under ENABLE_RBAC:true option
3. install Service Catalog chart

[nilebox]

Possibly related to kubernetes/minikube#1722 and kubernetes/minikube#1734

[MHBauer]

Strugled with minikube for a while. No result.

Minikube needs to be started with --extra-config=apiserver.Authorization.Mode=RBAC.

Does everything work with fixRBAC binding created or are there more problems?

[jpeeler]

I'm getting the same result with both local-up-cluster and minikube. After enabling RBAC and making the default service account cluster-admin, I see an authentication prompt when executing kubectl --context=service-catalog get brokers... I'm not sure what I could be doing wrong, but I haven't ruled out user error yet.

[kibbles-n-bytes]

@jpeeler In order to get rid of that, I had to set a user/password combo for that context. So the full setup would look like the following:

kubectl config set-credentials service-catalog-creds --username=admin --password=admin
kubectl config set-cluster service-catalog-cluster --server="https://${API_SERVER_HOST}:443" --insecure-skip-tls-verify=true
kubectl config set-context service-catalog --cluster=service-catalog-cluster --user=service-catalog-creds
kubectl config use-context service-catalog

Maybe @MHBauer you can comment on whether that is a true solution, and whether we should add that to the walkthrough and why it works?

[kibbles-n-bytes]

Other than that, though, I believe the issue here was addressed in the walkthrough additions in #1076 . Is there more to be done for this @nilebox or are we good to close it?

[MHBauer]

Okay, I think I was confused. That's with rbac, but without aggregation. Probably with auth. I'm not sure if there's a default that might be getting accidentally exposed here. I definitely didn't set admin:admin credentials for anything intentionally.

[pmorie]

@MHBauer do you think this is fixed, then?

…

On Tue, Aug 1, 2017 at 2:03 PM, Morgan Bauer ***@***.***> wrote: Okay, I think I was confused. That's with rbac, but without aggregation. Probably with auth. I'm not sure if there's a default that might be getting accidentally exposed here. I definitely didn't set admin:admin credentials for anything intentionally. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1069 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWXmN7NC7PPoWvnYmxmE609lE17aEueks5sT2hVgaJpZM4OkqVt> .

[MHBauer]

I need to check some things, but if the SA needs permissions because of the tiller pod for helm, then

 +kubectl create clusterrolebinding default-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default 

should really be a combination of

1. installing tiller a different way (with it's own serviceaccount)
2. giving that tiller SA permissions (similar to the ks:d SA)

It's up to @jpeeler and @nilebox to confirm or deny whether the combo of the existing "minikube extra rbac config" and "set up the SA permissions" is good enough, or if there need to be more credentials above and beyond.

[jpeeler]

I just tested with minikube again with RBAC enabled and I'm still getting prompted for credentials, but it works when adding them as described in #1069 (comment).

[MHBauer]

Okay, I didn't even think of the credentials, this sounds like a kubectl bug. As far as I know, we're not setting up credentials, and that user/pass has no effect other than getting you past a prompt.

[nilebox]

I can confirm that @kibbles-n-bytes's list of actions (preceded with fixRBAC actions mentioned by me above) worked for me.

whether we should add that to the walkthrough and why it works?

@kibbles-n-bytes Currently we don't have any guidance for minikube in our docs at all. Do we want to have one?

[kibbles-n-bytes]

@nilebox I meant more the credentials issue, which isn't unique to minikube. We have a section (here) that describes setting up the context, so I was thinking we could add the credentials setup there to get rid of the prompt until the issue is fixed.

However, I think also adding a mention of how to enable RBAC on minikube in the Troubleshooting section of the walkthrough would be good.

[arschles]

However, I think also adding a mention of how to enable RBAC on minikube in the Troubleshooting section of the walkthrough would be good.

@kibbles-n-bytes I am partially addressing this in #1163, and will continue to do so in follow-ups. cc/ @jboyd01 who is also doing work in this arena

[arschles]

I've just changed docs for installing on Kube 1.7 (see here) to include the minikube startup command (see here), and install works properly.

As such, I am closing this issue.

Note that there is now a problem running the API server & controller, which I'll open a new issue for.