Cleaning up the setup instructions for Kubernetes 1.7 #1005

arschles · 2017-07-03T22:01:52Z

This is a multi-part PR:

Moves the aggregated API TLS setup script into the charts/catalog directory
Fetches TLS file contents directly from the helm chart to remove the dependency on base64
- Fixes The Helm Install docs in the API Aggregation Setup don't work on a Mac #1004
Adds --authentication-skip-lookup=true to work around the lack of the requestheader-client-ca key in the extension-apiserver-authentication ConfigMap in the kube-system namespace
Adds a walkthrough doc for Kube 1.7

MHBauer

haven't tested it yet. have some questions.

MHBauer · 2017-07-03T22:39:33Z

.gitignore

@@ -20,3 +20,11 @@ contrib/build/*/tmp/*
 .pkg
 .kube
 .var
+charts/catalog/apiserver-key.pem
+charts/catalog/apiserver.csr
+charts/catalog/apiserver.pem


can we put these in a subdir to isolate them?

I'm not sure, I know this works at the top level chart directory though, and I'd rather make progress this way for now

MHBauer · 2017-07-03T22:41:32Z

charts/catalog/templates/apiserver-cert-secret.yaml

-  tls.crt: {{ .Values.apiserver.tls.cert }}
-  tls.key: {{ .Values.apiserver.tls.key }}
+  tls.crt: {{ (.Files.Get .Values.apiserver.tls.certFileName) | b64enc }}
+  tls.key: {{ (.Files.Get .Values.apiserver.tls.keyFileName) | b64enc }}


is b64enc a helm thing?

yes, it uses sprig for template functions. relevant docs: https://masterminds.github.io/sprig/encoding.html

MHBauer · 2017-07-03T22:45:41Z

charts/catalog/templates/apiregistration.yaml

@@ -13,6 +13,10 @@ spec:
  service:
    namespace: {{ .Release.Namespace }}
    name: {{ template "fullname" . }}-apiserver
-  caBundle: {{ .Values.apiserver.tls.ca }}
+  caBundle: {{ (.Files.Get .Values.apiserver.tls.caFileName) | b64enc }}
  priority: 100


I think this needs to be in the else of the below stuff

It doesn't seem to exist on the object in the beta version.

https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/kube-aggregator/pkg/apis/apiregistration/v1beta1/types.go#L39-L72

There's a note too
https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/kube-aggregator/pkg/apis/apiregistration/v1beta1/types.go#L71

@MHBauer seems like it would need to be under an if for v1alpha1.

that's fine too.

MHBauer · 2017-07-03T22:48:29Z

charts/catalog/values.yaml

+    keyFileName:
+    # The name of the file that contains the CA to authenticate connections from API server
+    # proxies. This field is not required. If it is not set, the service-catalog API server 
+    # will be started with the --authentication-skip-lookup flag, which will disable the auth checks


I'm not sure setting CA should tie to setting this other flag.

they seem to be one or the other

Agree with @MHBauer

MHBauer · 2017-07-03T22:50:34Z

docs/walkthrough-1.7.md

@@ -0,0 +1,489 @@
+# Service Catalog Demonstration Walkthrough for Kubernetes v1.7.0 and Above


I know we wanted a separate walkthrough, but I figured it would be for the older versions. Is it expected to look at this one or the existing one when first starting?

That would depend on the target Kubernetes cluster. We need to do a better job in directing users to the right doc in the README, but I'd like to do that in a follow-up

I'm fine with sorting out directing users in a follow-up.

MHBauer · 2017-07-03T22:51:34Z

docs/api-aggregation-setup.md

+
+```console
+helm delete --purge catalog
+k delete apiservice v1alpha1.servicecatalog.k8s.io


not sure we can expect everyone to have kubectl aliased to k. Can we explicitly right out kubectl?

luxas

Quick pass. I really think we should proceed with #936 and #981 first, and then this will be easier. The longer PRs just sit and wait, the more skew like this will happen.

luxas · 2017-07-06T09:11:46Z

docs/walkthrough-1.7.md

+instructions here for enabling cluster DNS for all Kubernetes cluster
+installations, but here are a few notes:
+
+* If you are using Google Container Engine or minikube, you likely have cluster


luxas · 2017-07-06T09:12:15Z

docs/walkthrough-1.7.md

+server is pluggable, and we currently support the following implementations:
+
+1. Etcd 3
+2. Third Party Resources (also, known as TPRs) - this is an _alpha_ feature right now. It has known issues


will be migrated to CRD + link to issue

luxas · 2017-07-06T09:13:29Z

docs/walkthrough-1.7.md

+
+For example, on a mac,
+```console
+curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.7.0/bin/darwin/amd64/kubectl


dl.k8s.io points to storage.googleapis.com/kubernetes-release, can use that instead...

luxas · 2017-07-06T09:14:34Z

docs/walkthrough-1.7.md

+Create the new `Broker` resource with the following command:
+
+```console
+kubectl create -f contrib/examples/walkthrough/ups-broker.yaml


Soon this will be in the helm chart: #980

luxas · 2017-07-06T09:18:57Z

charts/catalog/templates/apiserver-deployment.yaml

@@ -59,6 +59,8 @@ spec:
        - "{{ .Values.apiserver.verbosity }}"
        {{- if .Values.apiserver.tls.requestHeaderCA }}
        - --requestheader-client-ca-file=/var/run/kubernetes-service-catalog/requestheader-ca.crt
+        {{ else }}
+        - --authentication-skip-lookup=true


Hmm, I think it's possible to use delegated authn/authz without passing the requestheader-ca when the correct RBAC bindings are in place. See #936 and #981

luxas · 2017-07-06T09:20:04Z

charts/catalog/values.yaml

+    # This field is required if apiserver.auth.enabled is true.
+    caFileName:
+    # The name of the file that contains the x509 certificate for use in the aforementioned
+    # APIService object. This field is required if apiserver.auth.enabled is true.


I don't think so. RBAC and delegated authn/authz can happen while still serving with self-signed certs

luxas · 2017-07-06T09:20:35Z

charts/catalog/values.yaml

+    keyFileName:
+    # The name of the file that contains the CA to authenticate connections from API server
+    # proxies. This field is not required. If it is not set, the service-catalog API server 
+    # will be started with the --authentication-skip-lookup flag, which will disable the auth checks


Agree with @MHBauer

luxas · 2017-07-06T09:21:08Z

docs/api-aggregation-setup.md

-        --set apiserver.tls.ca=$(base64 --wrap 0 ${SC_SERVING_CA}) \
-        --set apiserver.tls.cert=$(base64 --wrap 0 ${SC_SERVING_CERT}) \
-        --set apiserver.tls.key=$(base64 --wrap 0 ${SC_SERVING_KEY})
+    --set useAggregator=true \


I think we should just switch this to true by default, see #981

I think it would be fine for #981 to do that.

luxas · 2017-07-06T09:21:17Z

docs/api-aggregation-setup.md

@@ -179,12 +178,14 @@ keys we just generated inline.

 ```
 helm install charts/catalog \
-    --name ${HELM_NAME} --namespace ${SVCCAT_NAMESPACE} \
+    --name ${HELM_NAME} \
+    --namespace ${SVCCAT_NAMESPACE} \
    --set apiserver.auth.enabled=true \


I think we should just switch this to true by default, see #981

luxas · 2017-07-06T09:21:23Z

docs/api-aggregation-setup.md

-        --set apiserver.tls.cert=$(base64 --wrap 0 ${SC_SERVING_CERT}) \
-        --set apiserver.tls.key=$(base64 --wrap 0 ${SC_SERVING_KEY})
+    --set useAggregator=true \
+    --set apiserver.insecure=false \


I think we should just switch this to false by default, see #981

MHBauer · 2017-07-07T17:34:55Z

charts/catalog/templates/apiregistration.yaml

+  {{ else if .Capabilities.APIVersions.Has "apiregistration.k8s.io/v1beta1" -}}
+  groupPriorityMinimum: 10000
+  versionPriority: 20
+  {{- end }}


@arschles can we pull this out into a separate PR (without the filesget -> base64 change) ? I think it's worth having while we figure out the details on everything else.

@MHBauer I will not be able to split this out today. Please feel free to pull this into another PR, and I'll rebase as necessary.

thanks @MHBauer

Particularly, making it easier and more straightforward to install service-catalog for use behind the aggregator

arschles · 2017-07-11T16:02:46Z

@luxas @MHBauer There are tweaks requested and also a request to wait until #936 and #981 are in. I'd like to get this PR in now, and address the tweaks and follow-ups in those PRs.

luxas · 2017-07-11T17:04:43Z

I'd like the tweaks/comments to be adressed and see the CI passing, but other than that it's fine by me to merge if follow-ups to enhance this are promised

pmorie

LGTM

pmorie · 2017-07-11T18:22:12Z

docs/api-aggregation-setup.md

-        --set apiserver.tls.ca=$(base64 --wrap 0 ${SC_SERVING_CA}) \
-        --set apiserver.tls.cert=$(base64 --wrap 0 ${SC_SERVING_CERT}) \
-        --set apiserver.tls.key=$(base64 --wrap 0 ${SC_SERVING_KEY})
+    --set useAggregator=true \


I think it would be fine for #981 to do that.

pmorie · 2017-07-11T18:27:09Z

docs/walkthrough-1.7.md

@@ -0,0 +1,489 @@
+# Service Catalog Demonstration Walkthrough for Kubernetes v1.7.0 and Above


I'm fine with sorting out directing users in a follow-up.

pmorie · 2017-07-13T05:40:14Z

@arschles jenkins looks like a legit error:

``` 1. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 5 seconds. 2. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 7 seconds. 3. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 11 seconds. 4. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 16 seconds. 5. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 25 seconds. 6. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 37 seconds. 7. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 56 seconds. 8. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 60 seconds. 9. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command failed. Will retry in 60 seconds. 10. Executing helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer Error: render error in "catalog/templates/apiserver-cert-secret.yaml": template: catalog/templates/apiserver-cert-secret.yaml:12:33: executing "catalog/templates/apiserver-cert-secret.yaml" at <.Values.apiserver.tl...>: wrong type for value; expected string; got interface {} Command 'helm install /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/charts/catalog --name catalog --namespace catalog --set debug=true,insecure=true,controllerManager.image=gcr.io/service-catalog-jenkins-jobs/catalog/controller-manager:01cdcb1,apiserver.image=gcr.io/service-catalog-jenkins-jobs/catalog/apiserver:01cdcb1,apiserver.service.type=LoadBalancer' failed 10 times, aborting. /var/lib/jenkins/workspace/service-catalog-PR-testing2/src/github.com/kubernetes-incubator/service-catalog/contrib/hack/test_walkthrough.sh: line 147: Error deploying service catalog to cluster. (exit 1) ```

arschles · 2017-08-28T21:31:26Z

Closing in favor of #1163

arschles self-assigned this Jul 3, 2017

arschles requested review from DirectXMan12, pmorie, krancour, duglin and MHBauer July 3, 2017 22:01

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jul 3, 2017

arschles mentioned this pull request Jul 3, 2017

The Helm Install docs in the API Aggregation Setup don't work on a Mac #1004

Closed

MHBauer suggested changes Jul 3, 2017

View reviewed changes

luxas reviewed Jul 6, 2017

View reviewed changes

MHBauer reviewed Jul 7, 2017

View reviewed changes

arschles force-pushed the agg-base64 branch from 4764b52 to 73ca989 Compare July 11, 2017 15:58

Cleaning up the setup instructions for Kubernetes 1.7

303ee4d

Particularly, making it easier and more straightforward to install service-catalog for use behind the aggregator

arschles force-pushed the agg-base64 branch from 73ca989 to 303ee4d Compare July 11, 2017 15:59

adding errexit

01cdcb1

pmorie reviewed Jul 11, 2017

View reviewed changes

pmorie added this to the 0.0.13 milestone Jul 11, 2017

pmorie added the LGTM1 label Jul 11, 2017

duglin modified the milestones: 0.0.13, 0.0.14 Jul 13, 2017

duglin modified the milestones: 0.0.14, 0.0.15 Jul 27, 2017

duglin modified the milestones: 0.0.15, 0.0.16 Aug 10, 2017

duglin modified the milestones: 0.0.16, 0.0.17 Aug 17, 2017

kibbles-n-bytes added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 17, 2017

arschles mentioned this pull request Aug 28, 2017

Splitting up the Walkthrough for 1.6 and 1.7 instructions #1163

Merged

arschles closed this Aug 28, 2017

arschles deleted the agg-base64 branch August 28, 2017 21:31

		@@ -0,0 +1,489 @@
		# Service Catalog Demonstration Walkthrough for Kubernetes v1.7.0 and Above

Cleaning up the setup instructions for Kubernetes 1.7 #1005

Cleaning up the setup instructions for Kubernetes 1.7 #1005

Conversation

arschles commented Jul 3, 2017

MHBauer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luxas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arschles commented Jul 11, 2017

luxas commented Jul 11, 2017

pmorie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmorie commented Jul 13, 2017

arschles commented Aug 28, 2017