Kubectl 1.8 broken for minikube - OpenAPI swagger file doesn't include `x-kubernetes-group-version-kind` #1996

apelisse · 2017-09-21T18:33:41Z

Environment:

Minikube version (use minikube version): v0.22.2

OS (e.g. from /etc/os-release): Linux
VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): VirtualBox
ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): v0.23.4
Install tools:
Others:

What happened:
Swagger doesn't have x-kubernetes-group-version-kind while it should.

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

minikube start
kubectl proxy &
curl 127.0.0.1:8001/swagger.json | less
# Look for v1.Service definition, it doesn't have `x-kubernetes-group-version-kind` field

Output of minikube logs (if applicable):

Anything else do we need to know:

@pwittrock @mbohlool

The text was updated successfully, but these errors were encountered:

r2d4 · 2017-09-25T17:23:54Z

localkube-swagger.txt
kubeadm-swagger.txt

@apelisse Can you upload the full swagger.json file or verify that the above outputs look similar to yours?

Here are the outputs from minikube start --bootstrapper localkube and minikube start --bootstrapper kubeadm. I did receive an error when I tried to use kubectl version 1.8.0-rc.1 on the 1.7.5 localkube cluster, but not on the kubeadm based cluster.

The localkube swagger.json does seem to include the x-kubernetes-group-version-kind headers, but there are some differences which I'm working through identifying.

mbohlool · 2017-09-25T17:49:51Z

@r2d4 Before we go into debuging this (and I would like to help debugging it), can you confirm if we found a fix here, we can patch minikube 1.7 images with it?

r2d4 · 2017-09-25T18:02:04Z

@mbohlool Well, minikube ships with a default version of a localkube, which we won't be able to patch. We would be able to patch non-default versions, since those are fetched from GCS.

The latest version includes a --bootstrapper kubeadm flag which would provide a workaround without relying on localkube, but it isn't the default option.

apelisse · 2017-09-25T18:19:46Z

localkube-swagger.txt

This one is not good

kubeadm-swagger.txt

This one is fine, as far as I can tell

r2d4 · 2017-09-25T18:23:47Z

@apelisse I've verified that, but what exactly is missing? The headers are there, but I'm not sure which ones are missing - they are present for things like service

     "x-kubernetes-action": "list",
     "x-kubernetes-group-version-kind": {
      "group": "",
      "version": "v1",
      "kind": "Service"
     }

apelisse · 2017-09-25T18:25:12Z

Yeah, you're looking at the "paths", not the "definitions" section

apelisse · 2017-09-25T18:43:34Z

IOW,

With kubeadm:

$ jq '.definitions."io.k8s.kubernetes.pkg.api.v1.Pod"."x-kubernetes-group-version-kind"' kubeadm-swagger.txt
[
 {
   "group": "",
   "version": "v1",
   "kind": "Pod"
 }
]

With localkube:

$ jq '.definitions."io.k8s.kubernetes.pkg.api.v1.Pod"."x-kubernetes-group-version-kind"' localkube-swagger.txt
null

apelisse · 2017-09-25T18:44:05Z

We had a similar type of issue in apiserver-builder, where x-kubernetes-group-version-kind was missing: kubernetes-sigs/apiserver-builder-alpha#149

mbohlool · 2017-09-26T00:04:50Z

Any update on this? Let me know if you guys need any help.

mbohlool · 2017-09-27T19:23:08Z

Found the problem after a long debugging :) Short Answer: This is the fix: kubernetes/kubernetes#53152 and you need to update your vendor folder after this gets into 1.7
You should not have any problem with 1.8 as this fix is already there.

Longer Answer: We do use full package name as the name of the OpenAPI definitions. The OpenAPI generator take the vendoring folder into account but (in 1.7 code) the extension in staging/src/k8s.io/apiserver/pkg/endpoints/openapi/openapi.go did not. We were OK in kubernetes at the time, but when we moved packages out and vendor some part of APIs in, that became a problem and we fixed it in 1.8. All you need to do is to backport this fix (that seems safe and should not affect kubernetes itself at all) to 1.7 branch and update your vendor folder.

@wojtek-t

Automatic merge from submit-queue. Manually cherry-pick 4379bbd as it is part of a larger PR The commit 4379bbd fixes a bug in openapi spec that resulted in some of the type missing group-version-kind extension. That is specificly serious for the projects that vendor kubernetes in (like minikube). It results in them not having GVKs at all (kubernetes/minikube#1996). ```release-note Bugfix: OpenAPI models may not get group-version-kind extension if kubernetes is vendored in another project (e.g. minikube). Kubectl 1.8 needs this extension to work with those projects. ``` @wojtek-t

@wojtek-t

Automatic merge from submit-queue. Manually cherry-pick 4379bbd as it is part of a larger PR The commit 4379bbd fixes a bug in openapi spec that resulted in some of the type missing group-version-kind extension. That is specificly serious for the projects that vendor kubernetes in (like minikube). It results in them not having GVKs at all (kubernetes/minikube#1996). ```release-note Bugfix: OpenAPI models may not get group-version-kind extension if kubernetes is vendored in another project (e.g. minikube). Kubectl 1.8 needs this extension to work with those projects. ``` @wojtek-t Kubernetes-commit: c33e2b9d5e7df8464a7464496526ee272ad44ff4

apelisse · 2017-10-04T15:46:29Z

It didn't take time for someone to notice: kubernetes/kubectl#67.
is there a new release of minikube? What are people supposed to do if they hit that bug? Thanks

r2d4 · 2017-10-04T15:51:41Z

We're working on cherry-picking the fix and backporting it to our localkube releases. We'll make sure that this is fixed in the next version, but for now the only solution is to not upgrade your kubectl to version 1.8.

apelisse · 2017-10-04T15:53:58Z

for now the only solution is to not upgrade your kubectl to version 1.8.

Or use --validate=false,
Or use --openapi-validation=false.

Thank you Matt

r2d4 · 2017-10-04T18:22:21Z

I've backported the fix to previous versions of localkube, and it should be fixed for the default version in the next release

@wojtek-t

Automatic merge from submit-queue. Manually cherry-pick 4379bbd as it is part of a larger PR The commit 4379bbd fixes a bug in openapi spec that resulted in some of the type missing group-version-kind extension. That is specificly serious for the projects that vendor kubernetes in (like minikube). It results in them not having GVKs at all (kubernetes/minikube#1996). ```release-note Bugfix: OpenAPI models may not get group-version-kind extension if kubernetes is vendored in another project (e.g. minikube). Kubectl 1.8 needs this extension to work with those projects. ``` @wojtek-t Kubernetes-commit: c33e2b9d5e7df8464a7464496526ee272ad44ff4

@wojtek-t

Automatic merge from submit-queue. Manually cherry-pick 4379bbd as it is part of a larger PR The commit 4379bbd fixes a bug in openapi spec that resulted in some of the type missing group-version-kind extension. That is specificly serious for the projects that vendor kubernetes in (like minikube). It results in them not having GVKs at all (kubernetes/minikube#1996). ```release-note Bugfix: OpenAPI models may not get group-version-kind extension if kubernetes is vendored in another project (e.g. minikube). Kubectl 1.8 needs this extension to work with those projects. ``` @wojtek-t Kubernetes-commit: c33e2b9d5e7df8464a7464496526ee272ad44ff4

sauravomar · 2017-10-23T10:09:16Z

Is it resolved in the latest release?

r2d4 · 2017-10-23T16:57:05Z

Yes @sauravomar

@wojtek-t

Automatic merge from submit-queue. Manually cherry-pick 4379bbd as it is part of a larger PR The commit 4379bbd fixes a bug in openapi spec that resulted in some of the type missing group-version-kind extension. That is specificly serious for the projects that vendor kubernetes in (like minikube). It results in them not having GVKs at all (kubernetes/minikube#1996). ```release-note Bugfix: OpenAPI models may not get group-version-kind extension if kubernetes is vendored in another project (e.g. minikube). Kubectl 1.8 needs this extension to work with those projects. ``` @wojtek-t Kubernetes-commit: c33e2b9d5e7df8464a7464496526ee272ad44ff4

@wojtek-t

Automatic merge from submit-queue. Manually cherry-pick 4379bbd as it is part of a larger PR The commit 4379bbd fixes a bug in openapi spec that resulted in some of the type missing group-version-kind extension. That is specificly serious for the projects that vendor kubernetes in (like minikube). It results in them not having GVKs at all (kubernetes/minikube#1996). ```release-note Bugfix: OpenAPI models may not get group-version-kind extension if kubernetes is vendored in another project (e.g. minikube). Kubectl 1.8 needs this extension to work with those projects. ``` @wojtek-t Kubernetes-commit: c33e2b9d5e7df8464a7464496526ee272ad44ff4

apelisse mentioned this issue Sep 25, 2017

Revert "Change default validation to openapi" kubernetes/kubernetes#52997

Closed

pwittrock changed the title ~~OpenAPI swagger file doesn't include x-kubernetes-group-version-kind~~ Kubectl 1.8 broken for minikube - OpenAPI swagger file doesn't include x-kubernetes-group-version-kind Sep 25, 2017

This was referenced Sep 26, 2017

v1.8.0 known issues / FAQ accumulator kubernetes/kubernetes#53004

Closed

Manually cherry-pick 4379bbd as it is part of a larger PR kubernetes/kubernetes#53152

Merged

r2d4 added the kind/bug Categorizes issue or PR as related to a bug. label Sep 28, 2017

muriithi mentioned this issue Sep 29, 2017

Minikube fails for Kubernetes v1.8.0 using kubeadm bootstrapper #2018

Closed

apelisse mentioned this issue Oct 2, 2017

Error when trying to run kubectl create command on yaml files kubernetes/kubectl#67

Closed

r2d4 mentioned this issue Oct 4, 2017

Add patch for openapi #2031

Merged

r2d4 closed this as completed in #2031 Oct 4, 2017

prateekpandey14 mentioned this issue Oct 5, 2017

"kubectl apply -f openebs-storageclasses.yaml" showing error while trying to add openebs related storage classes openebs/openebs#533

Closed

This was referenced Oct 6, 2017

kubectl 1.8 cannot validate simple namespace yml kubernetes/kubernetes#53309

Closed

Kubectl 1.8 does not work with minikube kubernetes/kubernetes#53552

Closed

AlainODea mentioned this issue Oct 9, 2017

Issue with k8s.io/docs/concepts/workloads/controllers/deployment/ kubernetes/website#5818

Closed

2 tasks

ahammond mentioned this issue Oct 15, 2017

Beginners guide should specify k8s version openebs/openebs#633

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kubectl 1.8 broken for minikube - OpenAPI swagger file doesn't include `x-kubernetes-group-version-kind` #1996

Kubectl 1.8 broken for minikube - OpenAPI swagger file doesn't include `x-kubernetes-group-version-kind` #1996

apelisse commented Sep 21, 2017 •

edited

Loading

r2d4 commented Sep 25, 2017

mbohlool commented Sep 25, 2017

r2d4 commented Sep 25, 2017

apelisse commented Sep 25, 2017

r2d4 commented Sep 25, 2017

apelisse commented Sep 25, 2017

apelisse commented Sep 25, 2017

apelisse commented Sep 25, 2017

mbohlool commented Sep 26, 2017

mbohlool commented Sep 27, 2017

apelisse commented Oct 4, 2017

r2d4 commented Oct 4, 2017

apelisse commented Oct 4, 2017 •

edited

Loading

r2d4 commented Oct 4, 2017

sauravomar commented Oct 23, 2017

r2d4 commented Oct 23, 2017

Kubectl 1.8 broken for minikube - OpenAPI swagger file doesn't include x-kubernetes-group-version-kind #1996

Kubectl 1.8 broken for minikube - OpenAPI swagger file doesn't include x-kubernetes-group-version-kind #1996

Comments

apelisse commented Sep 21, 2017 • edited Loading

r2d4 commented Sep 25, 2017

mbohlool commented Sep 25, 2017

r2d4 commented Sep 25, 2017

apelisse commented Sep 25, 2017

r2d4 commented Sep 25, 2017

apelisse commented Sep 25, 2017

apelisse commented Sep 25, 2017

apelisse commented Sep 25, 2017

mbohlool commented Sep 26, 2017

mbohlool commented Sep 27, 2017

apelisse commented Oct 4, 2017

r2d4 commented Oct 4, 2017

apelisse commented Oct 4, 2017 • edited Loading

r2d4 commented Oct 4, 2017

sauravomar commented Oct 23, 2017

r2d4 commented Oct 23, 2017

Kubectl 1.8 broken for minikube - OpenAPI swagger file doesn't include `x-kubernetes-group-version-kind` #1996

Kubectl 1.8 broken for minikube - OpenAPI swagger file doesn't include `x-kubernetes-group-version-kind` #1996

apelisse commented Sep 21, 2017 •

edited

Loading

apelisse commented Oct 4, 2017 •

edited

Loading