controller-gen doesn't work with k8s.io/apiextensions-apiserver #291

jiachengxu · 2019-07-31T09:15:12Z

Hi folks,

I use controller-gen integrated with Kubebuilder to generate RBAC, webhook and CRDs for my project, there is a command called make manifests to run:
controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
to generate everything, but it doesn't work for me since I have k8s.io/apiextensions-apiserver in my go.mod as a dependency, the following error messages are poping out:
/Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:63:2: enountered struct field "Raw" without JSON tag in type "JSON" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:29:24: unsupported type "float64" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:31:24: unsupported type "float64" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:39:24: unsupported type "float64" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:85:2: enountered struct field "Schema" without JSON tag in type "JSONSchemaPropsOrArray" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:86:2: enountered struct field "JSONSchemas" without JSON tag in type "JSONSchemaPropsOrArray" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:105:2: enountered struct field "Allows" without JSON tag in type "JSONSchemaPropsOrBool" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:106:2: enountered struct field "Schema" without JSON tag in type "JSONSchemaPropsOrBool" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:127:2: enountered struct field "Schema" without JSON tag in type "JSONSchemaPropsOrStringArray" /Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:128:2: enountered struct field "Property" without JSON tag in type "JSONSchemaPropsOrStringArray" Error: not all generators ran successfully
Any ideas about why this happened?

Thanks,
Jiacheng

The text was updated successfully, but these errors were encountered:

munnerz · 2019-08-01T12:08:39Z

Can you share your types.go file containing your API definitions? It looks like you are depending on something in the apiextensions-apiserver API group 🤔 fwiw, #277 is a fix for:

/Users/maxwell/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.0.0-20190409022649-727a075fdec8/pkg/apis/apiextensions/v1beta1/types_jsonschema.go:63:2: enountered struct field "Raw" without JSON tag in type "JSON"

jiachengxu · 2019-08-01T13:26:20Z

@munnerz , yes, you are right, my API definitions depend on the apiextensions-apiserver API group, and thanks for the info :)!

munnerz · 2019-08-01T13:33:46Z

It looks like similar PRs for the JSONSchemaPropsOrArray, JSONSchemaPropsOrBool and JSONSchemaPropsOrStringArray need to be made in order to make controller-tools work for you too!

jiachengxu · 2019-08-01T13:45:29Z

Hi @munnerz, yes, you are right, would you mind me working on this? I think your PR is a good example for me to start and I can send the similar PRs for JSONSchemaPropsOrArray, JSONSchemaPropsOrBool and JSONSchemaPropsOrStringArray.

DirectXMan12 · 2019-09-30T21:40:04Z

I'm curious as to why your types depend on apiextensions. Are you embedding a CRD spec in your types? Just want to make sure we're solving the right problems here

maxsmythe · 2019-11-12T23:28:06Z

Gatekeeper is embedding a CRD spec into its ConstraintTemplate resource:

https://github.com/open-policy-agent/frameworks/blob/1307ba72bce38ee3cf44f94def1bbc41eb4ffa90/constraint/pkg/apis/templates/v1beta1/constrainttemplate_types.go#L46

thetechnick · 2019-11-14T12:51:53Z

@jiachengxu and I are also embedding the CRD Spec into one of our resources.
We are working on tooling that manages CRDs dynamically.

DirectXMan12 · 2019-11-15T17:21:00Z

We might want to better generically detect when something implements the JSON interfaces manually -- we have the typechecking info, so it should be possilbe.

/help

k8s-ci-robot · 2019-11-15T17:21:01Z

@DirectXMan12:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

We might want to better generically detect when something implements the JSON interfaces manually -- we have the typechecking info, so it should be possilbe.

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

fejta-bot · 2020-02-13T18:29:09Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

maxsmythe · 2020-02-13T21:24:01Z

/remove-lifecycle stale

muvaf · 2020-04-08T14:42:34Z

I have a type that embeds CustomResourceDefinitionSpec type which suffers from this issue.

We might want to better generically detect when something implements the JSON interfaces manually -- we have the typechecking info, so it should be possilbe.

@DirectXMan12 If you can give me some pointers, I can work on a PR to make it happen.

DirectXMan12 · 2020-04-20T22:39:13Z

I think someone posted something recently: #427

just waiting on a test

maxsmythe · 2020-04-20T23:03:57Z

Great news! I look forward to ditching the old, old version of controller-gen

fejta-bot · 2020-07-19T23:41:36Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

maxsmythe · 2020-07-20T17:51:35Z

/remove-lifecycle stale

How is this looking? I notice #427 is merged, but this isn't resolved. Is this just an oversight?

DirectXMan12 · 2020-09-09T23:25:57Z

I think this is mostly resolved. Technically, we should have type info for the other types, but we at least have a workaround, so I think we can probably close this for now.

tshak · 2020-11-19T19:45:32Z

I'm able to repro this issue on v0.4.1. It does not appear to be resolved. 🙏

julianKatz · 2020-11-20T22:48:37Z

@tshak did you use the following CRD option?

controller-gen crd:allowDangerousTypes=true

That got it going for me. I only figured that out by looking at the PR. Further documentation would make this a lot friendlier.

tshak · 2020-11-21T10:15:39Z

Thanks Julian. Sadly that does not work for me:

/Users/tshak/code/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.18.12/pkg/apis/apiextensions/v1/types_jsonschema.go:191:2: encountered struct field "Schema" without JSON tag in type "JSONSchemaPropsOrArray"
/Users/tshak/code/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.18.12/pkg/apis/apiextensions/v1/types_jsonschema.go:192:2: encountered struct field "JSONSchemas" without JSON tag in type "JSONSchemaPropsOrArray"
/Users/tshak/code/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.18.12/pkg/apis/apiextensions/v1/types_jsonschema.go:211:2: encountered struct field "Allows" without JSON tag in type "JSONSchemaPropsOrBool"
/Users/tshak/code/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.18.12/pkg/apis/apiextensions/v1/types_jsonschema.go:212:2: encountered struct field "Schema" without JSON tag in type "JSONSchemaPropsOrBool"
/Users/tshak/code/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.18.12/pkg/apis/apiextensions/v1/types_jsonschema.go:233:2: encountered struct field "Schema" without JSON tag in type "JSONSchemaPropsOrStringArray"
/Users/tshak/code/go/pkg/mod/k8s.io/apiextensions-apiserver@v0.18.12/pkg/apis/apiextensions/v1/types_jsonschema.go:234:2: encountered struct field "Property" without JSON tag in type "JSONSchemaPropsOrStringArray"

I'm running controller-gen as part of go code generation:

//go:generate go run -tags generate sigs.k8s.io/controller-tools/cmd/controller-gen object:headerFile=../../hack/boilerplate.go.txt paths=./... crd:trivialVersions=true,crdVersions=v1,allowDangerousTypes=true output:artifacts:config=../../config/base

maxsmythe · 2020-11-23T23:22:05Z

Is the following code being vendored in?

https://github.com/kubernetes/apiextensions-apiserver/blob/a7ee1efe41fc19623ce57f047c7595d624df4c17/pkg/apis/apiextensions/v1/marshal.go#L28-L37

If gomod is pruning that code for some reason, then the implementsJSONMarshaler interface wouldn't be satisfied, which means the fix in #427 wouldn't be triggered.

tshak · 2020-11-24T14:43:22Z

I'm using Go Modules but nevertheless I confirmed that the code exists when I Go to Definition in VS code. I'm on v0.18.12 of k8s.io/apiextensions-apiserver. I'm not sure if any of this is relevant though as the fix in #427 was reverted in #505.

maxsmythe · 2020-11-25T20:55:51Z

It looks like that reversion is in v0.4.1, but not v0.4.0.

I can confirm that generating CRDs that embed JSONSchema is broken due to the reversion in #505

@DirectXMan12 Should the reversion be un-reverted? Are there any plans to accommodate embedding JSONSchema in other ways?

maxsmythe · 2020-11-25T22:48:26Z

/reopen

Re-opening as this issue has been de-mitigated

k8s-ci-robot · 2020-11-25T22:48:32Z

@maxsmythe: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Re-opening as this issue has been de-mitigated

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

maxsmythe · 2020-11-25T22:49:01Z

Looks like I don't have permissions to re-open. Can it be re-opened?

This marker will avoid trying to do any type detection on any struct field on which it is set. This gives users a safety valve when they hit an edge case where type inference does the wrong thing for them. This fixes kubernetes-sigs#291, which was recently re-broken by fixing kubernetes-sigs#502 Signed-off-by: Max Smythe <smythe@google.com>

jiachengxu mentioned this issue Sep 23, 2019

🐛Add apiextension JSONSchemaPropsOrArray, JSONSchemaPropsOrBool, JSONSchemaOrStringArray to known_types.go #326

Closed

k8s-ci-robot added the help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. label Nov 15, 2019

jiachengxu mentioned this issue Dec 4, 2019

Tender operator integration kubermatic/kubecarrier#48

Merged

jiachengxu mentioned this issue Dec 19, 2019

[Catalog] [Controller-tools] Manifests which generated by controller-gen from CRD that contains CustomResourceValidation can not pass the schema checks of kubernetes v1.16.0+ kubermatic/kubecarrier#106

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 13, 2020

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 13, 2020

muvaf mentioned this issue Apr 8, 2020

InfrastructureDefinition type and CRD generation crossplane/crossplane#1401

Closed

4 tasks

maxsmythe mentioned this issue Apr 18, 2020

enum in ConstraintTemplate OpenAPI schema causes gatekeeper controller panic open-policy-agent/gatekeeper#557

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 19, 2020

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 20, 2020

DirectXMan12 closed this as completed Sep 9, 2020

tshak mentioned this issue Nov 20, 2020

controller-gen does not allow embedding of JSONSchemaProps #525

Closed

maxsmythe mentioned this issue Nov 25, 2020

🐛 Don't generate schema with Any in it, its not supported #505

Merged

maxsmythe mentioned this issue Nov 26, 2020

✨ Add crd:validation:Schemaless marker #528

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

controller-gen doesn't work with k8s.io/apiextensions-apiserver #291

controller-gen doesn't work with k8s.io/apiextensions-apiserver #291

jiachengxu commented Jul 31, 2019

munnerz commented Aug 1, 2019

jiachengxu commented Aug 1, 2019

munnerz commented Aug 1, 2019

jiachengxu commented Aug 1, 2019

DirectXMan12 commented Sep 30, 2019 •

edited

Loading

maxsmythe commented Nov 12, 2019 •

edited

Loading

thetechnick commented Nov 14, 2019

DirectXMan12 commented Nov 15, 2019

k8s-ci-robot commented Nov 15, 2019

fejta-bot commented Feb 13, 2020

maxsmythe commented Feb 13, 2020

muvaf commented Apr 8, 2020

DirectXMan12 commented Apr 20, 2020

maxsmythe commented Apr 20, 2020

fejta-bot commented Jul 19, 2020

maxsmythe commented Jul 20, 2020

DirectXMan12 commented Sep 9, 2020

tshak commented Nov 19, 2020 •

edited

Loading

julianKatz commented Nov 20, 2020

tshak commented Nov 21, 2020

maxsmythe commented Nov 23, 2020

tshak commented Nov 24, 2020

maxsmythe commented Nov 25, 2020

maxsmythe commented Nov 25, 2020

k8s-ci-robot commented Nov 25, 2020

maxsmythe commented Nov 25, 2020

controller-gen doesn't work with k8s.io/apiextensions-apiserver #291

controller-gen doesn't work with k8s.io/apiextensions-apiserver #291

Comments

jiachengxu commented Jul 31, 2019

munnerz commented Aug 1, 2019

jiachengxu commented Aug 1, 2019

munnerz commented Aug 1, 2019

jiachengxu commented Aug 1, 2019

DirectXMan12 commented Sep 30, 2019 • edited Loading

maxsmythe commented Nov 12, 2019 • edited Loading

thetechnick commented Nov 14, 2019

DirectXMan12 commented Nov 15, 2019

k8s-ci-robot commented Nov 15, 2019

fejta-bot commented Feb 13, 2020

maxsmythe commented Feb 13, 2020

muvaf commented Apr 8, 2020

DirectXMan12 commented Apr 20, 2020

maxsmythe commented Apr 20, 2020

fejta-bot commented Jul 19, 2020

maxsmythe commented Jul 20, 2020

DirectXMan12 commented Sep 9, 2020

tshak commented Nov 19, 2020 • edited Loading

julianKatz commented Nov 20, 2020

tshak commented Nov 21, 2020

maxsmythe commented Nov 23, 2020

tshak commented Nov 24, 2020

maxsmythe commented Nov 25, 2020

maxsmythe commented Nov 25, 2020

k8s-ci-robot commented Nov 25, 2020

maxsmythe commented Nov 25, 2020

DirectXMan12 commented Sep 30, 2019 •

edited

Loading

maxsmythe commented Nov 12, 2019 •

edited

Loading

tshak commented Nov 19, 2020 •

edited

Loading