Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate 'generate openapi', add 'generate crds' #2276

Merged
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
- Support for vars in top level ansible watches. ([#2147](https://github.com/operator-framework/operator-sdk/pull/2147))
- Support for `"ansible.operator-sdk/verbosity"` annotation on Custom Resources watched by Ansible based operators to override verbosity on an individual resource. ([#2102](https://github.com/operator-framework/operator-sdk/pull/2102))
- Support for relative helm chart paths in the Helm operator's watches.yaml file. ([#2287](https://github.com/operator-framework/operator-sdk/pull/2287))
- New `operator-sdk generate crds` subcommand, which generates CRDs from Go types. ([#2276](https://github.com/operator-framework/operator-sdk/pull/2276))

### Changed
- Upgrade minimal Ansible version in the init projects from `2.4` to `2.6`. ([#2107](https://github.com/operator-framework/operator-sdk/pull/2107))
Expand All @@ -15,10 +16,10 @@
- Upgrade the Ansible version from `2.8` to `2.9` on the Ansible based operators image. ([#2168](https://github.com/operator-framework/operator-sdk/issues/2168))
- Updated CRD generation for non-Go operators to use valid structural schema. ([#2275](https://github.com/operator-framework/operator-sdk/issues/2275))
- Replace Role verb `"*"` with list of verb strings in generated files so the Role is compatible with OpenShift and Kubernetes. ([#2175](https://github.com/operator-framework/operator-sdk/pull/2175))
- **Breaking change:** Renamed `operator-sdk generate openapi` to `operator-sdk generate crds`. ([#2276](https://github.com/operator-framework/operator-sdk/pull/2276))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if we should move forward with. See that kubebuilder is not supporting it and users are asking for here and a track issue from them here which in my understand shows that require it as well.

Copy link
Member

@estroz estroz Nov 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first issue is fair; based on feedback in #2042 some users do not use zz_generated.openapi.go, which, on top of it not being necessary for API code unlike zz_generated.deepcopy.go, is why I decided to push for removing it. We should either:

  1. Add documentation on building and using openapi-gen to generate the files yourself, which is fairly straightforward.
  2. Keep generate openapi as a command but move CRD generation into generate crd.

I vote for option 1 since we can offer Go OpenAPI code by wrapping openapi-gen in a Makefile recipe (in the future).

The second issue describes something we do not currently support in API code right now since we use the same tool as kubebuilder to generate CRDs, controller-gen from controller-tools. Generating zz_generated.openapi.go does not change the state of supported annotations for CRD manifest validation in the SDK.

We agreed to make this change because it de-couples CRD generation from OpenAPI spec generation, two different albeit related processes. This change will help users debug errors more easily as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option would be to add generate crds but keep generate openapi as is and deprecate it. We could hide it from the help output and log a deprecation message when it is called.

Then in a later version it could be removed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And we could/should still do one of the options @estroz suggested.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joelanford I like the combination of what you suggested and my first option. The docs referencing Go OpenAPI spec generation (I believe the only documentation doing so is the one being changed in this PR already) should warn that the subcommand is deprecated and users should switch to installing and running openapi-gen themselves if they still want that spec file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea of phasing out the old command with a deprecation warning.

@estroz @joelanford we need to make it easy to download and use the openapi-gen command if that is the guidance, we either do it for the user and attempt to install the binary into a known location and give them the option to supply their own openapi-gen.

Could we vendor the openapi-gen command and use it in the operator-sdk command? was that an option already looked?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shawn-hurley we currently vendor the exposed openapi-gen generator in generate openapi. If we want to install this binary for the user, I suggest this as the first candidate for a Makefile:

openapi-gen:
	which ./bin/openapi-gen > /dev/null || go build -o ./bin/openapi-gen k8s.io/kube-openapi/cmd/openapi-gen
	./bin/openapi-gen ...

Otherwise documentation to the effect of the above script should suffice. I'd rather have documentation than a make recipe.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, our vendoring of the code generation libraries is what causes the GOROOT and panic issues described in this issue: #1854 (comment)

If we direct users to build openapi-gen from source like Eric suggests, we'll help them avoid that issue, at least for the openapi generation (I think it will still be an issue for deepcopy generation).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, updated the PR to keep generate openapi, but to hide it from help and deprecate it.


### Deprecated

### Removed
- Deprecated the `operator-sdk generate openapi` command. CRD generation is still supported with `operator-sdk generate crds`. It is now recommended to use [openapi-gen](https://github.com/kubernetes/kube-openapi/tree/master/cmd/openapi-gen) directly for OpenAPI code generation. The `generate openapi` subcommand will be removed in a future release. ([#2276](https://github.com/operator-framework/operator-sdk/pull/2276))

### Bug Fixes
- Fix issue faced in the Ansible based operators when `jmespath` queries are used because it was not installed. ([#2252](https://github.com/operator-framework/operator-sdk/pull/2252))
Expand Down
21 changes: 10 additions & 11 deletions cmd/operator-sdk/add/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,17 +40,17 @@ func newAddApiCmd() *cobra.Command {
apiCmd := &cobra.Command{
Use: "api",
Short: "Adds a new api definition under pkg/apis",
Long: `operator-sdk add api --kind=<kind> --api-version=<group/version> creates the
api definition for a new custom resource under pkg/apis. This command must be
run from the project root directory. If the api already exists at
pkg/apis/<group>/<version> then the command will not overwrite and return an
error.
Long: `operator-sdk add api --kind=<kind> --api-version=<group/version> creates
the api definition for a new custom resource under pkg/apis. This command
must be run from the project root directory. If the api already exists at
pkg/apis/<group>/<version> then the command will not overwrite and return
an error.

By default, this command runs Kubernetes deepcopy and OpenAPI V3 generators on
By default, this command runs Kubernetes deepcopy and CRD generators on
tagged types in all paths under pkg/apis. Go code is generated under
pkg/apis/<group>/<version>/zz_generated.{deepcopy,openapi}.go. CRD's are
generated, or updated if they exist for a particular group + version + kind,
under deploy/crds/<full group>_<resource>_crd.yaml; OpenAPI V3 validation YAML
pkg/apis/<group>/<version>/zz_generated.deepcopy.go. CRD's are generated,
or updated if they exist for a particular group + version + kind, under
deploy/crds/<full group>_<resource>_crd.yaml; OpenAPI V3 validation YAML
is generated as a 'validation' object. Generation can be disabled with the
--skip-generation flag.

Expand All @@ -67,7 +67,6 @@ Example:
├── register.go
├── appservice_types.go
├── zz_generated.deepcopy.go
├── zz_generated.openapi.go
$ tree deploy/crds
├── deploy/crds/app.example.com_v1alpha1_appservice_cr.yaml
├── deploy/crds/app.example.com_appservices_crd.yaml
Expand Down Expand Up @@ -143,7 +142,7 @@ func apiRun(cmd *cobra.Command, args []string) error {
}

// Generate a validation spec for the new CRD.
if err := genutil.OpenAPIGen(); err != nil {
if err := genutil.CRDGen(); err != nil {
return err
}
}
Expand Down
1 change: 1 addition & 0 deletions cmd/operator-sdk/generate/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ func NewCmd() *cobra.Command {
Long: `The operator-sdk generate command invokes specific generator to generate code as needed.`,
}
cmd.AddCommand(newGenerateK8SCmd())
cmd.AddCommand(newGenerateCRDsCmd())
cmd.AddCommand(newGenerateOpenAPICmd())
return cmd
}
49 changes: 49 additions & 0 deletions cmd/operator-sdk/generate/crds.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
// Copyright 2019 The Operator-SDK Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package generate

import (
"fmt"

"github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil"
"github.com/spf13/cobra"
)

func newGenerateCRDsCmd() *cobra.Command {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a flag for passing in the location of the CRD's?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once this merges, I'll make an issue to discuss this idea for a follow-up.

return &cobra.Command{
Use: "crds",
Short: "Generates CRDs for API's",
Long: `generate crds generates CRDs or updates them if they exist,
under deploy/crds/<full group>_<resource>_crd.yaml; OpenAPI
V3 validation YAML is generated as a 'validation' object.

Example:

$ operator-sdk generate crds
$ tree deploy/crds
├── deploy/crds/app.example.com_v1alpha1_appservice_cr.yaml
├── deploy/crds/app.example.com_appservices_crd.yaml
`,
RunE: crdsFunc,
}
}

func crdsFunc(cmd *cobra.Command, args []string) error {
if len(args) != 0 {
return fmt.Errorf("command %s doesn't accept any arguments", cmd.CommandPath())
}

return genutil.CRDGen()
}
31 changes: 28 additions & 3 deletions cmd/operator-sdk/generate/openapi.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,9 @@ import (

func newGenerateOpenAPICmd() *cobra.Command {
return &cobra.Command{
Use: "openapi",
Short: "Generates OpenAPI specs for API's",
Hidden: true,
Use: "openapi",
Short: "Generates OpenAPI specs for API's",
Long: `generate openapi generates OpenAPI validation specs in Go from tagged types
in all pkg/apis/<group>/<version> directories. Go code is generated under
pkg/apis/<group>/<version>/zz_generated.openapi.go. CRD's are generated, or
Expand All @@ -48,10 +49,34 @@ Example:
}
}

const deprecationTemplate = "\033[1;36m%s\033[0m"

func openAPIFunc(cmd *cobra.Command, args []string) error {
fmt.Printf(deprecationTemplate, `[Deprecation notice] The 'operator-sdk generate openapi' command is deprecated!

- To generate CRDs, use 'operator-sdk generate crds'.
- To generate Go OpenAPI code, use 'openapi-gen'. For example:

# Build the latest openapi-gen from source
which ./bin/openapi-gen > /dev/null || go build -o ./bin/openapi-gen k8s.io/kube-openapi/cmd/openapi-gen

# Run openapi-gen for each of your API group/version packages
./bin/openapi-gen --logtostderr=true -o "" -i ./pkg/apis/<group>/<version> -O zz_generated.openapi -p ./pkg/apis/<group>/<version> -h ./hack/boilerplate.go.txt -r "-"

`)

if len(args) != 0 {
return fmt.Errorf("command %s doesn't accept any arguments", cmd.CommandPath())
}

return genutil.OpenAPIGen()
fs := []func() error{
genutil.OpenAPIGen,
genutil.CRDGen,
}
for _, f := range fs {
if err := f(); err != nil {
return err
}
}
return nil
}
81 changes: 81 additions & 0 deletions cmd/operator-sdk/internal/genutil/crds.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
// Copyright 2018 The Operator-SDK Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package genutil

import (
"fmt"
"path/filepath"
"strings"

"github.com/operator-framework/operator-sdk/internal/scaffold"
"github.com/operator-framework/operator-sdk/internal/scaffold/input"
"github.com/operator-framework/operator-sdk/internal/util/k8sutil"
"github.com/operator-framework/operator-sdk/internal/util/projutil"

log "github.com/sirupsen/logrus"
)

// CRDGen generates CRDs for all APIs in pkg/apis.
func CRDGen() error {
projutil.MustInProjectRoot()

absProjectPath := projutil.MustGetwd()
repoPkg := projutil.GetGoPkg()

gvMap, err := k8sutil.ParseGroupSubpackages(scaffold.ApisDir)
if err != nil {
return fmt.Errorf("failed to parse group versions: (%v)", err)
}
gvb := &strings.Builder{}
for g, vs := range gvMap {
gvb.WriteString(fmt.Sprintf("%s:%v, ", g, vs))
}

log.Infof("Running CRD generation for Custom Resource group versions: [%v]\n", gvb.String())

s := &scaffold.Scaffold{}
cfg := &input.Config{
Repo: repoPkg,
AbsProjectPath: absProjectPath,
ProjectName: filepath.Base(absProjectPath),
}
crds, err := k8sutil.GetCRDs(scaffold.CRDsDir)
if err != nil {
return err
}
for _, crd := range crds {
g, v, k := crd.Spec.Group, crd.Spec.Version, crd.Spec.Names.Kind
if v == "" {
if len(crd.Spec.Versions) != 0 {
v = crd.Spec.Versions[0].Name
} else {
return fmt.Errorf("crd of group %s kind %s has no version", g, k)
}
}
r, err := scaffold.NewResource(g+"/"+v, k)
if err != nil {
return err
}
err = s.Execute(cfg,
&scaffold.CRD{Resource: r, IsOperatorGo: projutil.IsOperatorGo()},
)
if err != nil {
return err
}
}

log.Info("CRD generation complete.")
return nil
}
33 changes: 0 additions & 33 deletions cmd/operator-sdk/internal/genutil/openapi.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ import (
"strings"

"github.com/operator-framework/operator-sdk/internal/scaffold"
"github.com/operator-framework/operator-sdk/internal/scaffold/input"
"github.com/operator-framework/operator-sdk/internal/util/k8sutil"
"github.com/operator-framework/operator-sdk/internal/util/projutil"

Expand All @@ -36,7 +35,6 @@ import (
func OpenAPIGen() error {
projutil.MustInProjectRoot()

absProjectPath := projutil.MustGetwd()
repoPkg := projutil.GetGoPkg()

gvMap, err := k8sutil.ParseGroupSubpackages(scaffold.ApisDir)
Expand All @@ -57,37 +55,6 @@ func OpenAPIGen() error {
return err
}

s := &scaffold.Scaffold{}
cfg := &input.Config{
Repo: repoPkg,
AbsProjectPath: absProjectPath,
ProjectName: filepath.Base(absProjectPath),
}
crds, err := k8sutil.GetCRDs(scaffold.CRDsDir)
if err != nil {
return err
}
for _, crd := range crds {
g, v, k := crd.Spec.Group, crd.Spec.Version, crd.Spec.Names.Kind
if v == "" {
if len(crd.Spec.Versions) != 0 {
v = crd.Spec.Versions[0].Name
} else {
return fmt.Errorf("crd of group %s kind %s has no version", g, k)
}
}
r, err := scaffold.NewResource(g+"/"+v, k)
if err != nil {
return err
}
err = s.Execute(cfg,
&scaffold.CRD{Resource: r, IsOperatorGo: projutil.IsOperatorGo()},
)
if err != nil {
return err
}
}

log.Info("Code-generation complete.")
return nil
}
Expand Down
19 changes: 9 additions & 10 deletions doc/cli/operator-sdk_add_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@ Adds a new api definition under pkg/apis

### Synopsis

operator-sdk add api --kind=<kind> --api-version=<group/version> creates the
api definition for a new custom resource under pkg/apis. This command must be
run from the project root directory. If the api already exists at
pkg/apis/<group>/<version> then the command will not overwrite and return an
error.
operator-sdk add api --kind=<kind> --api-version=<group/version> creates
the api definition for a new custom resource under pkg/apis. This command
must be run from the project root directory. If the api already exists at
pkg/apis/<group>/<version> then the command will not overwrite and return
an error.

By default, this command runs Kubernetes deepcopy and OpenAPI V3 generators on
By default, this command runs Kubernetes deepcopy and CRD generators on
tagged types in all paths under pkg/apis. Go code is generated under
pkg/apis/<group>/<version>/zz_generated.{deepcopy,openapi}.go. CRD's are
generated, or updated if they exist for a particular group + version + kind,
under deploy/crds/<full group>_<resource>_crd.yaml; OpenAPI V3 validation YAML
pkg/apis/<group>/<version>/zz_generated.deepcopy.go. CRD's are generated,
or updated if they exist for a particular group + version + kind, under
deploy/crds/<full group>_<resource>_crd.yaml; OpenAPI V3 validation YAML
is generated as a 'validation' object. Generation can be disabled with the
--skip-generation flag.

Expand All @@ -31,7 +31,6 @@ Example:
├── register.go
├── appservice_types.go
├── zz_generated.deepcopy.go
├── zz_generated.openapi.go
$ tree deploy/crds
├── deploy/crds/app.example.com_v1alpha1_appservice_cr.yaml
├── deploy/crds/app.example.com_appservices_crd.yaml
Expand Down
2 changes: 1 addition & 1 deletion doc/cli/operator-sdk_generate.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,6 @@ The operator-sdk generate command invokes specific generator to generate code as
### SEE ALSO

* [operator-sdk](operator-sdk.md) - An SDK for building operators with ease
* [operator-sdk generate crds](operator-sdk_generate_crds.md) - Generates CRDs for API's
* [operator-sdk generate k8s](operator-sdk_generate_k8s.md) - Generates Kubernetes code for custom resource
* [operator-sdk generate openapi](operator-sdk_generate_openapi.md) - Generates OpenAPI specs for API's

32 changes: 32 additions & 0 deletions doc/cli/operator-sdk_generate_crds.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
## operator-sdk generate crds

Generates CRDs for API's

### Synopsis

generate crds generates CRDs or updates them if they exist,
under deploy/crds/<full group>_<resource>_crd.yaml; OpenAPI
V3 validation YAML is generated as a 'validation' object.

Example:

$ operator-sdk generate crds
$ tree deploy/crds
├── deploy/crds/app.example.com_v1alpha1_appservice_cr.yaml
├── deploy/crds/app.example.com_appservices_crd.yaml


```
operator-sdk generate crds [flags]
```

### Options

```
-h, --help help for crds
```

### SEE ALSO

* [operator-sdk generate](operator-sdk_generate.md) - Invokes specific generator

Loading