cmd/operator-sdk: `bundle build` builds operator bundle images #2076

dinhxuanvu · 2019-10-17T07:14:26Z

Description of the change:

bundle build command to build build manifests image from local bundle manifests.

This bundle commands will use bundle library from operator-registry repository.

Motivation for the change:

Enable operator-sdk to build bundle manifests images using bundle library from
operator-registry repository.

1. `bundle generate` command to generate `annotations.yaml` metadata from bundle manifests 2. `bundle build` command to build build manifests image from local bundle manifests. In the background, the `generate` command will be run as a part of `build` command to generate `annotations.yaml` metadata. This bundle commands will use bundle library from operator-registry repository. Signed-off-by: Vu Dinh <vdinh@redhat.com>

camilamacedo86 · 2019-10-22T11:30:58Z

Hi @dinhxuanvu,

Thank you for your contribution. 🥇

I'd like to share that IMHO we should do it using the current operator-sdk olm commands instead of changing the build one and in the current structure used by SDK. See my comments here #1913 in order to make it clear.

However, please wait to see what the others in the team think about it before doing changes. They may have another point of view over it. I will just let my comments in order to share/raise it.

c/c @joelanford

camilamacedo86 · 2019-10-22T11:42:03Z

cmd/operator-sdk/bundle/build.go

+// newBundleBuildCmd returns a command that will build operator bundle image.
+func newBundleBuildCmd() *cobra.Command {
+	bundleBuildCmd := &cobra.Command{
+		Use:   "build",


I think it should be something as operator-sdk olm-catalog gen-csv --csv-version 0.1.0 --create-bundle

Following why I think it.

The --csv-version 0.1.0 is required to know the version which should be used to create the bundle

I understand that the CSV is part of the bundle according to the info in https://operatorhub.io/bundle and https://github.com/operator-framework/community-operators

The operator-SDK has already a structure to generate the olm-catalogue files then in POV would be easier/better to keep it.

Hey Camila,

Thank you for the initial review. I would like to add the bundle enhancement and bundle library docs so you can take a look and have a better understanding on what my team is trying to do with this PR. I should've added those docs on commit message but I forgot. My apology.

Bundle Enhancement

Bundle Library Doc

One note that the PR for adding bundle package to operator-registry is still pending-merge so that's why this PR fails some tests due to the package is not available yet.

The goal of this PR to add a new bundle command under the top-level operator-sdk command. The bundle command itself has two subcommands to generate bundle metadata and build bundle container image. It should strictly use bundle package/library to perform its tasks without interfering with any existing functionalities of SDK. This PR shouldn't change or impact any existing SDK commands/subcommands. In a way, this is a "plug-in" that is independent in some way. I do not wish to nest/add/wrap this bundle command tool under other existing SDK commands as it is a brand-new feature and if we do so, it will affect other existing commands which I believe something Joe doesn't want. Secondly, bundle command requires a few flags that if nested under other commands will create very messy UX as the commands will be too longer and confusing especially if the command contains something that has nothing to do with creating bundle image such as olm-catalog or even olm itself. OLM is supposed to be a consumer of bundle images, not creator.

If you have any further questions, please feel free to ask. :)

Hi @dinhxuanvu,

Thank you for share the info. Following my main concerns/review on it.

IMHO

it should follow up the same idea/standard used in the other SDK commands

it should by default use the SDK structure project and the flags are optional instead of mandatory

It is not passing in the tests

it is missing docs, tests, changelog

it shows missing the impl of each command.

camilamacedo86 · 2019-10-23T06:43:42Z

cmd/operator-sdk/bundle/build.go

+        After the build process is completed, a container image would be built
+        locally in docker and available to push to a container registry.
+
+        $ operator-sdk bundle build -d /test/ -t quay.io/example/operator:v0.1.0 \


here is with the short flags values when down it is with the full flag values. IMHO all should follow the same standard. I'd keep the full flag in order to make clear.

camilamacedo86 · 2019-10-23T06:46:43Z

cmd/operator-sdk/bundle/generate.go

+	bundleGenerateCmd := &cobra.Command{
+		Use:   "generate",
+		Short: "Generate operator bundle metadata and Dockerfile",
+		Long: `The "operator-sdk bundle generate" command will generate operator


Why is required a operator-sdk bundle build and a operator-sdk bundle generate one?
What are the diff between both?

@dinhxuanvu based on the current command descriptions, a user wouldn't necessarily be able to tell the difference between the commands. Can you specify that generate only creates the operator metadata without building the image, and that a non-default builder can be used to build the metadata image?

Additional flags are added to include package and channels information into the annotations.yaml. Plus, several optional flags for different use case options. Signed-off-by: Vu Dinh <vdinh@redhat.com>

openshift-ci-robot · 2019-10-24T02:13:01Z

@dinhxuanvu: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/prow/unit	`b700d51`	link	`/test unit`
ci/prow/sanity	`b700d51`	link	`/test sanity`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

camilamacedo86

Not passing in the tests and missing the common requirements: #2076 (comment)

estroz · 2019-12-11T23:45:02Z

cmd/operator-sdk/bundle/build.go

+	"github.com/spf13/cobra"
+)
+
+var (


This command would be easier to manage by having these variables be struct fields:

type bundleCmd struct { dirBuildArgs string tagBuildArgs string imageBuilderArgs string packageNameArgs string channelsArgs string channelDefaultArgs string overwriteArgs bool } func newBundleBuildCmd() *cobra.Command { c := bundleCmd{} bundleBuildCmd := &cobra.Command{ ... RunE: func(cmd *cobra.Command, args []string) error { return bundle.BuildFunc(c.dirBuildArgs, c.tagBuildArgs, c.imageBuilderArgs, c.packageNameArgs, c.channelsArgs, c.channelDefaultArgs, c.overwriteArgs) } } bundleBuildCmd.Flags().StringVarP(&c.dirBuildArgs, "directory", "d", "", "The directory where bundle manifests are located") ... }

Since generate args are a subset of build's, this struct can be shared between commands.

estroz

This command should be put under the alpha subcommand since it is a new feature that may change in the near future.

CLI wrappers for operator-registry bundle image functions

cmd/operator-sdk/alpha/bundle/generate.go

cmd/operator-sdk/alpha/bundle/build.go

joelanford · 2020-01-14T01:30:19Z

cmd/operator-sdk/alpha/cmd.go

+	cmd.AddCommand(
+		olm.NewCmd(),
+		kubebuilder.NewCmd(),
+		bundle.NewCmd(),


We need to pretty quickly decide where to put the OLM integration subcommands in the CLI. I think alpha is probably okay for this PR, but we'll need a follow-up on the heels of this to promote these OLM alpha subcommands.

doc/cli/operator-sdk_alpha.md

cmd/operator-sdk/alpha/bundle/build.go

camilamacedo86 · 2020-01-15T11:40:45Z

Hi @dinhxuanvu,

Really tks for your contribution. 🥇 I just add a few comments and I'd like to highlight that it is not passing in the CI.

camilamacedo86 · 2020-01-15T19:58:33Z

Hi @dinhxuanvu,

It still not passing see:

--- FAIL: TestGenerate (0.01s)
    --- FAIL: TestGenerate/Generate_Go_CRD (0.00s)
        crd_test.go:87: Wanted nil error, got: error generating CRD manifests: error running CRD generator: unable to parse option "output:crd:cache:dir=//tmp/019565286/5577006791947779410": [invalid digit '9' in octal literal (at <input>:1:11

dinhxuanvu · 2020-01-15T20:43:57Z

@camilamacedo86 Hey Camila. Eric is taking over this PR and currently making changes to align with what SDK team plans to do with the bundle library integration. So he is in charge of this PR now. I will help out with the review and making changes to bundle library on operator-registry repo if needed.

camilamacedo86

It shows fine for me. 👍
Great contribution 🥇
/lgtm
/apprroved

PS.: It needs to be checked by @joelanford and/or @estroz before be merged.

openshift-ci-robot · 2020-01-16T22:25:05Z

New changes are detected. LGTM label has been removed.

openshift-ci-robot requested review from AlexNPavel and joelanford October 17, 2019 07:14

openshift-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 17, 2019

dinhxuanvu changed the title ~~Add bundle commands cli to build bundle manifests images~~ [WIP] Add bundle commands cli to build bundle manifests images Oct 17, 2019

dinhxuanvu changed the title ~~[WIP] Add bundle commands cli to build bundle manifests images~~ Add bundle commands cli to build bundle manifests images Oct 22, 2019

camilamacedo86 reviewed Oct 22, 2019

View reviewed changes

dinhxuanvu force-pushed the bundle-cli branch from 8e93155 to 03530ea Compare October 23, 2019 02:47

camilamacedo86 reviewed Oct 23, 2019

View reviewed changes

Update bundle command to add new flags

b700d51

Additional flags are added to include package and channels information into the annotations.yaml. Plus, several optional flags for different use case options. Signed-off-by: Vu Dinh <vdinh@redhat.com>

dinhxuanvu force-pushed the bundle-cli branch from 03530ea to b700d51 Compare October 24, 2019 02:11

camilamacedo86 added kind/feature Categorizes issue or PR as related to a new feature. olm-integration Issue relates to the OLM integration labels Nov 7, 2019

camilamacedo86 suggested changes Nov 19, 2019

View reviewed changes

joelanford mentioned this pull request Nov 21, 2019

Operator courier future #2245

Closed

camilamacedo86 added needs-changes and removed needs-changes labels Nov 21, 2019

estroz self-requested a review December 3, 2019 18:10

estroz reviewed Dec 11, 2019

View reviewed changes

estroz added 2 commits January 13, 2020 14:43

Merge branch 'master' into bundle-cli

65bc6ae

cmd/operator-sdk/alpha/bundle: 'bundle build/generate' commands are

3de1842

CLI wrappers for operator-registry bundle image functions

estroz requested review from camilamacedo86, hasbro17 and jmrodri January 14, 2020 00:48

CHANGELOG.md,doc/cli: add bundle build/generate additions

5da8844

joelanford reviewed Jan 14, 2020

View reviewed changes

camilamacedo86 reviewed Jan 15, 2020

View reviewed changes

doc/cli/operator-sdk_alpha.md Outdated Show resolved Hide resolved

camilamacedo86 reviewed Jan 15, 2020

View reviewed changes

cmd/operator-sdk/alpha/bundle/build.go Show resolved Hide resolved

add license headers

9256390

improve command descriptions and flags

2de234b

estroz force-pushed the bundle-cli branch from 25ff108 to 2de234b Compare January 15, 2020 20:32

add another example

e9df5b9

estroz requested review from camilamacedo86 and joelanford January 15, 2020 20:38

estroz self-assigned this Jan 15, 2020

fix typos

828cc14

estroz force-pushed the bundle-cli branch from d854b9f to 828cc14 Compare January 15, 2020 21:15

estroz added 2 commits January 15, 2020 14:07

Merge branch 'master' into bundle-cli

57d32d7

update bundle build with better defaults and validation

d7c91a7

camilamacedo86 approved these changes Jan 16, 2020

View reviewed changes

openshift-ci-robot assigned camilamacedo86 Jan 16, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 16, 2020

a few updates based on prior discussion

5516f47

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Jan 16, 2020

update docs

ed1dd1e

estroz requested review from shawn-hurley and estroz January 17, 2020 18:29

image tag is a positional argument

1448393

estroz changed the title ~~Add bundle commands cli to build bundle manifests images~~ cmd/operator-sdk: bundle build builds operator bundle images Jan 17, 2020

estroz merged commit 4ea8701 into operator-framework:master Jan 17, 2020

estroz deleted the bundle-cli branch January 17, 2020 19:51

joelanford mentioned this pull request Apr 2, 2020

Auto-generating the CHANGELOG and Migration Guide #2761

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/operator-sdk: `bundle build` builds operator bundle images #2076

cmd/operator-sdk: `bundle build` builds operator bundle images #2076

dinhxuanvu commented Oct 17, 2019 •

edited by estroz

Loading

camilamacedo86 commented Oct 22, 2019 •

edited

Loading

camilamacedo86 Oct 22, 2019 •

edited

Loading

dinhxuanvu Oct 23, 2019 •

edited

Loading

camilamacedo86 Oct 23, 2019

camilamacedo86 Oct 23, 2019

camilamacedo86 Oct 23, 2019

estroz Dec 11, 2019

openshift-ci-robot commented Oct 24, 2019

camilamacedo86 left a comment

estroz Dec 11, 2019 •

edited

Loading

estroz left a comment

joelanford Jan 14, 2020

camilamacedo86 commented Jan 15, 2020

camilamacedo86 commented Jan 15, 2020

dinhxuanvu commented Jan 15, 2020

camilamacedo86 left a comment •

edited

Loading

openshift-ci-robot commented Jan 16, 2020

cmd/operator-sdk: bundle build builds operator bundle images #2076

cmd/operator-sdk: bundle build builds operator bundle images #2076

Conversation

dinhxuanvu commented Oct 17, 2019 • edited by estroz Loading

camilamacedo86 commented Oct 22, 2019 • edited Loading

camilamacedo86 Oct 22, 2019 • edited Loading

Choose a reason for hiding this comment

dinhxuanvu Oct 23, 2019 • edited Loading

Choose a reason for hiding this comment

camilamacedo86 Oct 23, 2019

Choose a reason for hiding this comment

camilamacedo86 Oct 23, 2019

Choose a reason for hiding this comment

camilamacedo86 Oct 23, 2019

Choose a reason for hiding this comment

estroz Dec 11, 2019

Choose a reason for hiding this comment

openshift-ci-robot commented Oct 24, 2019

camilamacedo86 left a comment

Choose a reason for hiding this comment

estroz Dec 11, 2019 • edited Loading

Choose a reason for hiding this comment

estroz left a comment

Choose a reason for hiding this comment

joelanford Jan 14, 2020

Choose a reason for hiding this comment

camilamacedo86 commented Jan 15, 2020

camilamacedo86 commented Jan 15, 2020

dinhxuanvu commented Jan 15, 2020

camilamacedo86 left a comment • edited Loading

Choose a reason for hiding this comment

openshift-ci-robot commented Jan 16, 2020

cmd/operator-sdk: `bundle build` builds operator bundle images #2076

cmd/operator-sdk: `bundle build` builds operator bundle images #2076

dinhxuanvu commented Oct 17, 2019 •

edited by estroz

Loading

camilamacedo86 commented Oct 22, 2019 •

edited

Loading

camilamacedo86 Oct 22, 2019 •

edited

Loading

dinhxuanvu Oct 23, 2019 •

edited

Loading

estroz Dec 11, 2019 •

edited

Loading

camilamacedo86 left a comment •

edited

Loading