Skip to content

Commit

Permalink
Updated comments and README
Browse files Browse the repository at this point in the history 
Signed-off-by: jubittajohn <jujohn@redhat.com>
  • Loading branch information
@jubittajohn
jubittajohn committed Aug 11, 2023
1 parent b2a70c9 commit 321525f
Show file tree
Hide file tree
Showing 4 changed files with 155 additions and 86 deletions.
137 changes: 103 additions & 34 deletions test/operator-framework-e2e/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,18 @@ The steps in the ginkgo test can be summarized as follows:
- With the scale of the OLM v1 project, it is useful to have a means to test components in the operator development + lifecycle pipeline together to create a more cohesive experience for all users.

## Getting Started

- This test currently only works with the container runtime `docker`.
- Building operator-controller, deploying it into the cluster and rest of the configuration is done in the `MakeFile` of this repo under the target `operator-developer-e2e`. This includes:

- Setting up a kind cluster named `operator-controller-op-dev-e2e`.
- Installing the operator controller onto the cluster.
- Setting up `opm`, `operator-sdk` and `kustomize` using bingo.
- Setting up a local registry server for building and loading images.
### Input Values used

Below are the input values used in the test which is specified in the `operator_framework_test.go`.

- The following structs defined are required as input for both plain+v0 and registry+v1 bundles:
- The following structs defined are required, to accept input for both `plain+v0` and `registry+v1` bundles:
- For getting bundle related inputs:
```
type BundleInfo struct {
Expand All @@ -48,11 +51,11 @@ The steps in the ginkgo test can be summarized as follows:
imageRef string
}
```
- `baseFolderPath` - Base/root path of the folder for the specific bundle type input data.[path to plain-v0 or registry-v1 bundles testdata]
- `baseFolderPath` - Base/root path of the folder where the specific bundle type input data is stored.[root path to plain-v0 or registry-v1 bundles testdata]
- `bundles` - Stores the data relevant to different versions of the bundle.
- `bInputDir` - The input directory containing the specific version of the bundle data.
- `bInputDir` - The directory that stores the specific version of the bundle data. The name of the directory is formed and is of the format `<operatorName>.v<bundleVersion>`.
- `bundleVersion` - The specific version of the bundle data.
- `imageRef` - This is formed. Stores the bundle image reference which will be of the format `localhost:5001/<operator_name>-bundle:v.<bundleVersion>`
- `imageRef` - This is formed. Stores the bundle image reference which will be of the format `<registry_repo>/< operatorName>-bundle:v.<bundleVersion>`
- For getting catalog related inputs:
```
type CatalogDInfo struct {
Expand All @@ -64,11 +67,11 @@ The steps in the ginkgo test can be summarized as follows:
fbcFileName string
}
```
- `baseFolderPath` - Base/root path of the folder for the catalogs.
- `baseFolderPath` - Base/root path of the folder that stores the catalogs.
- `operatorName` - Name of the operator to be installed from the bundles.
- `desiredChannelName` - Desired channel name for the operator.
- `catalogDir` - This is formed. The directory to store the FBC. The formed value will be of the format: `<operator-name>-catalog`
- `imageRef` - This is formed. Stores the FBC image reference which will be of the format: `localhost:5001/<username>/<catalogDir>:test`
- `catalogDir` - This is formed. The directory to store the catalog/FBC. The directory name will be of the format: `<operator-name>-catalog`
- `imageRef` - This is formed. Stores the FBC image reference which will be of the format: `<registry_repo>/<catalogDir>:test`
- `fbcFileName` - Name of the FBC file. This is hard-coded as `catalog.yaml`.
- For getting information related to the install/upgrade action for operators:
```
Expand All @@ -80,9 +83,8 @@ The steps in the ginkgo test can be summarized as follows:
- `installVersion` - Version of the operator to be installed on the cluster.
- `upgradeVersion` - Version of the operator to be upgraded on the cluster.

### Plain bundles
- The plain+v0 bundles are formed using `operator-sdk` and `kustomize`.
- The below input is used to form the bundle using operator-sdk.
- The below inputs are used to form the bundle using operator-sdk.

```
type SdkProjectInfo struct {
projectName string
Expand All @@ -92,42 +94,109 @@ The steps in the ginkgo test can be summarized as follows:
kind string
}
```
## How to run
- Makefile target `operator-developer-e2e` : Runs the entire E2E setup.
- Makefile target `test-op-dev-e2e`: Runs the ginkgo test.
- Makefile target `deploy-local-registry`: Deploys local registry
- Makefile target `cleanup-local-registry` : Stops and removes local registry
- Makefile target `kind-cluster-cleanup`: Deletes the kind cluster

## Bundle Types
### Plain bundles
- The `plain+v0` bundles are formed using `operator-sdk` and `kustomize`.
- The `kustomize` organizes the different resources present in the `operator-sdk` project into a single yaml file.
- The Dockerfile for the bundle is named `plainbundle.Dockerfile` and is generated using a custom routine.
- The generated bundle is stored in the format:
```
plain-v0
└── <operatorName>.v<bundleVersion>
└── manifests
│ └── mainfest.yaml
└── plainbundle.Dockerfile
```

- After the bundle image is created and loaded, the FBC is formed by a custom routine by using the operatorName, desiredChannelName, bundle imageRefs and bundleVersions.

- The generated FBC will be present in the file path: `testdata/catalogs/<operatorName>-catalog/catalog.yaml` and the Dockerfile will be formed in the file path: `testdata/catalogs/<operatorName>-catalog.Dockerfile`
- `Default channel` is not used in forming the FBC as it is not an OLMv1 concept.
- The package schema name will be the operatorName and the bundle name will be `<operatorName>.v<version>`.

- The catalog resource is then formed with the name `<operatorName>-catalog`.
- The FBC template is formed by a custom routine by using the operatorName, desiredChannelName, bundle imageRefs and bundleVersions.
- `Default channel` is not used in forming the FBC as it is not an OLMv1 concept.
- Only one `olm.channel` is generated which is the given <desiredChannelName>.
- Upgrade graph is formed with only replaces edge.
- The generated FBC is not validated using `opm` as the tool has no support for plain bundles.
- The Dockerfile for the catalog is named `<operator-name>-catalog.Dockerfile` and is generated using a custom routine.
- The generated catalog is stored in the format:
```
catalogs
└── <operator-name>-catalog
│ └── catalog.yaml
└── <operator-name>-catalog.Dockerfile
```
- The catalog CR is then formed with the name `<operatorName>-catalog`.

- The operator is then installed and has the name `<operatorName>`.

### Registry Bundles

- The registry+v1 bundles are formed using operator-sdk.
- The below input is used to form the bundle using operator-sdk.
- The generated CSV uses the default values.
- The bundle content is formed within the operator-sdk project directory in the folder `bundle`. This is moved to the bundle directory folder.
- The generated Dockerfile has the name `bundle.Dockerfile`. The Dockerfile and bundle structure is genearted by the `operator-sdk` tool.
- The generated bundle is stored in the format:
```
type SdkProjectInfo struct {
projectName string
domainName string
group string
version string
kind string
}
registry-v1
└── <operatorName>.v<bundleVersion>
└── manifests
└── metadata
└── bundle.Dockerfile
```

- The FBC is formed using `opm alpha render-template semver` tool.
- The semver template named `registry-semver.yaml` is formed using a custom routine by passing the bundle imageRefs.
- `generatemajorchannels` and `generateminorchannels` is set to false in the semver template.
- The generated catalog is stored in the format:
```
catalogs
└── <operator-name>-catalog
│ └── catalog.yaml
└── <operator-name>-catalog.Dockerfile
```
- The generated CSV uses the default values.
- The bundle content is formed in the operator-sdk project directory in the folder `bundle`. This is moved to the folder path: `testdata/bundles/registry-v1/<operatorName>.v<version>`.
- The generated Dockerfile has the name `bundle.Dockerfile`.
- The same project directory needs to be updated for forming a different version of the `bundle`.
- After the bundle image is created and loaded, the FBC is formed using `semver` and `opm` tool.
- The semver named `registry-semver.yaml` is formed by passing the bundle imageRefs.
- The generated FBC will be present in the file path: `testdata/catalogs/<operatorName>-catalog/catalog.yaml` and the Dockerfile will be formed in the file path: `testdata/catalogs/<operatorName>-catalog.Dockerfile`
- The package schema name will be the operator-sdk projectName.

- The catalog resource is then formed with the name `<operatorName>-catalog`.

- The operator is then installed and has the name `<operatorName>`.


- After the e2e workflow, all the files formed are cleared and the catalog is deleted.
- After the e2e workflow, all the files formed are cleared.


## To-do
1. The resources are read from input manifests using universal decoder from `k8s.io/apimachinery/pkg/runtime/serializer`.
- However, in cases where a single file contains multiple YAML documents separated by `---,` the `UniversalDecoder` recognizes only the first resource. This situation is relevant as for `plain+v0` bundles generated through `kustomize,` the manifest has multiple YAML documents are combined into one file using --- separators. This is now handled by splitting the content of the YAML file and decoding each of them using the `UniversalDecoder`.
- This workaround can be improved using `YAMLToJSONDecoder` from `k8s.io/apimachinery/pkg/util/yaml`. And the kind, api version and name can be get by decoding into `Unstructured` from `k8s.io/apimachinery/pkg/apis/meta/v1/unstructured`.
2. All the tests pass and the operator is installed successfully. The bundledeployment succeeds and the resources are created. But the pod for the new operator failes due to `ImagePullBackOff`.
- This is because the `Deployment` controller-manager uses the image `controller:latest` which is not present on the cluster.
- The solution would be to replace `controller:latest` with the `busybox:latest` and then pulling and loading `busybox:latest` onto cluster.
- The replacement could possibly be achieved by adding the following to `config/default/kustomization.yaml` under `operator-sdk` project:
```
images:
- name: controller
newName: controller
newTag: latest
```
## Issues
1. This test currently only works with the container runtime `docker`.
- The default value of CONTAINER_RUNTIME defined in the Makefile is `docker`. Therefore the correct runtime has to be assigned to the variable `CONTAINER_RUNTIME` before calling the make target if docker is what is not being used. The test routine also assumes the runtime as `docker` if it is unable to retrieve the value of the environment variable.
- But this is only a partial fix to the problem. With this change the test for `plain+v0` bundles will pass but for `registry+v1` will fail for other container runtimes. This is because `registry+v1` uses `operator-sdk` support. Thus to mimic the user experience, the targets `bundle-build` and `bundle-push` from the generated Makefile by operator-sdk tool, which has docker being hard coded as the container runtime, is used to build and push the bundle images. This could be marked as an issue and addressed when hard coding docker as container runtime in the generated Makefile is addressed by operator-sdk.

2. The `opm`,`operator-sdk` and `kustomize` binaries are added in operator-controller using `bingo`.
- But based on discussions, if required test should be changed so that it has `opm` and `operator-sdk` in `PATH` and simply runs it like `exec.Command("opm", ...)`.
- This will enable in running [a matrix](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs) for the tests and to use different versions of `opm` and `operator-sdk`.
- This might help in emulating the user experience better.

## Tooling gaps

Following are the tooling gaps identified while testing `operator-framework` end-to-end:
- `opm` doesn't have plain bundle support.
- No tool for forming FBC for plain bundles.
- No tool for generating Dockerfile for plain bundles.
- No tool for generating Dockerfile for plain catalogs.
- Since `opm` doesn't have plain bundle support, there is no means to validate the FBC generated for plain bundles.

20 changes: 9 additions & 11 deletions test/operator-framework-e2e/create_fbc_helper.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,18 @@ import (
)

const (
SchemaPackage = "olm.package"
SchemaChannel = "olm.channel"
SchemaBundle = "olm.bundle"
SchemaBundleMediatype = "olm.bundle.mediatype"
BundleMediatype = "plain+v0"
)

// Forms the FBC declartive config and creates the FBC by calling functions for forming the package, channel and bundles.
func CreateFBC(operatorName, defaultChannel string, bundleRefsVersions map[string]string) *declcfg.DeclarativeConfig {
func CreateFBC(operatorName, channelName string, bundleRefsVersions map[string]string) *declcfg.DeclarativeConfig {
dPackage := formPackage(operatorName)
bundleVersions := make([]string, 0)
for _, bundleVersion := range bundleRefsVersions {
bundleVersions = append(bundleVersions, bundleVersion)
}
dChannel := formChannel(operatorName, defaultChannel, bundleVersions)
dChannel := formChannel(operatorName, channelName, bundleVersions)

dBundle := formBundle(operatorName, bundleRefsVersions)

Expand All @@ -42,7 +39,7 @@ func CreateFBC(operatorName, defaultChannel string, bundleRefsVersions map[strin
// Forms package schema for the FBC
func formPackage(pkgName string) declcfg.Package {
packageFormed := declcfg.Package{
Schema: SchemaPackage,
Schema: declcfg.SchemaPackage,
Name: pkgName,
}
return packageFormed
Expand All @@ -52,15 +49,16 @@ func formPackage(pkgName string) declcfg.Package {
func formChannel(pkgName, channelName string, bundleVersions []string) declcfg.Channel {
channelEntries := formChannelEntries(pkgName, bundleVersions)
channelFormed := declcfg.Channel{
Schema: SchemaChannel,
Schema: declcfg.SchemaChannel,
Name: channelName,
Package: pkgName,
Entries: channelEntries,
}
return channelFormed
}

// Forms the uprade graph for the FBC
// Forms the uprade graph for the FBC. Only forms replaces edge. For forming replaces edge,
// bundleVersions are assumed to be in increasing version number order.
func formChannelEntries(pkgName string, bundleVersions []string) []declcfg.ChannelEntry {
channelEntries := make([]declcfg.ChannelEntry, 0, len(bundleVersions))
for i, version := range bundleVersions {
Expand All @@ -83,7 +81,7 @@ func formBundle(pkgName string, imgRefsVersions map[string]string) []declcfg.Bun
for imgRef, version := range imgRefsVersions {
var properties []property.Property
properties = append(properties, property.Property{
Type: SchemaPackage,
Type: declcfg.SchemaPackage,
Value: json.RawMessage(fmt.Sprintf(`{"packageName": "%s", "version": "%s"}`, pkgName, version)),
})
properties = append(properties, property.Property{
Expand All @@ -92,7 +90,7 @@ func formBundle(pkgName string, imgRefsVersions map[string]string) []declcfg.Bun
})

bundleFormed = append(bundleFormed, declcfg.Bundle{
Schema: SchemaBundle,
Schema: declcfg.SchemaBundle,
Name: pkgName + ".v" + version,
Package: pkgName,
Image: imgRef,
Expand All @@ -102,7 +100,7 @@ func formBundle(pkgName string, imgRefsVersions map[string]string) []declcfg.Bun
return bundleFormed
}

// Writes the formed FBC into catalog.yaml file
// Writes the formed FBC into catalog.yaml file in the path fbcFilePath
func WriteFBC(fbc declcfg.DeclarativeConfig, fbcFilePath, fbcFileName string) error {
var buf bytes.Buffer
if err := declcfg.WriteYAML(fbc, &buf); err != nil {
Expand Down
14 changes: 7 additions & 7 deletions test/operator-framework-e2e/generate_dockerfile.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"text/template"
)

// GenerateDockerFile generates Dockerfile and its contents for a given set of yaml files
// GenerateDockerFile generates Dockerfile and its contents for the data in yamlFolderName
func generateDockerFile(dockerFilePath, yamlFolderName, dockerfileTmpl string) error {
t, err := template.New("dockerfile").Parse(dockerfileTmpl)
if err != nil {
Expand All @@ -26,14 +26,14 @@ func generateDockerFile(dockerFilePath, yamlFolderName, dockerfileTmpl string) e
return err
}

// GenerateCatalogDockerFile generates Dockerfile for the catalog
func generateCatalogDockerFile(dockerFilePath, yamlFolderName string) error {
return generateDockerFile(dockerFilePath, yamlFolderName, catalogDockerfileTmpl)
// GenerateCatalogDockerFile generates Dockerfile for the catalog content in catalogFolderName
func generateCatalogDockerFile(dockerFilePath, catalogFolderName string) error {
return generateDockerFile(dockerFilePath, catalogFolderName, catalogDockerfileTmpl)
}

// GenerateBundleDockerFile generates Dockerfile for the bundle
func generateBundleDockerFile(dockerFilePath, yamlFolderName string) error {
return generateDockerFile(dockerFilePath, yamlFolderName, bundleDockerfileTmpl)
// GenerateBundleDockerFile generates Dockerfile for the bundle content in bundleFolderName
func generateBundleDockerFile(dockerFilePath, bundleFolderName string) error {
return generateDockerFile(dockerFilePath, bundleFolderName, bundleDockerfileTmpl)
}

// Dockerfile templates
Expand Down
Loading

0 comments on commit 321525f

Please sign in to comment.