Cross-component E2E for operator framework #286

jubittajohn · 2023-07-05T15:10:44Z

Description

This PR is for a cross-component demo with all OLM v1 repositories. The ginkgo test does the following:

Uses operator-sdk and kustomize to build plain+v0 bundles and create catalogs to include the bundles.
Installs, upgrades and deletes a plain+v0 operator.
Uses operator-sdk to build registry+v1 bundles and create catalogs to include the bundles.
Installs, upgrades and deletes a registry+v1 operator.

Purpose of the E2E tests.

Development on OLM v1 is split across multiple repositories, and the list of relevant repositories may grow over time. While we have demos showing improvements in functionality of components over time, it can be difficult to have a picture of the state of OLM v1 at any given time for someone not following its development closely. Having a single source to look for OLM v1 behavior can provide more clarity about the state of the project.
With the scale of the OLM v1 project, it is useful to have a means to test components in the operator development + lifecycle pipeline together to create a more cohesive experience for all users.
Test workflows for common personas as per user documentation to guarantee the accuracy of documented steps.

Workflow

start with an empty directory
call operator-sdk to initialize and generate an operator
generate a bundle directory
build/push/kind load bundle images from the bundle directories
repeat steps 2-4 as necessary to get bundles for multiple operator versions
generate a catalog directory
build/push/kind load the catalog
create a Catalog CR (with kubectl operator)
create an Operator CR (with kubectl operator)
trigger Operator upgrades (with kubectl operator)
delete the Operator CR (with kubectl operator)
delete the Catalog CR (with kubectl operator)
repeat steps 2-12 for each bundle format (e.g. registry+v1 and plain+v0)

Checks to be ensured:
- Ensuring packages and bundlemetadata are created for the catalog.
- Ensuring operator operations succeed.
- Verifying presence/absence of relevant manifests.
Added new Makefile target/Github action:
- makefile target: operator-developer-e2e
- Github action: /workflows/operator-developer-e2e.yaml
- Note: The current test-e2e are only for testing the functionality of operator-controller. The new e2e test is meant for an operator-framework e2e or an OLM v1 e2e and to capture the developer experience.

How to run

Makefile target operator-developer-e2e : Runs the entire E2E setup.
Makefile target test-op-dev-e2e: Runs the ginkgo test.
Makefile target deploy-local-registry: Deploys local registry
Makefile target cleanup-local-registry : Stops and removes local registry
Makefile target kind-cluster-cleanup: Deletes the kind cluster

Getting Started

Below are the input values used in the test.

Building operator-controller, deploying it into the cluster and rest of the configuration is done in the MakeFile of this repo under the target operator-developer-e2e. This includes:
- Setting up a kind cluster named operator-controller-op-dev-e2e.
- Installing the operator controller onto the cluster.
- Setting up opm, operator-sdk and kustomize using bingo.
- Setting up a local registry server for building and loading images.
The following structs defined are required as input for both plain+v0 and registry+v1 bundles:
- For getting bundle related inputs:
```
type BundleInfo struct {
    baseFolderPath string 
    bundles        []BundleContent
}

type BundleContent struct {
    bInputDir     string
    bundleVersion string
    imageRef      string
}
```
  - baseFolderPath - Base/root path of the folder for the specific bundle type input data.[path to plain-v0 or registry-v1 bundles testdata]
  - bundles - Stores the data relevant to different versions of the bundle.
  - bInputDir - The input directory containing the specific version of the bundle data.
  - bundleVersion - The specific version of the bundle data.
  - imageRef - This is formed. Stores the bundle image reference which will be of the format localhost:5001/<operator_name>-bundle:v.<bundleVersion>
- For getting catalog related inputs:
```
type CatalogDInfo struct {
    baseFolderPath     string
    catalogDir         string
    operatorName       string
    desiredChannelName string
    imageRef           string
    fbcFileName        string
}
```
  - baseFolderPath - Base/root path of the folder for the catalogs.
  - operatorName - Name of the operator to be installed from the bundles.
  - desiredChannelName - Desired channel name for the operator.
  - catalogDir - This is formed. The directory to store the FBC. The formed value will be of the format: <operator-name>-catalog
  - imageRef - This is formed. Stores the FBC image reference which will be of the format: localhost:5001/<username>/<catalogDir>:test
  - fbcFileName - Name of the FBC file. This is hard-coded as catalog.yaml.
For getting information related to the install/upgrade action for operators:
type OperatorActionInfo struct { installVersion string upgradeVersion string }
- installVersion - Version of the operator to be installed on the cluster.
- upgradeVersion - Version of the operator to be upgraded on the cluster.
- The below input is used to form the bundle using operator-sdk.
```
type SdkProjectInfo struct {
    projectName string
    domainName  string
    group       string
    version     string
    kind        string
}
```

Reviewer Checklist

API Go Documentation
Tests: Unit Tests (and E2E Tests, if appropriate)
Comprehensive Commit Messages
Links to related GitHub Issue(s)

codecov · 2023-07-05T15:14:38Z

Codecov Report

Merging #286 (321525f) into main (28da6bd) will not change coverage.
The diff coverage is n/a.

❗ Current head 321525f differs from pull request most recent head 88776d0. Consider uploading reports for the commit 88776d0 to get more accurate results

@@           Coverage Diff           @@
##             main     #286   +/-   ##
=======================================
  Coverage   82.84%   82.84%           
=======================================
  Files          21       21           
  Lines         892      892           
=======================================
  Hits          739      739           
  Misses        107      107           
  Partials       46       46

Flag	Coverage Δ
e2e	`62.66% <ø> (ø)`
unit	`78.59% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Makefile

test/operator-framework-e2e/README.md

dtfranz · 2023-07-28T22:00:22Z

test/operator-framework-e2e/operator_framework_test.go

+
+			By("Verifying the operator doesn't exist")
+			Eventually(func(g Gomega) {
+				err = checkOperatorDeleted(catalogDInfo.operatorName)


Given the func name checkOperatorDeleted(...), it might make more sense for it to return a bool and do something like return err != nil && errors.IsNotFound(err) inside of the func, which would also simplify the checks here a bit.

Err is returned, so that the reason for the failure of the test case would be shown if the assertion fails. This is the case for the other functions starting with check as well. So wouldn't it be better to change the name to validateOperatorDeletion instead of returning a bool ?

test/operator-framework-e2e/operator_framework_test.go

joelanford · 2023-08-03T20:03:55Z

test/operator-framework-e2e/operator_framework_test.go

+		catalogDInfo.catalogDir = catalogDInfo.operatorName + "-catalog"
+		catalogDInfo.imageRef = remoteRegistryRepo + catalogDInfo.catalogDir + ":test"
+	})
+	When("Build and load plain+v0 bundle images into the test environment", func() {


Rather than directly committing plain bundle manifests, it should be possible to generate the manifests from an SDK project with kustomize build config/default > ${bundleRoot}/manifests/manifest.yaml

joelanford · 2023-08-03T20:26:20Z

Firstly, this looks great! It's awesome to have that full coast-to-coast test that starts from scratch that we've been talking about.

I looks like we cover the following steps (correct me if I'm wrong):

start with an empty directory
call operator-sdk to initialize and generate an operator
generate a bundle directory
build/push/kind load bundle images from the bundle directories
repeat steps 2-4 as necessary to get bundles for multiple operator versions
generate a catalog directory
build/push/kind load the catalog
create a Catalog CR (with kubectl operator)
create an Operator CR (with kubectl operator)
trigger Operator upgrades (with kubectl operator)
delete the Operator CR (with kubectl operator)
delete the Catalog CR (with kubectl operator)
repeat steps 2-12 for each bundle format (e.g. registry+v1 and plain+v0)

What I'm really interested in is which of these steps is more difficult than one or two shell commands, because that is where I think we should focus our efforts on UX to make things easier for our users. It might be nice to organize the test such that each of these steps is a function. If the function calls a single shell command that calls one of our tools, that's the ideal. If the function is a bunch of go code, that likely means that we have a tooling gap.

m1kola

I think this is super useful. I left a few comments/suggestions.

Most of them are minor. Major themes are:

I didn't manage to run it locally: macos uses port 5000 + something is up with pushing images to the local registry
If I understnad correctly, It specs depend on each other and it might be an issue when ran out of expected order.
There is quite a lot of code duplication in test/operator-framework-e2e/operator_framework_test.go. I'm wondering if we can reduce it.

m1kola · 2023-08-07T08:53:04Z

Makefile

+opm: $(OPM)
+	$(OPM) $(OPM_ARGS)
+
+operator-sdk: $(OPERATOR_SDK)
+	(cd $(OPERATOR_SDK_PROJECT_PATH) && $(OPERATOR_SDK) $(OPERATOR_SDK_ARGS))


Unless I'm missing something - we do not use these two targets.

Ah... I see - we use them in tests. This is not obvous and I think deserves a comment.

However in my opinion E2E tests should not be calling make to install opm and operator-sdk binaries. I think it is not critical for now, but we need to create an issue for this (unless there are objections to the idea below).

I think we need to change tests so that it:

Either has opm and operator-sdk in PATH and simply runs it like exec.Command("opm", ...)

Or receives a path to opm and friends via arguments/env vars or somehow else.

This way we can run a matrix for our tests and use different versions of opm and operator-sdk.

Also it will make test code a bit cleaner (but would require some more test setup code instead).

Makefile

test/operator-framework-e2e/operator_framework_test.go

Makefile

m1kola · 2023-08-07T13:07:32Z

test/operator-framework-e2e/read_manifests.go

+type Object struct {
+	Kind       string `yaml:"kind"`
+	APIVersion string `yaml:"apiVersion"`
+	Metadata   struct {
+		Name string `yaml:"name"`
+	} `yaml:"metadata"`
+}


I think you are looking for something like k8s.io/apimachinery/pkg/runtime/serializer. Here is an example usage in our project:

operator-controller/cmd/resolutioncli/input_manifests.go

Lines 44 to 45 in fa84c73

decoder := codecs.UniversalDecoder(scheme.PrioritizedVersionsAllGroups()...)

object, _, err := decoder.Decode(fileContent, nil, nil)

Updated the code using the universal decoder. However, in cases where a single file contains multiple YAML documents separated by ---, the universal decoder recognizes only the first resource. This situation is relevant as for plain+v0 bundle generated through kustomize, the manifest has multiple YAML documents are combined into one file using --- separators. Is there a decoder solution that can effectively handle scenarios involving multiple YAML documents within a single file? This is now handled by splitting the content of the YAML file and decoding each of them using the universal decoder.

Good point!

I think YAMLToJSONDecoder from k8s.io/apimachinery/pkg/util/yaml should do the trick.

I think you should be able to get kind, api version and name if you decode into Unstructured from k8s.io/apimachinery/pkg/apis/meta/v1/unstructured.

m1kola · 2023-08-07T13:14:48Z

test/operator-framework-e2e/operator_framework_test.go

+	cmd.Stderr = w
+	cmd.Stdout = w
+	if err := cmd.Run(); err != nil {
+		return fmt.Errorf("Error pushing Docker container image: %s to the registry: %v", tag, err)


I'm gettign this error:

Error pushing Docker container image: localhost:5001/plain-catalog:test to the registry: exit status 125

I use podman with an alias from docker to podman.

I changed the local port from 5000 to 5001 due to issue with control center on macos mentioned above, but still fails.

The test ran successfully on my macos. So I guess I am missing out something here. A local registry was used over the container registry quay.io for improved performance. What could be an ideal solution for the issue with the port?

I'm on version Ventura 13.4.1 right now. It looks like it started listening on port 5000 starting from Monterey.

What could be an ideal solution for the issue with the port?

I would change the localhost port to something else other than 5000. For example, kind suggests using port 5001. And it looks like kind changed from 5000 to 5001 precisely due to this reason:

Local registry example uses port 5001 to be more mac friendly kubernetes-sigs/kind#2621

Prow integration test: mac friendly by using port 5001 kubernetes/test-infra#25134

Updated the port to use 5001 on the host. But the port 5001 on the host maps to port 5000 on the container. This is because, the docker container uses the image registry:2 which listens on port 5000. This works for me. I am on version Ventura 13.4. Could you please confirm if the error is now resolved.

But the port 5001 on the host maps to port 5000 on the container.

@jubittajohn that's good. This is what we want. I just quickly chekced - registry is running fine now, but I'm still getting an error on an attempt to push the image:

[FAILED] in [It] - /Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/operator_framework_test.go:318 @ 08/09/23 10:13:38.056 << Timeline [FAILED] Unexpected error: <*errors.errorString | 0x1400014f4f0>: Error pushing bundle image localhost:5001/registry-operator-bundle:v0.1.0 with tag 0.1.0 : make[1]: Entering directory '/Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/registry-operator' make docker-push IMG=localhost:5001/registry-operator-bundle:v0.1.0 make[2]: Entering directory '/Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/registry-operator' docker push localhost:5001/registry-operator-bundle:v0.1.0 Getting image source signatures Copying blob sha256:dea1f3bba92bc840b14e6ae4e75f6d1538be9c7d11b13e27dadb691cd16c6a04 Copying blob sha256:b6d433180825c5881b9565b9cbff353bace1926fe429c2cc4c67b75b203c8706 Copying blob sha256:7e13d0a7484e3297efc0eb92610081b75a115f7a24b2ffde5e25bde7f6619634 Error: trying to reuse blob sha256:b6d433180825c5881b9565b9cbff353bace1926fe429c2cc4c67b75b203c8706 at destination: pinging container registry localhost:5001: Get "https://localhost:5001/v2/": http: server gave HTTP response to HTTPS client make[2]: *** [Makefile:131: docker-push] Error 125 make[2]: Leaving directory '/Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/registry-operator' make[1]: *** [Makefile:240: bundle-push] Error 2 make[1]: Leaving directory '/Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/registry-operator' : exit status 2 { s: "Error pushing bundle image localhost:5001/registry-operator-bundle:v0.1.0 with tag 0.1.0 : make[1]: Entering directory '/Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/registry-operator'\nmake docker-push IMG=localhost:5001/registry-operator-bundle:v0.1.0\nmake[2]: Entering directory '/Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/registry-operator'\ndocker push localhost:5001/registry-operator-bundle:v0.1.0\nGetting image source signatures\nCopying blob sha256:dea1f3bba92bc840b14e6ae4e75f6d1538be9c7d11b13e27dadb691cd16c6a04\nCopying blob sha256:b6d433180825c5881b9565b9cbff353bace1926fe429c2cc4c67b75b203c8706\nCopying blob sha256:7e13d0a7484e3297efc0eb92610081b75a115f7a24b2ffde5e25bde7f6619634\nError: trying to reuse blob sha256:b6d433180825c5881b9565b9cbff353bace1926fe429c2cc4c67b75b203c8706 at destination: pinging container registry localhost:5001: Get \"https://localhost:5001/v2/\": http: server gave HTTP response to HTTPS client\nmake[2]: *** [Makefile:131: docker-push] Error 125\nmake[2]: Leaving directory '/Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/registry-operator'\nmake[1]: *** [Makefile:240: bundle-push] Error 2\nmake[1]: Leaving directory '/Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/registry-operator'\n: exit status 2", } occurred In [It] at: /Users/mradchuk/workspace/src/github.com/operator-framework/operator-controller/test/operator-framework-e2e/operator_framework_test.go:318 @ 08/09/23 10:13:38.056

@m1kola
The issue is with using podman.

The error in pushing was due to docker being hard coded as the container runtime in the test. This is now changed to get the current container runtime using the environment variable CONTAINER_RUNTIME. The default value of CONTAINER_RUNTIME defined in the Makefile is docker. Therefore the correct runtime has to be assigned to the variable CONTAINER_RUNTIME before calling the make target if docker is what is not being used. The test routine also assumes the runtime as docker if it is unable to retrieve the value of the environment variable.

But this is only a partial fix to the problem. With this change the test for plain+v0 bundles will pass but for registry+v1 will fail. This is because registry+v1 uses operator-sdk support. Thus to mimic the user experience, the targets bundle-build and bundle-push from the generated Makefile by operator-sdk tool, which has docker being hard coded as the container runtime, is used to build and push the bundle images. This could be marked as an issue and addressed when hard coding docker as container runtime in the generated Makefile is addressed by operator-sdk.

In conclusion, the current test will only work for docker.

I have a symlink from docker to podman and Podman's CLI is compatible with docker. So make operator-developer-e2e CONTAINER_RUNTIME=podman and make operator-developer-e2e give me the same results.

It has to be something else.

I don't know how often (if at all) we will be running this test locally. It might be not worth spinning wheels on this (at least in context of this PR).

test/operator-framework-e2e/operator_framework_test.go

tmshort · 2023-08-09T16:20:55Z

I tried running this on my Ventura Mac (M1 Pro):

With docker installed via brew install docker --cask, I was able to get the tests ran, but there were timeout and others errors:

Summarizing 2 Failures:
  [FAIL] Operator Framework E2E for registry+v1 bundles [It] should succeed
  /Users/tshort/git/operator-controller/test/operator-framework-e2e/operator_framework_test.go:846
  [FAIL] Operator Framework E2E for plain+v0 bundles [It] should succeed
  /Users/tshort/git/operator-controller/test/operator-framework-e2e/operator_framework_test.go:840

Ran 2 of 2 Specs in 309.603 seconds
FAIL! -- 0 Passed | 2 Failed | 0 Pending | 0 Skipped
--- FAIL: TestOperatorFramework (309.60s)

I couldn't get docker working via brew install docker docker-machine virtualbox, as it requires x86_64 (qemu would probably work, but I didn't try).
With podman installed via brew install podman; it fails not finding the docker command. After creating a docker symlink to podman, it had the same behavior as docker, the tests ran, and encountered the same error.
(This was after a bunch of cleanup of my podman/docker config.)

I tried running this on my F39 (x86_64):

I have docker installed on here, because I was running into podman issues. Ran into the same errors.

tmshort · 2023-08-09T19:49:47Z

Makefile

+.PHONY: stop-local-registry
+stop-local-registry: ## Stop local registry
+	$(CONTAINER_RUNTIME) container stop local-registry
+
+.PHONY: remove-local-registry
+remove-local-registry: ## Remove local registry
+	$(CONTAINER_RUNTIME) container rm -v local-registry


Having these as separate targets means two things:
They aren't cleaned up when an error occurs, which is good if you want to look at the status of an error, but bad if you want to run the tests again. You might want to force stop in the deploy-local-registry. Or possibly combine these into a single "cleanup" target to make it easier to clean up after failure,

Added cleanup-local-registry which combines both the targets mentioned above. Need to look into force stop in deploy-local-registry if that is required to be addressed.

jubittajohn · 2023-08-10T05:33:13Z

I tried running this on my Ventura Mac (M1 Pro):

With docker installed via brew install docker --cask, I was able to get the tests ran, but there were timeout and others errors:
Summarizing 2 Failures:
  [FAIL] Operator Framework E2E for registry+v1 bundles [It] should succeed
  /Users/tshort/git/operator-controller/test/operator-framework-e2e/operator_framework_test.go:846
  [FAIL] Operator Framework E2E for plain+v0 bundles [It] should succeed
  /Users/tshort/git/operator-controller/test/operator-framework-e2e/operator_framework_test.go:840

Ran 2 of 2 Specs in 309.603 seconds
FAIL! -- 0 Passed | 2 Failed | 0 Pending | 0 Skipped
--- FAIL: TestOperatorFramework (309.60s)
I couldn't get docker working via brew install docker docker-machine virtualbox, as it requires x86_64 (qemu would probably work, but I didn't try).

With podman installed via brew install podman; it fails not finding the docker command. After creating a docker symlink to podman, it had the same behavior as docker, the tests ran, and encountered the same error.
(This was after a bunch of cleanup of my podman/docker config.)

I tried running this on my F39 (x86_64):

I have docker installed on here, because I was running into podman issues. Ran into the same errors.

The podman issue is due to reasons specified in the comment. The test case failure is now fixed.

m1kola

CI if failing at the moment:

Status conditions for the operator registry-operator for the version 0.1.1 is not as expected:Expected status: True, but got: Unknown

Once this failure is deailt with - I think we will be in a good place to get it merged. After that we will be able to iterate in smaller PRs.

I wasn't able to run this locally on ARM mac with podman, but I'm not sure whether we want to hold this PR because of this.

test/operator-framework-e2e/operator_framework_test.go

Signed-off-by: jubittajohn <jujohn@redhat.com>

…ADME Signed-off-by: jubittajohn <jujohn@redhat.com>

Signed-off-by: jubittajohn <jujohn@redhat.com> Refractored the plain bundle and registry bundle interactions; Updated the comments and README Signed-off-by: jubittajohn <jujohn@redhat.com> Added make target to install operator-sdk; Added make target to deploy a local registry server; Changed test to use the local registry server Signed-off-by: jubittajohn <jujohn@redhat.com>

…s to be consistent in styling Signed-off-by: jubittajohn <jujohn@redhat.com>

…e;Restructured code to have a single It block in a describe block; Signed-off-by: jubittajohn <jujohn@redhat.com> Combined local-registry cleanup make target; Added support for container runtime in consideration Signed-off-by: jubittajohn <jujohn@redhat.com> Bumped the upgrade version from 0.1.1 to 0.2.0 Signed-off-by: jubittajohn <jujohn@redhat.com> Updated comments and README Signed-off-by: jubittajohn <jujohn@redhat.com>

m1kola

Great work! I think we are in the state where we can merge and we will be able to iterate on top of what you built (e.g. fixing issues with local podman on mac os, etc).

Thank you for all your contributions, best of luck with your studies and I hope to see you around again at some point :)

dtfranz · 2023-08-11T16:53:34Z

/lgtm
Great job Jubitta! We really appreciate your work here!

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 5, 2023

jubittajohn requested review from m1kola, dtfranz and ankitathomas July 5, 2023 16:41

openshift-merge-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jul 7, 2023

jubittajohn force-pushed the operator-framework-e2e branch 6 times, most recently from 6a5266d to d57d09e Compare July 27, 2023 15:13

jubittajohn marked this pull request as ready for review July 27, 2023 17:23

jubittajohn requested a review from a team as a code owner July 27, 2023 17:23

jubittajohn force-pushed the operator-framework-e2e branch 7 times, most recently from b39378b to 60dde4f Compare July 27, 2023 20:05

dtfranz reviewed Jul 28, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

dtfranz reviewed Jul 28, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

jubittajohn changed the title ~~WIP: Cross-component E2E for operator framework~~ Cross-component E2E for operator framework Jul 28, 2023

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 28, 2023

dtfranz reviewed Jul 28, 2023

View reviewed changes

test/operator-framework-e2e/README.md Show resolved Hide resolved

dtfranz reviewed Jul 28, 2023

View reviewed changes

test/operator-framework-e2e/operator_framework_test.go Outdated Show resolved Hide resolved

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 29, 2023

jubittajohn requested a review from dtfranz August 3, 2023 18:59

joelanford reviewed Aug 3, 2023

View reviewed changes

jubittajohn force-pushed the operator-framework-e2e branch from a84335f to 87fa9b5 Compare August 4, 2023 19:25

jubittajohn requested a review from tmshort August 4, 2023 19:33

m1kola mentioned this pull request Aug 7, 2023

Create a package for scheme setup #322

Closed

m1kola reviewed Aug 7, 2023

View reviewed changes

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 8, 2023

jubittajohn force-pushed the operator-framework-e2e branch from bba9a01 to f3fce19 Compare August 9, 2023 03:35

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 9, 2023

jubittajohn force-pushed the operator-framework-e2e branch from 6785df4 to 8011376 Compare August 9, 2023 06:54

jubittajohn requested a review from m1kola August 9, 2023 07:14

tmshort reviewed Aug 9, 2023

View reviewed changes

m1kola reviewed Aug 10, 2023

View reviewed changes

test/operator-framework-e2e/operator_framework_test.go Outdated Show resolved Hide resolved

jubittajohn added 5 commits August 10, 2023 14:30

Added e2e for plain+v0 operator install, upgrade and delete

c233618

Signed-off-by: jubittajohn <jujohn@redhat.com>

Added tests to automate bundle and FBC image creation for plain bundles

fa139cf

Signed-off-by: jubittajohn <jujohn@redhat.com>

Added error messages for each failed assertion; Added comments and RE…

737c0de

…ADME Signed-off-by: jubittajohn <jujohn@redhat.com>

Modified adding opm and operator-sdk binary using bingo; Updated file…

85f1a24

…s to be consistent in styling Signed-off-by: jubittajohn <jujohn@redhat.com>

jubittajohn force-pushed the operator-framework-e2e branch from 0d30cd3 to 321525f Compare August 11, 2023 07:02

jubittajohn force-pushed the operator-framework-e2e branch from 321525f to 88776d0 Compare August 11, 2023 07:14

m1kola approved these changes Aug 11, 2023

View reviewed changes

openshift-ci bot assigned dtfranz Aug 11, 2023

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 11, 2023

dtfranz approved these changes Aug 11, 2023

View reviewed changes

dtfranz merged commit 4eba8f3 into operator-framework:main Aug 11, 2023
10 of 11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cross-component E2E for operator framework #286

Cross-component E2E for operator framework #286

jubittajohn commented Jul 5, 2023 •

edited

Loading

codecov bot commented Jul 5, 2023 •

edited

Loading

dtfranz Jul 28, 2023

jubittajohn Jul 31, 2023

joelanford Aug 3, 2023

joelanford commented Aug 3, 2023

m1kola left a comment

m1kola Aug 7, 2023

m1kola Aug 7, 2023

m1kola Aug 7, 2023

jubittajohn Aug 9, 2023 •

edited

Loading

m1kola Aug 10, 2023

m1kola Aug 7, 2023

jubittajohn Aug 7, 2023

m1kola Aug 7, 2023

jubittajohn Aug 9, 2023

m1kola Aug 9, 2023

jubittajohn Aug 10, 2023 •

edited

Loading

m1kola Aug 10, 2023

tmshort commented Aug 9, 2023 •

edited

Loading

tmshort Aug 9, 2023

jubittajohn Aug 10, 2023

jubittajohn commented Aug 10, 2023 •

edited

Loading

m1kola left a comment

m1kola left a comment

dtfranz commented Aug 11, 2023

	decoder := codecs.UniversalDecoder(scheme.PrioritizedVersionsAllGroups()...)
	object, _, err := decoder.Decode(fileContent, nil, nil)

Cross-component E2E for operator framework #286

Cross-component E2E for operator framework #286

Conversation

jubittajohn commented Jul 5, 2023 • edited Loading

Description

Purpose of the E2E tests.

Workflow

How to run

Getting Started

Reviewer Checklist

codecov bot commented Jul 5, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joelanford commented Aug 3, 2023

m1kola left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jubittajohn Aug 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jubittajohn Aug 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tmshort commented Aug 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jubittajohn commented Aug 10, 2023 • edited Loading

m1kola left a comment

Choose a reason for hiding this comment

m1kola left a comment

Choose a reason for hiding this comment

dtfranz commented Aug 11, 2023

jubittajohn commented Jul 5, 2023 •

edited

Loading

codecov bot commented Jul 5, 2023 •

edited

Loading

jubittajohn Aug 9, 2023 •

edited

Loading

jubittajohn Aug 10, 2023 •

edited

Loading

tmshort commented Aug 9, 2023 •

edited

Loading

jubittajohn commented Aug 10, 2023 •

edited

Loading