Extend E2E test to include coverage for creating Operator from OperatorSource #135

ldimaggi · 2019-04-22T18:16:24Z

This pull request will include:

OperatorSource definition file extend E2E test to include verifying creating Operator from OperatorSource as defined here: https://github.com/operator-framework/operator-marketplace
Updates to the E2E test

Ref: https://jira.coreos.com/browse/ODC-580

openshift-ci-robot · 2019-04-22T18:16:38Z

Hi @ldimaggi. Thanks for your PR.

I'm waiting for a redhat-developer member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

sbose78 · 2019-04-22T19:00:26Z

test/e2e/devconsole.operatorsource.cr.yaml

+  namespace: openshift-operators
+spec:
+  type: appregistry
+  endpoint: https://quay.io/repository/redhat-developer/devconsole-operator


Does this actually work like an app registry?

It allowed the subscription to be created - but you are correct - it's not an app registry. Will retry after having created an app registry in Quay.

sbose78 · 2019-04-24T20:30:03Z

/ok-to-test

sbose78 · 2019-04-24T20:30:43Z

test/e2e/devconsole.operatorsource.cr.yaml

+  namespace: openshift-operators
+spec:
+  type: appregistry
+  endpoint: https://quay.io/application/ldimaggi-test-org/mychart


Nice! Do paste a screenshot in this PR how it looks in the operatorhub page on the cluster.

I spent a lot of time with Helm this morning and I think I have the correct sequence - but while the subscription is showing up inthe UI, the catalog is not. The problem that I am seeing is: operator-framework/operator-lifecycle-manager#822 - currently blocked on this.

sbose78 · 2019-04-25T00:15:25Z

What are the oc commands you've tried to debug?

ldimaggi · 2019-04-25T02:52:49Z

What are the oc commands you've tried to debug?

kubectl apply -f devconsole.operatorsource.catalog.yaml

The odd thing is that the catalog seems to be created - but it is not showing up in the UI:

kubectl get CatalogSourceConfig devconsole-operator-catalog -n openshift-operators -o=yaml
apiVersion: operators.coreos.com/v1
kind: CatalogSourceConfig
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"operators.coreos.com/v1","kind":"CatalogSourceConfig","metadata":{"annotations":{},"name":"devconsole-operator-catalog","namespace":"openshift-operators"},"spec":{"csDisplayName":"Red Hat Devconsole Operator","csPublisher":"Red Hat","packages":"devconsole-operator","targetNamespace":"local-operators"}}
  creationTimestamp: 2019-04-24T17:46:04Z
  generation: 1
  name: devconsole-operator-catalog
  namespace: openshift-operators
  resourceVersion: "60420"
  selfLink: /apis/operators.coreos.com/v1/namespaces/openshift-operators/catalogsourceconfigs/devconsole-operator-catalog
  uid: d4d2d887-66b8-11e9-a787-0a44d6fe60be
spec:
  csDisplayName: Red Hat Devconsole Operator
  csPublisher: Red Hat
  packages: devconsole-operator
  targetNamespace: local-operators

pmacik · 2019-04-25T15:31:28Z

@ldimaggi The resource that is being created is CatalogSourceConfig with a name of devconsole-operator-catalog - here.
But in the Subscription where you try to reference it by spec.source: devconsole-operator-catalog - here - what the subscription actually references is a CatalogSource instance (and that does not exist), not CatalogSourceConfig.

@sbose78 WDYT, do I understand it correctly?

ldimaggi · 2019-04-25T17:21:19Z

@ldimaggi The resource that is being created is CatalogSourceConfig with a name of devconsole-operator-catalog - here.
But in the Subscription where you try to reference it by spec.source: devconsole-operator-catalog - here - what the subscription actually references is a CatalogSource instance (and that does not exist), not CatalogSourceConfig.

@sbose78 WDYT, do I understand it correctly?

I modeled this on these files:

https://github.com/redhat-developer/devconsole-operator/blob/master/test/e2e/catalog_source_OS4.yaml

https://github.com/redhat-developer/devconsole-operator/blob/master/test/e2e/subscription_OS4.yaml

ldimaggi · 2019-04-25T17:36:08Z

It seems like this is a contributing factor to the subscription failing:

Create a CatalogSource:

cat devconsole.operatorsource.catalog.yamlapiVersion: operators.coreos.com/v1
kind: CatalogSourceConfig
metadata:
  name: devconsole-operator-catalog
  namespace: openshift-operators
spec:
  targetNamespace: local-operators
  packages: devconsole-operator
  displayName: Red Hat Devconsole Operator
  publisher: Red Hat

Apply the CatalogSource:

kubectl apply -f devconsole.operatorsource.catalog.yamlcatalogsourceconfig "devconsole-operator-catalog" configured

Verify the CatalogSource in the CLI:

kubectl get CatalogSourceConfig devconsole-operator-catalog -n openshift-operators -o=yaml
apiVersion: operators.coreos.com/v1
kind: CatalogSourceConfig
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"operators.coreos.com/v1","kind":"CatalogSourceConfig","metadata":{"annotations":{},"name":"devconsole-operator-catalog","namespace":"openshift-operators"},"spec":{"displayName":"Red Hat Devconsole Operator","packages":"devconsole-operator","publisher":"Red Hat","targetNamespace":"local-operators"}}
  creationTimestamp: 2019-04-25T17:19:11Z
  generation: 1
  name: devconsole-operator-catalog
  namespace: openshift-operators
  resourceVersion: "41007"
  selfLink: /apis/operators.coreos.com/v1/namespaces/openshift-operators/catalogsourceconfigs/devconsole-operator-catalog
  uid: 3e1a55ff-677e-11e9-8124-06472ef5fc02
spec:
  displayName: Red Hat Devconsole Operator
  packages: devconsole-operator
  publisher: Red Hat
  targetNamespace: local-operators

But - the CatalogSource is not displayed in the UI:

ldimaggi · 2019-04-25T17:38:09Z

See:
operator-framework/operator-lifecycle-manager#822

pmacik · 2019-04-26T12:19:36Z

test/e2e/devconsole.operatorsource.cr.yaml

+  namespace: openshift-operators
+spec:
+  type: appregistry
+  endpoint: https://quay.io/application/ldimaggi-test-org/mychart


According to the official example:

metadata.namespace should be openshift-marketplace

spec.endpoint should be https://quay.io/cnr

spec.registryNamespace should be ldimaggi-test-org

pmacik · 2019-04-26T12:23:09Z

test/e2e/devconsole.operatorsource.catalog.yaml

+kind: CatalogSourceConfig
+metadata:
+  name: devconsole-operator-catalog
+  namespace: openshift-operators


According to the official example:

metadata.namespace should be openshift-marketplace

pmacik · 2019-04-26T12:56:35Z

I just realized, that the OperatorSource or CatalogSourceConfig should probably be placed in the openshift-marketplace namespace:

(operator-test-env)➜   oc get csc --all-namespaces
NAMESPACE               NAME                  STATUS      MESSAGE                                       AGE
openshift-marketplace   certified-operators   Succeeded   The object has been successfully reconciled   29h
openshift-marketplace   community-operators   Succeeded   The object has been successfully reconciled   29h
openshift-marketplace   redhat-operators      Succeeded   The object has been successfully reconciled   29h

(operator-test-env)➜   oc get catsrc --all-namespaces
NAMESPACE                              NAME                  NAME                  TYPE       PUBLISHER   AGE
openshift-marketplace                  certified-operators   Certified Operators   grpc       Red Hat     29h
openshift-marketplace                  community-operators   Community Operators   grpc       Red Hat     29h
openshift-marketplace                  redhat-operators      Red Hat Operators     grpc       Red Hat     29h
openshift-operator-lifecycle-manager   olm-operators         OLM Operators         internal   Red Hat     30h

(operator-test-env)➜   oc get opsrc --all-namespaces
NAMESPACE               NAME                  TYPE          ENDPOINT              REGISTRY              DISPLAYNAME           PUBLISHER   STATUS      MESSAGE                                       AGE
openshift-marketplace   certified-operators   appregistry   https://quay.io/cnr   certified-operators   Certified Operators   Red Hat     Succeeded   The object has been successfully reconciled   30h
openshift-marketplace   community-operators   appregistry   https://quay.io/cnr   community-operators   Community Operators   Red Hat     Succeeded   The object has been successfully reconciled   30h
openshift-marketplace   redhat-operators      appregistry   https://quay.io/cnr   redhat-operators      Red Hat Operators     Red Hat     Succeeded   The object has been successfully reconciled   30h

And when an OperatorSource is created in that namespace, the CatalogSourceConfig and CatalogSource are created right away with the same name:

(operator-test-env)➜  cat devconsole.operatorsource.yaml

apiVersion: operators.coreos.com/v1
kind: OperatorSource
metadata:
  name: devconsole-operators
  namespace: openshift-marketplace
spec:
  type: appregistry
  endpoint: https://quay.io/cnr
  registryNamespace: odcqe
  displayName: "Red Hat Devconsole Operator"
  publisher: "Red Hat"

(odcqe is my testing app registry namespace in quay.io)

(operator-test-env)➜  oc get csc --all-namespaces          
NAMESPACE               NAME                   STATUS      MESSAGE                                       AGE
openshift-marketplace   certified-operators    Succeeded   The object has been successfully reconciled   30h
openshift-marketplace   community-operators    Succeeded   The object has been successfully reconciled   30h
openshift-marketplace   devconsole-operators   Succeeded   The object has been successfully reconciled   90s
openshift-marketplace   redhat-operators       Succeeded   The object has been successfully reconciled   30h

(operator-test-env)➜  oc get catsrc --all-namespaces 
NAMESPACE                              NAME                   NAME                          TYPE       PUBLISHER   AGE
openshift-marketplace                  certified-operators    Certified Operators           grpc       Red Hat     30h
openshift-marketplace                  community-operators    Community Operators           grpc       Red Hat     30h
openshift-marketplace                  devconsole-operators   Red Hat Devconsole Operator   grpc       Red Hat     98s
openshift-marketplace                  redhat-operators       Red Hat Operators             grpc       Red Hat     30h
openshift-operator-lifecycle-manager   olm-operators          OLM Operators                 internal   Red Hat     30h

(operator-test-env)➜  oc get opsrc --all-namespaces
NAMESPACE               NAME                   TYPE          ENDPOINT              REGISTRY              DISPLAYNAME                   PUBLISHER   STATUS      MESSAGE                                       AGE
openshift-marketplace   certified-operators    appregistry   https://quay.io/cnr   certified-operators   Certified Operators           Red Hat     Succeeded   The object has been successfully reconciled   30h
openshift-marketplace   community-operators    appregistry   https://quay.io/cnr   community-operators   Community Operators           Red Hat     Succeeded   The object has been successfully reconciled   30h
openshift-marketplace   devconsole-operators   appregistry   https://quay.io/cnr   odcqe                 Red Hat Devconsole Operator   Red Hat     Succeeded   The object has been successfully reconciled   102s
openshift-marketplace   redhat-operators       appregistry   https://quay.io/cnr   redhat-operators      Red Hat Operators             Red Hat     Succeeded   The object has been successfully reconciled   30h

pmacik · 2019-04-26T13:18:48Z

Then a pod is created:

(operator-test-env)➜  oc get pods -n openshift-marketplace

NAME                                    READY   STATUS             RESTARTS   AGE
certified-operators-d58db9c6b-thm2r     1/1     Running            0          30h
community-operators-7cfbbbb6b-v5cvg     1/1     Running            0          30h
devconsole-operators-585dcb889c-4lv2p   0/1     CrashLoopBackOff   9          22m
marketplace-operator-5cc4974ddc-mqzmt   1/1     Running            0          30h
redhat-operators-77848897d7-s28vh       1/1     Running            0          30h

but the pod fails (my helm chart for the operator in the app registry in quay is likely packaged incorrectly):

(operator-test-env)➜  ~ oc logs devconsole-operators-585dcb889c-4lv2p -n openshift-marketplace
time="2019-04-26T13:12:16Z" level=info msg="Using in-cluster kube client config" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="operator source(s) specified are - [https://quay.io/cnr|odcqe]" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="package(s) specified are - devconsole-operator-test" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="input has been sanitized" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="sources: [https://quay.io/cnr/odcqe]" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="packages: [devconsole-operator-test]" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="resolved the following packages: [odcqe/devconsole-operator-test:0.1.0]" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="downloading repository: odcqe/devconsole-operator-test:0.1.0 from https://quay.io/cnr" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="download complete - 1 repositories have been downloaded" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="decoding the downloaded operator manifest(s)" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="manifest format is - nested" port=50051 repository="odcqe/devconsole-operator-test:0.1.0" type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="downloaded/devconsole-operator-test/devconsole.operatorsource.catalog.yaml - type=file" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=error msg="skipping due to error - error happened while processing tar file - open downloaded/devconsole-operator-test/devconsole.operatorsource.catalog.yaml: no such file or directory" port=50051 repository="odcqe/devconsole-operator-test:0.1.0" type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="decoded 0 flattened and 1 nested operator manifest(s)" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=info msg="loading nested operator bundle(s) from downloaded into sqlite" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=error msg="stat failed on target directory[downloaded] - stat downloaded: no such file or directory" port=50051 type=appregistry
time="2019-04-26T13:12:16Z" level=fatal msg="error loading manifest from remote registry - stat downloaded: no such file or directory" port=50051 type=appregistry

pmacik · 2019-04-26T13:31:02Z

This line in the pod log got my attention:

time="2019-04-26T13:12:16Z" level=error msg="skipping due to error - error happened while processing tar file - open downloaded/devconsole-operator-test/devconsole.operatorsource.catalog.yaml: no such file or directory" port=50051 repository="odcqe/devconsole-operator-test:0.1.0" type=appregistry

Avni-Sharma · 2019-04-29T11:01:22Z

/test lint

… verifying creating Operator from OperatorSource. https://jira.coreos.com/browse/ODC-580

pmacik · 2019-05-02T11:34:37Z

@ldimaggi We should split the OperatorSource test into 2 steps - first, pushing the operator into the app registry and second, installing the operator via the OperatorSource from that app registry.

#174 Covers the pushing of the operator to the Quay app registry.

ldimaggi · 2019-05-03T12:42:28Z

/assign @sbose78

baijum

LGTM

openshift-ci-robot · 2019-05-03T12:50:41Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: baijum

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [baijum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ldimaggi · 2019-05-03T17:50:20Z

/retest

pmacik · 2019-05-06T12:46:47Z

This PR is made obsolete by #181

ldimaggi · 2019-05-06T12:53:50Z

Closing this pull request - un-merged - in favor of pull request #181

openshift-ci-robot requested review from alexeykazakov and tinakurian April 22, 2019 18:16

openshift-ci-robot added the size/M label Apr 22, 2019

openshift-ci-robot added the needs-ok-to-test label Apr 22, 2019

ldimaggi changed the title ~~OperatorSource related files to support extending E2E test to include…~~ DO NOT MERGE - OperatorSource related files to support extending E2E test to include… Apr 22, 2019

sbose78 reviewed Apr 22, 2019

View reviewed changes

ldimaggi force-pushed the 580-operator-source branch 2 times, most recently from 4fab7ab to d8ac519 Compare April 24, 2019 18:12

openshift-ci-robot added ok-to-test and removed needs-ok-to-test labels Apr 24, 2019

sbose78 reviewed Apr 24, 2019

View reviewed changes

ldimaggi force-pushed the 580-operator-source branch from d8ac519 to 61ddcb2 Compare April 25, 2019 23:23

pmacik reviewed Apr 26, 2019

View reviewed changes

pmacik mentioned this pull request Apr 26, 2019

Operator catalog is created but now shown in the OLM UI operator-framework/operator-lifecycle-manager#822

Closed

ldimaggi force-pushed the 580-operator-source branch from 61ddcb2 to a513b79 Compare April 26, 2019 16:47

ldimaggi changed the title ~~DO NOT MERGE - OperatorSource related files to support extending E2E test to include…~~ [WIP] OperatorSource related files to support extending E2E test to include… Apr 29, 2019

openshift-ci-robot added the do-not-merge/work-in-progress label Apr 29, 2019

ldimaggi force-pushed the 580-operator-source branch 3 times, most recently from 5cef52d to 43f84dc Compare April 30, 2019 14:30

openshift-ci-robot added size/S and removed size/M labels Apr 30, 2019

ldimaggi changed the title ~~[WIP] OperatorSource related files to support extending E2E test to include…~~ [WIP] Extend E2E test to include coverage for creating Operator from OperatorSource Apr 30, 2019

OperatorSource related files to support extending E2E test to include…

f1fd4f8

… verifying creating Operator from OperatorSource. https://jira.coreos.com/browse/ODC-580

ldimaggi force-pushed the 580-operator-source branch from 43f84dc to f1fd4f8 Compare May 1, 2019 18:24

openshift-ci-robot added size/M and removed size/S labels May 1, 2019

ldimaggi changed the title ~~[WIP] Extend E2E test to include coverage for creating Operator from OperatorSource~~ Extend E2E test to include coverage for creating Operator from OperatorSource May 3, 2019

openshift-ci-robot removed the do-not-merge/work-in-progress label May 3, 2019

openshift-ci-robot assigned sbose78 May 3, 2019

baijum approved these changes May 3, 2019

View reviewed changes

openshift-ci-robot added the approved label May 3, 2019

Merge branch 'master' into 580-operator-source

94ba3e3

ldimaggi force-pushed the 580-operator-source branch from 9bf0cf1 to 94ba3e3 Compare May 3, 2019 12:57

pmacik mentioned this pull request May 6, 2019

Test operator source #181

Merged

ldimaggi closed this May 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extend E2E test to include coverage for creating Operator from OperatorSource #135

Extend E2E test to include coverage for creating Operator from OperatorSource #135

ldimaggi commented Apr 22, 2019 •

edited

Loading

openshift-ci-robot commented Apr 22, 2019

sbose78 Apr 22, 2019

ldimaggi Apr 22, 2019

sbose78 commented Apr 24, 2019

sbose78 Apr 24, 2019

ldimaggi Apr 24, 2019

sbose78 commented Apr 25, 2019

ldimaggi commented Apr 25, 2019 •

edited

Loading

pmacik commented Apr 25, 2019 •

edited

Loading

ldimaggi commented Apr 25, 2019

ldimaggi commented Apr 25, 2019

ldimaggi commented Apr 25, 2019

pmacik Apr 26, 2019 •

edited

Loading

pmacik Apr 26, 2019

pmacik commented Apr 26, 2019 •

edited

Loading

pmacik commented Apr 26, 2019 •

edited

Loading

pmacik commented Apr 26, 2019 •

edited

Loading

Avni-Sharma commented Apr 29, 2019

pmacik commented May 2, 2019 •

edited

Loading

ldimaggi commented May 3, 2019

baijum left a comment

openshift-ci-robot commented May 3, 2019

ldimaggi commented May 3, 2019

pmacik commented May 6, 2019

ldimaggi commented May 6, 2019

Extend E2E test to include coverage for creating Operator from OperatorSource #135

Extend E2E test to include coverage for creating Operator from OperatorSource #135

Conversation

ldimaggi commented Apr 22, 2019 • edited Loading

openshift-ci-robot commented Apr 22, 2019

sbose78 Apr 22, 2019

Choose a reason for hiding this comment

ldimaggi Apr 22, 2019

Choose a reason for hiding this comment

sbose78 commented Apr 24, 2019

sbose78 Apr 24, 2019

Choose a reason for hiding this comment

ldimaggi Apr 24, 2019

Choose a reason for hiding this comment

sbose78 commented Apr 25, 2019

ldimaggi commented Apr 25, 2019 • edited Loading

pmacik commented Apr 25, 2019 • edited Loading

ldimaggi commented Apr 25, 2019

ldimaggi commented Apr 25, 2019

ldimaggi commented Apr 25, 2019

pmacik Apr 26, 2019 • edited Loading

Choose a reason for hiding this comment

pmacik Apr 26, 2019

Choose a reason for hiding this comment

pmacik commented Apr 26, 2019 • edited Loading

pmacik commented Apr 26, 2019 • edited Loading

pmacik commented Apr 26, 2019 • edited Loading

Avni-Sharma commented Apr 29, 2019

pmacik commented May 2, 2019 • edited Loading

ldimaggi commented May 3, 2019

baijum left a comment

Choose a reason for hiding this comment

openshift-ci-robot commented May 3, 2019

ldimaggi commented May 3, 2019

pmacik commented May 6, 2019

ldimaggi commented May 6, 2019

ldimaggi commented Apr 22, 2019 •

edited

Loading

ldimaggi commented Apr 25, 2019 •

edited

Loading

pmacik commented Apr 25, 2019 •

edited

Loading

pmacik Apr 26, 2019 •

edited

Loading

pmacik commented Apr 26, 2019 •

edited

Loading

pmacik commented Apr 26, 2019 •

edited

Loading

pmacik commented Apr 26, 2019 •

edited

Loading

pmacik commented May 2, 2019 •

edited

Loading