CSV required CRDs are resolved from an unpredictable catalog

[kaczyns]

Bug Report

What did you do?
We are trying to define dependencies that our operator has on other operators, by using the required: stanza in the CSV. I have an operator with a CSV that contains:

    required:
      - description: Knative serving
        displayName: Knative serving
        kind: KnativeServing
        name: knativeservings.serving.knative.dev
        version: v1alpha1
      - description: Knative eventing
        displayName: Knative eventing
        kind: KnativeEventing
        name: knativeeventings.eventing.knative.dev
        version: v1alpha1
      - description: Tekton Pipelines
        displayName: Tekton Pipelines
        kind: Config
        name: config.operator.tekton.dev
        version: v1alpha1
      - description: Appsody
        displayName: Appsody
        kind: AppsodyApplication
        name: appsodyapplications.appsody.dev
        version: v1beta1

Using the Openshift console I create a subscription to this operator, in the openshift-operators namespace, at the cluster scope. OLM resolves the required CRDs and I am left with the following subscriptions, from oc get subscriptions --namespace=openshift-operators:

NAME                                                                                 PACKAGE                        SOURCE                CHANNEL
appsody-community-operator-beta-community-operators-openshift-marketplace            appsody-community-operator     community-operators   beta
kabanero-operator                                                                    kabanero-operator              example-manifests     release-0.3.0
knative-eventing-operator-alpha-community-operators-openshift-marketplace            knative-eventing-operator      community-operators   alpha
openshift-pipelines-operator-dev-preview-community-operators-openshift-marketplace   openshift-pipelines-operator   community-operators   dev-preview
serverless-operator-release-0.3.0-example-manifests-openshift-marketplace            serverless-operator            example-manifests     release-0.3.0

I delete everything and do it again. This time I get:

NAME                                                                                 PACKAGE                        SOURCE                CHANNEL
appsody-operator-certified-beta-certified-operators-openshift-marketplace            appsody-operator-certified     certified-operators   beta
kabanero-operator                                                                    kabanero-operator              example-manifests     release-0.3.0
knative-eventing-operator-alpha-community-operators-openshift-marketplace            knative-eventing-operator      community-operators   alpha
knative-serving-operator-alpha-community-operators-openshift-marketplace             knative-serving-operator       community-operators   alpha
openshift-pipelines-operator-dev-preview-community-operators-openshift-marketplace   openshift-pipelines-operator   community-operators   dev-preview

What did you expect to see?
I expected the subscriptions to be the same the second time (pulled from the same catalog and channel).

What did you see instead? Under which circumstances?
I see that the first time, the appsody operator was pulled from the community-operators catalog, beta channel (which happens to be version 0.1.0). The second time, from certified-operators, beta channel (version 0.2.0). For KnativeServing, the first time pulled from the Serverless Operator, example-manifests catalog (private catalog), release-0.3.0 channel. The second time, from the Knative Serving Operator, community-operators catalog, alpha channel.

Environment

• operator-lifecycle-manager version:

NAME                                       VERSION                         AVAILABLE   PROGRESSING   DEGRADED   SINCE
operator-lifecycle-manager                 4.2.0-0.okd-2019-10-12-001757   True        False         False      6h14m
operator-lifecycle-manager-catalog         4.2.0-0.okd-2019-10-12-001757   True        False         False      6h15m
operator-lifecycle-manager-packageserver   4.2.0-0.okd-2019-10-12-001757   True        False         False      93m


  Versions:
    Name:     operator
    Version:  4.2.0-0.okd-2019-10-12-001757
    Name:     operator-lifecycle-manager
    Version:  0.11.0

• Kubernetes version information:

oc version
Client Version: version.Info{Major:"4", Minor:"1+", GitVersion:"v4.1.1", GitCommit:"4b47d3394", GitTreeState:"clean", BuildDate:"2019-06-05T20:42:01Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.6+f9b5405", GitCommit:"f9b5405", GitTreeState:"clean", BuildDate:"2019-10-11T22:06:47Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes cluster kind:

 oc get clusterversion
NAME      VERSION                         AVAILABLE   PROGRESSING   SINCE
version   4.2.0-0.okd-2019-10-12-001757   True        False         6h6m

Possible Solution
Not sure....

Additional context
We're just hoping to determine that the order is predictable, and what that order is. For example, do certain catalogs (maybe certified-operators) take precedence over others? Or, are the catalogs processed in some order (alphanumeric)?

I had a private catalog defined for this test that contained duplicates of some of these operators, in the same channel, in hopes that it would take precedence, but that was not the case. I can re-produce without this private catalog if necessary.

[ecordell]

Thanks for the bug report @kaczyns.

The first thing we are doing is prioritizing dependencies in the same catalog as the initial operator. There's a PR implementing this that will merge soon ( #1080 ) which should fix the specific problem you are facing.

The next thing will be to define a deterministic order. We'll have a proposal up for review soon to describe how this will work. Currently I'm thinking we will assign a priority to each catalog source, and then prefer (in order) catalogs in your current namespace, and then (in order) catalogs that are available to you globally.

The reason you're hitting this issue in okd is because we are publishing two operators, in two global catalogs, that both provide the same APIs. Our vision for how dependencies would be used did not originally account for this (i.e. we would only publish one owner in the global catalogs we ship), and we are now revisiting those opinions and hope to have solutions out soon.

[kaczyns]

@ecordell Thanks for responding to this so quickly - we look forward to trying it out when it's available.