(feat): (testing): Add extensible unpacking interface + controller tests

[everettraven]

Description

• Adds the extensible unpacking interface introduced by @joelanford's WIP PR WIP: introduce source union type #59 but limits it to only adding an image source.
• Adds tests for the Catalog controller

Motivation

• Fixes Introduce spec.Source type in the CatalogSource CRD #62
• Fixes Create an unpacking interface #52
• Fixes Create an OwnerReference based watch on the unpack job #63
• Fixes (maybe?) Improve CatalogSource controller error handling during reconciliation #6
• Fixes Add unit tests for the CatalogSource reconciler #40
• Fixes The catalog example 'catalog-sample' can't be Ready #73

[everettraven]

@joelanford Follow up for exploring potential alternatives as we discussed #66

[anik120]

Is the switch to pods instead of jobs only because rukpak provisioner uses an unpack pod instead of a job? This may have subtle but salient implications for catalogd due to the fact that there's product level differences between bundle images and catalog images. For eg, bundle images are generally publicly pull-able(at least has been advised to be that way, but that's totally up for debate possible in v1), however, catalog images are private/public, and private images will need pull secrets to be pulled, that's easily passable to a Job, vs a pod that needs additional configuration in the code.

With that context, think more differences between bundle and catalog images showing up, and therefore the unpack pod needing to be more and more configured in code with for each difference.

[joelanford]

1. I don't think there really is a difference between catalog and bundle images. It is just as possible for a bundle image to require a pull secret as a catalog image
2. Jobs and Pods are (as far as I know) both easily capable of dealing with pull secrets.

[everettraven]

Is the switch to pods instead of jobs only because rukpak provisioner uses an unpack pod instead of a job?

Since we are pretty much copying the logic that rukpak uses internally and rukpak uses a Pod instead of Job, yes.

This may have subtle but salient implications for catalogd due to the fact that there's product level differences between bundle images and catalog images

I'm not entirely sure I am following here. With FBC aren't catalog images just going to be images that contain a specific directory/file structure? IIUC this is exactly what a bundle is just on a smaller scale and the files are instead Kubernetes manifests. I understand that there are different views on what a catalog vs a bundle is, hence the creation of a Catalog API, but at the end of the day their base architecture follows a similar pattern of putting everything as files in a filesystem and they are just read accordingly. I don't think we should embed any specific logic to differentiate between a catalog and a bundle as IMO it just adds unnecessary complexity. IMO someone should actually be able to put an image reference that references a bundle image instead of a catalog image and catalogd should still successfully be able to "unpack" it. The place where it should fail is when attempting to render contents that would only exist in a catalog.

@joelanford also makes good points - for reference, a Pod using a pull secret looks something like (Kubernetes docs on this):

apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: <your-private-image>
  imagePullSecrets:
  - name: regcred

[anik120]

We've been bitten too many times for using pods directly instead of existing Kubernetes primitives in v0. When trying to use the pull secret example, I wasn't trying to suggest that you cannot do that with pods, but to the fact that the job/deployment controller does exactly that, pass on the pull secret to the pod it creates, but instead of letting the job/deployment controller do that we sign up for doing that ourselves, and then have to keep including tasks like this in our controller that we could have just delegated to the existing controllers. Eg tomorrow if there's a policy change that requires change in how pods are bootstrapped (think something similar to PSA), then there'll be three controllers incorporating those changes: the jobs controller, the deployment controller, and our controller. And then we have to worry about OCP releases/backports etc etc. Instead, using Kubernetes primitives mean we make minimal changes on our controller to incorporate policy changes, the same change that every other project will be making.

But, sounds like we need to have this discussion in rukpak instead.

Not resolving the conversation now so that we remember to capture this in an issue in rukpak by referencing this later.

[anik120]

I don't remember what the rukpak CRD validations exactly are, but this is probably highlighting a difference, catalogd doesn't need to be concerned with CRD validations.

[joelanford]

It does need to be concerned. This would make sure that you can't create a catalog with a non-sensical spec.source field. e.g.

• specify type, but not specify the corresponding struct
• specify type, but specify a different struct
• specify multiple structs
• not specify type
• etc...

[anik120]

Don't think we have to solve this in this PR, but mentioning this for the record:

Essentially there's concerns about the fidelity of pod logs (specifically, logs being complete, and therefore a reliable source of complete content). Instead unpack jobs/pods should send the data back to catalogd using a service endpoint:port that catalogd makes available for these uploads, and the logs are then stored reliably in the filesystem/create confimap/s with the data, before they're consumed

[joelanford]

+1, and I agree with putting this out of scope.

[everettraven]

FWIW, this sounds like it is likely also something that rukpak needs to consider

[joelanford]

I'm concerned that this PR review is treading into: "let's dissect and tweak stuff about how rukpak does things", and I think:
a) we shouldn't diverge from rukpak's way of doing things because at the end of the day, we need to do the exact same thing: source a filesystem from some remote place. If we need something in rukpak to change, let's make sure we open an issue in rukpak about it.
b) we should capture the things about the implementation that we think should be improved (again, probably starting in rukpak).
c) if we haven't already we should capture an issue about converging the implementations.

If we diverge here, it just means more work when we go back and converge.

[everettraven]

I'm concerned that this PR review is treading into: "let's dissect and tweak stuff about how rukpak does things", and I think:
a) we shouldn't diverge from rukpak's way of doing things because at the end of the day, we need to do the exact same thing: source a filesystem from some remote place. If we need something in rukpak to change, let's make sure we open an issue in rukpak about it.
b) we should capture the things about the implementation that we think should be improved (again, probably starting in rukpak).
c) if we haven't already we should capture an issue about converging the implementations.

If we diverge here, it just means more work when we go back and converge.

+1. FWIW we could use #66 as the issue for the convergence of the implementations

[anik120]

/lgtm

Sounds like we have to resolve some of the issues I brought up in rukpak instead of here, so we can move these conversations to rukpak instead.

[openshift-ci]

New changes are detected. LGTM label has been removed.