Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(feat): (testing): Add extensible unpacking interface + controller tests #65

Merged
merged 11 commits into from
May 24, 2023
Merged

(feat): (testing): Add extensible unpacking interface + controller tests #65

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
fdd0f2c
add unpacker interface and image source type from #59
everettraven May 16, 2023
00490ce
add tests and some minor cleanup
everettraven May 17, 2023
8139725
remove unnecessary function as per review comments
everettraven May 18, 2023
a8dd3c9
address review comments
everettraven May 19, 2023
dd71148
commit forgotten files
everettraven May 19, 2023
66586db
add AfterEach() blocks
everettraven May 22, 2023
cdc5d15
refactor the happy path test to be multiple smaller tests
everettraven May 22, 2023
d0fe3b3
more consistent image source field naming
everettraven May 22, 2023
fbf0653
address review comments
everettraven May 23, 2023
2868d7f
add comment above template variables
everettraven May 24, 2023
79d2334
remove line numbers from comment
everettraven May 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions cmd/manager/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import (
"sigs.k8s.io/controller-runtime/pkg/healthz"
"sigs.k8s.io/controller-runtime/pkg/log/zap"

"github.com/operator-framework/catalogd/internal/source"
"github.com/operator-framework/catalogd/internal/version"
"github.com/operator-framework/catalogd/pkg/apis/core/v1beta1"
corecontrollers "github.com/operator-framework/catalogd/pkg/controllers/core"
Expand All @@ -56,7 +57,7 @@ func main() {
metricsAddr string
enableLeaderElection bool
probeAddr string
opmImage string
unpackImage string
profiling bool
catalogdVersion bool
)
Expand All @@ -65,7 +66,8 @@ func main() {
flag.BoolVar(&enableLeaderElection, "leader-elect", false,
"Enable leader election for controller manager. "+
"Enabling this will ensure there is only one active controller manager.")
flag.StringVar(&opmImage, "opm-image", "quay.io/operator-framework/opm:v1.26", "The opm image to use when unpacking catalog images")
// TODO: should we move the unpacker to some common place? Or... hear me out... should catalogd just be a rukpak provisioner?
flag.StringVar(&unpackImage, "unpack-image", "quay.io/operator-framework/rukpak:v0.12.0", "The unpack image to use when unpacking catalog images")
everettraven marked this conversation as resolved.
Show resolved Hide resolved
opts := zap.Options{
Development: true,
}
Expand Down Expand Up @@ -94,11 +96,15 @@ func main() {
os.Exit(1)
}

unpacker, err := source.NewDefaultUnpacker(mgr, "catalogd-system", unpackImage)
everettraven marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
setupLog.Error(err, "unable to create unpacker")
os.Exit(1)
}

if err = (&corecontrollers.CatalogReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
Cfg: mgr.GetConfig(),
OpmImage: opmImage,
Unpacker: unpacker,
}).SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "Catalog")
os.Exit(1)
Expand Down
67 changes: 55 additions & 12 deletions config/crd/bases/catalogd.operatorframework.io_catalogs.yaml
View file
Open in desktop
everettraven marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -34,19 +34,34 @@ spec:
spec:
description: CatalogSpec defines the desired state of Catalog
properties:
image:
description: Image is the Catalog image that contains Operators' metadata
in the FBC format https://olm.operatorframework.io/docs/reference/file-based-catalogs/#docs
type: string
pollingInterval:
description: PollingInterval is used to determine the time interval
between checks of the latest index image version. The image is polled
to see if a new version of the image is available. If available,
the latest image is pulled and the cache is updated to contain the
new content.
type: string
source:
description: Source is the source of a Catalog that contains Operators'
metadata in the FBC format https://olm.operatorframework.io/docs/reference/file-based-catalogs/#docs
properties:
image:
description: Image is the catalog image that backs the content
of this catalog.
properties:
pullSecret:
description: ImagePullSecretName contains the name of the
everettraven marked this conversation as resolved.
Show resolved Hide resolved
image pull secret in the namespace that the provisioner
is deployed.
type: string
ref:
description: Ref contains the reference to a container image
grokspawn marked this conversation as resolved.
Show resolved Hide resolved
containing Catalog contents.
type: string
required:
- ref
type: object
type:
description: Type defines the kind of Catalog content being sourced.
type: string
required:
- type
type: object
required:
- image
- source
type: object
status:
description: CatalogStatus defines the observed state of Catalog
Expand Down Expand Up @@ -121,6 +136,34 @@ spec:
- type
type: object
type: array
phase:
type: string
resolvedSource:
description: CatalogSource contains the sourcing information for a
Catalog
properties:
image:
description: Image is the catalog image that backs the content
of this catalog.
properties:
pullSecret:
description: ImagePullSecretName contains the name of the
image pull secret in the namespace that the provisioner
is deployed.
type: string
ref:
description: Ref contains the reference to a container image
containing Catalog contents.
type: string
required:
- ref
type: object
type:
description: Type defines the kind of Catalog content being sourced.
type: string
required:
- type
type: object
type: object
type: object
served: true
Expand Down
13 changes: 4 additions & 9 deletions config/rbac/role.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,6 @@ kind: ClusterRole
metadata:
name: manager-role
rules:
- apiGroups:
- batch
resources:
- jobs
verbs:
- create
- get
- list
- watch
Comment on lines -7 to -15
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the switch to pods instead of jobs only because rukpak provisioner uses an unpack pod instead of a job? This may have subtle but salient implications for catalogd due to the fact that there's product level differences between bundle images and catalog images. For eg, bundle images are generally publicly pull-able(at least has been advised to be that way, but that's totally up for debate possible in v1), however, catalog images are private/public, and private images will need pull secrets to be pulled, that's easily passable to a Job, vs a pod that needs additional configuration in the code.

With that context, think more differences between bundle and catalog images showing up, and therefore the unpack pod needing to be more and more configured in code with for each difference.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I don't think there really is a difference between catalog and bundle images. It is just as possible for a bundle image to require a pull secret as a catalog image
  2. Jobs and Pods are (as far as I know) both easily capable of dealing with pull secrets.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the switch to pods instead of jobs only because rukpak provisioner uses an unpack pod instead of a job?

Since we are pretty much copying the logic that rukpak uses internally and rukpak uses a Pod instead of Job, yes.

This may have subtle but salient implications for catalogd due to the fact that there's product level differences between bundle images and catalog images

I'm not entirely sure I am following here. With FBC aren't catalog images just going to be images that contain a specific directory/file structure? IIUC this is exactly what a bundle is just on a smaller scale and the files are instead Kubernetes manifests. I understand that there are different views on what a catalog vs a bundle is, hence the creation of a Catalog API, but at the end of the day their base architecture follows a similar pattern of putting everything as files in a filesystem and they are just read accordingly. I don't think we should embed any specific logic to differentiate between a catalog and a bundle as IMO it just adds unnecessary complexity. IMO someone should actually be able to put an image reference that references a bundle image instead of a catalog image and catalogd should still successfully be able to "unpack" it. The place where it should fail is when attempting to render contents that would only exist in a catalog.

@joelanford also makes good points - for reference, a Pod using a pull secret looks something like (Kubernetes docs on this):

apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: <your-private-image>
  imagePullSecrets:
  - name: regcred

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've been bitten too many times for using pods directly instead of existing Kubernetes primitives in v0. When trying to use the pull secret example, I wasn't trying to suggest that you cannot do that with pods, but to the fact that the job/deployment controller does exactly that, pass on the pull secret to the pod it creates, but instead of letting the job/deployment controller do that we sign up for doing that ourselves, and then have to keep including tasks like this in our controller that we could have just delegated to the existing controllers. Eg tomorrow if there's a policy change that requires change in how pods are bootstrapped (think something similar to PSA), then there'll be three controllers incorporating those changes: the jobs controller, the deployment controller, and our controller. And then we have to worry about OCP releases/backports etc etc. Instead, using Kubernetes primitives mean we make minimal changes on our controller to incorporate policy changes, the same change that every other project will be making.

But, sounds like we need to have this discussion in rukpak instead.

Not resolving the conversation now so that we remember to capture this in an issue in rukpak by referencing this later.

- apiGroups:
- catalogd.operatorframework.io
resources:
Expand Down Expand Up @@ -96,8 +87,12 @@ rules:
resources:
- pods
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
Expand Down
14 changes: 5 additions & 9 deletions config/samples/core_v1beta1_catalog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,9 @@
apiVersion: catalogd.operatorframework.io/v1beta1
kind: Catalog
metadata:
labels:
app.kubernetes.io/name: catalog
app.kubernetes.io/instance: catalog-sample
app.kubernetes.io/part-of: catalogd
app.kuberentes.io/managed-by: kustomize
app.kubernetes.io/created-by: catalogd
name: catalog-sample
name: operatorhubio
spec:
image: quay.io/operatorhubio/catalog:latest
pollingInterval: 45m
source:
type: image
image:
ref: quay.io/operatorhubio/catalog:latest
4 changes: 4 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@ module github.com/operator-framework/catalogd
go 1.19

require (
github.com/nlepage/go-tarfs v1.1.0
github.com/onsi/ginkgo/v2 v2.6.0
github.com/onsi/gomega v1.24.1
github.com/operator-framework/operator-registry v1.26.3
github.com/psanford/memfs v0.0.0-20230130182539-4dbf7e3e865e
everettraven marked this conversation as resolved.
Show resolved Hide resolved
k8s.io/api v0.26.0
k8s.io/apimachinery v0.26.0
k8s.io/client-go v0.26.0
Expand Down
7 changes: 7 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,8 +240,12 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8m
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/nlepage/go-tarfs v1.1.0 h1:bsACOiZMB/zFjYG/sE01070i9Fl26MnRpw0L6WuyfVs=
github.com/nlepage/go-tarfs v1.1.0/go.mod h1:IhxRcLhLkawBetnwu/JNuoPkq/6cclAllhgEa6SmzS8=
github.com/onsi/ginkgo/v2 v2.6.0 h1:9t9b9vRUbFq3C4qKFCGkVuq/fIHji802N1nrtkh1mNc=
github.com/onsi/ginkgo/v2 v2.6.0/go.mod h1:63DOGlLAH8+REH8jUGdL3YpCpu7JODesutUjdENfUAc=
github.com/onsi/gomega v1.24.1 h1:KORJXNNTzJXzu4ScJWssJfJMnJ+2QJqhoQSRwNlze9E=
github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM=
github.com/operator-framework/api v0.17.2-0.20220915200120-ff2dbc53d381 h1:/XHgTzfI0O/RP3I6WF0BiPLVuVkfgVyiw04b0MyCJ2M=
github.com/operator-framework/api v0.17.2-0.20220915200120-ff2dbc53d381/go.mod h1:wof6IrBhVAufc+ZiQo/BB68fKctXiuSEAMbOO29kZdI=
github.com/operator-framework/operator-registry v1.26.3 h1:U+HTGgjAT5RCXU2WkDwa525wcqdo97BsO7WfMhwL5MA=
Expand Down Expand Up @@ -278,6 +282,8 @@ github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1
github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
github.com/prometheus/procfs v0.8.0 h1:ODq8ZFEaYeCaZOJlZZdJA2AbQR98dSHSM1KW/You5mo=
github.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4=
github.com/psanford/memfs v0.0.0-20230130182539-4dbf7e3e865e h1:51xcRlSMBU5rhM9KahnJGfEsBPVPz3182TgFRowA8yY=
github.com/psanford/memfs v0.0.0-20230130182539-4dbf7e3e865e/go.mod h1:tcaRap0jS3eifrEEllL6ZMd9dg8IlDpi2S1oARrQ+NI=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
Expand All @@ -295,6 +301,7 @@ github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81P
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
github.com/xanzy/ssh-agent v0.3.0/go.mod h1:3s9xbODqPuuhK9JV1R321M/FlMZSBvE5aY6eAcqrDh0=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
Expand Down
Loading
Loading