Propose external library for sharing logic for operator checks

[sebrandon1]

Is your feature request related to a problem? Please describe.

The CNF Test Suite team and the Preflight team are both working to verify customer workloads and we would like to use some of the logic/checks that the preflight repo does for operators (and maybe even containers) from here and bring that same logic into the CNF test suite.

Describe the solution you'd like.

Some sort of consumable, external library (redhat-openshift-ecosystem/preflight-lib or some-name) where we can share the logic for performing checks against customer workloads. We could/should also think about sharing the logic we have on our side as well for some additional collaboration.

[sebrandon1]

Maybe it doesn't even need to be that extreme as the full check being broken out but possibly just the engine(s).

The wrapper around the operator-sdk commands would be cool to be able to consume for example: https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/certification/internal/engine/operatorsdk.go

[sebrandon1]

I have been working on implementing this the last couple of weeks and I think I'm at the point where I'm ready to demo/ask additional questions of the preflight team to help me complete the work.

Here is my WIP PoC PR for the work in the TNF suite: redhat-best-practices-for-k8s/certsuite#592
Here is my branch in my fork of the preflight repo where I'm building out the preflight "lib" functionality: https://github.com/sebrandon1/openshift-preflight/tree/preflight_lib_test

Essentially what I'm doing is trying to expose the functionality already living in preflight to be able to be consumed by the TNF suite.

Currently the preflight repo uses its cmd package to interact with the user + build test runners based on parameters passed in via the CLI.

What I did is break the the cobra/viper logic away from the test builder logic and put that into the "lib" package and corresponding folder/files. By doing this, I can reference the lib package directly from the TNF repo and pass in the parameters I want to kick off the tests.

The TNF repo also uses Ginkgo so I report the results in the claim file output and it looks like a native test was ran as part of the TNF code.

Outstanding questions I need some clarification on:

• Pyxis and connect credentials. I have currently left those out of the scope of my changes because I didn't want to have to deal with credentials while getting this PoC running. Maybe the preflight team can also consume oct when it's ready?
• Operator scorecard tests. The documentation seems geared towards someone running this via the CLI and passing in a valid KUBECONFIG. If I'm already running this application within a cluster, how do I get the scorecard binary to correctly use an in-cluster-config? Not sure if that's possible?

[acornett21]

• Pyxis and connect credentials. I have currently left those out of the scope of my changes because I didn't want to have to deal with credentials while getting this PoC running. Maybe the preflight team can also consume oct when it's ready?

Just to provide some clarity, the Pyxis credentials are used for submission of images/rpm-manifests/results/artifacts to a partners certification project on their behalf. Any Pyxis call that preflight makes in a check currently does not need credentials as we call public endpoints.

I will also note that if only operator policies are being exposed in the lib, these are not needed since any interacting with the partners project is done in the Hosted or Ci Pipeline, and not in preflight directly.

[komish]

I'm going to mark this issue as closed. We're currently tidying up and refactoring to get this merged into main, but the work done by @sebrandon1 here is merged to main, and the library can be consumed in the lib branch until it's merged to main in its final form!

[sebrandon1]

Thanks @komish!