Build Out Integration Tests for check container

[acornett21]

Is your feature request related to a problem? Please describe.

Since preflight now takes on the role of CVP as well as the connect UI, it would be nice if we had integration tests in place that interacted with pyxis, so we knew we have not broken a flow or given functionality with a change.

Describe the solution you'd like.

I think we would want to do something like

1. Setup net-new projects in connect. We could use a sample json based on what a net new project json looks like.
2. Run check container --submit
3. Wait ~30seconds - 1 minute
4. Call pyxis to see if the project looks like what we expect, ie repository/registry/status and that their is a health index score
5. Run check container --submit for the same project and make sure that the same values are as expected
   • With the same sha
   • With a new sha
6. Archive the project

Describe alternatives you've considered.

N/A

Additional context.

There might be more scenarios that we need to test, but this just a common one I've seen since check container went GA.

[vikasmulaje]

@acornett21 @jomkz We are already doing these things in E2E container Certification Pipeline.
Can we sync and discuss more on this?

[tkrishtop]

@acornett21 I'm currently working on adding --submit option to DCI checks for each preflight PR.
Implementation details: simple-demo-operator in "private" repository + check container with --submit option.

[tkrishtop]

@acornett21 it's now merged. All open PRs in the Preflight repository will be now tested against simple-demo-operator:v0.0.6 and the test results will be automatically submitted into the container project 62824c134db48a54640b9104 with the help of out-of-box Preflight Pyxis submission.

[lslavkov]

@tkrishtop is there a PR for this?