Enhance Container Submission Flow to Better Handle Timeouts

[acornett21]

Is your feature request related to a problem? Please describe.

When we first build check container we only had REST calls to pyxis in the submission flow, now that we have calls to pyxis in the checks, it leaves for a potential undesirable experience for partners. If there is a timeout in the checks, we then post an image with the certified: false for the image, and no way to change that, even if the partner re-runs preflight and gets valid results from pyxis on the 2nd, 3rd, nth run.

Describe the solution you'd like.

I'm not sure if the ideal solution is

• If there is a pyxis timeout in the check, exit preflight/stop the submit call
• Do a POST, GET, and if there is a diff do a PATCH for /images (see additional comments below)

Additional context.

PUT, PATCH was not available when we built preflight, and are not in the REST implementation yet, but were added in later see JIRA

[jfrancin]

Another partner got bit by this same issue over the weekend. Case 03253477 - partner ran Preflight on three images; they all failed with a weird error:

time="2022-06-25T00:44:23-04:00" level=error msg="could not marshal GraphQL query: Message: Unexpected error occurred, Locations: []"
time="2022-06-25T00:44:23-04:00" level=error msg="Error when querying pyxis for uncompressed top layer idsMessage: Unexpected error occurred, Locations: []"
time="2022-06-25T00:44:23-04:00" level=error msg="Unable to verify layer hashesMessage: Unexpected error occurred, Locations: []"
time="2022-06-25T00:44:23-04:00" level=info msg="check completed: BasedOnUbi" ERROR="Message: Unexpected error occurred, Locations: []" result="Message: Unexpected error occurred, Locations: []"

Subsequent tests today (27 June) passed. Filed Jira https://issues.redhat.com/browse/ISVISSUE-149 to get results changed.

[jfrancin]

Partner decided to rebuild (to cause different image SHAs), which allowed the containers to be re-submitted with passing grades, and are now published.