certification-project-id with multiple dashes causes 404 status when trying to submit

[jrsmroz]

Bug Description

When a certification-project-id has multiple dashes, the preflight fails with 404 error as a result of truncating the project id.

$ docker run                                                              \
  -v <my_home_path>/.docker/:/docker/                                     \
  quay.io/opdev/preflight:stable                                          \
  check container scan.connect.redhat.com/ospid-f8304390-ff4c-4b7f-8c7e-1ed13370b3c2/nightly-ingress-controller:nightly-redhat \
  --docker-config=/docker/config.json                                     \
  --submit                                                                \
  --certification-project-id="ospid-f8304390-ff4c-4b7f-8c7e-1ed13370b3c2" \
  --pyxis-api-token="<my-pyxis-token>"                                    \
  --loglevel=debug

time="2022-08-09T22:31:47Z" level=debug msg="config file not found, proceeding without it"
time="2022-08-09T22:31:47Z" level=info msg="certification library version 1.3.3 <commit: 6c30f7caae731ecc2faeaebd78c6d3981c708fe6>"
time="2022-08-09T22:31:47Z" level=debug msg="URL is: https://catalog.redhat.com/api/containers/v1/projects/certification/id/f8304390"
Error: could not retrieve project: status code: 404: body: {"detail": "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.", "status": 404, "title": "Not Found", "type": "about:blank", "trace_id": "0x7ded39401a01e8813f796704de288202"}
Usage:
  preflight check container [flags]

Examples:
  preflight check container quay.io/repo-name/container-name:version

Flags:
      --certification-project-id string   Certification Project ID from connect.redhat.com/projects/{certification-project-id}/overview
                                          URL paramater. This value may differ from the PID on the overview page. (env: PFLT_CERTIFICATION_PROJECT_ID)
  -h, --help                              help for container
      --pyxis-api-token string            API token for Pyxis authentication (env: PFLT_PYXIS_API_TOKEN)
      --pyxis-env string                  Env to use for Pyxis submissions. (default "prod")
      --pyxis-host string                 Host to use for Pyxis submissions. This will override Pyxis Env. Only set this if you know what you are doing.
                                          If you do set it, it should include just the host, and the URI path. (env: PFLT_PYXIS_HOST)
  -s, --submit                            submit check container results to red hat

Global Flags:
      --artifacts string       Where check-specific artifacts will be written. (env: PFLT_ARTIFACTS)
  -d, --docker-config string   Path to docker config.json file. This value is optional for publicly accessible images.
                               However, it is strongly encouraged for public Docker Hub images,
                               due to the rate limit imposed for unauthenticated requests. (env: PFLT_DOCKERCONFIG)
      --logfile string         Where the execution logfile will be written. (env: PFLT_LOGFILE)
      --loglevel string        The verbosity of the preflight tool itself. Ex. warn, debug, trace, info, error. (env: PFLT_LOGLEVEL)

time="2022-08-09T22:31:48Z" level=fatal msg="could not retrieve project: status code: 404: body: {\"detail\": \"The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.\", \"status\": 404, \"title\": \"Not Found\", \"type\": \"about:blank\", \"trace_id\": \"0x7ded39401a01e8813f796704de288202\"}"

Version and Command Invocation

preflight version 1.3.3 <commit: 6c30f7c>

Steps to Reproduce:

1. Create a project that will have a dashes in project id (I do not know how that happened)
2. Run the preflight utility specifying the project id. See the command above.

Expected Result

Should behave same as with project id with a single dash or no dashes at all.

Actual Result

The "URL is: https://catalog.redhat.com/api/containers/v1/projects/certification/id/f8304390" indicates that the project id was truncated. As a result project was no found, thus 404 status.

Additional Context

N/A

[jrsmroz]

Seems like the problem is this line https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/certification/runtime/config.go#L69

[acornett21]

@jrsmroz This is confusing, but the projectid is not the ospid value that you used to push the image to scan.connect.redhat.com the value that needs to be passed in to prelfight would be 5eab1b8ea165648bd1353266 this is the value in the URL. Sorry for the confusion.

[jrsmroz]

That is misleading, because for the other image we publish, providing ospid works fine. I will modify the pipeline and verify. The code, that checks for ospid also suggests that.

[jrsmroz]

I confirm that it worked. Would it be possible to prevent those kind of failures somehow?

[acornett21]

That is misleading, because for the other image we publish, providing ospid works fine. I will modify the pipeline and verify. The code, that checks for ospid also suggests that.

I understand that this is confusing, since images need to be pushed to our registry using ospid-f8304390-ff4c-4b7f-8c7e-1ed13370b3c2 and then other systems(ie databases and services) use 5eab1b8ea165648bd1353266, unfortunately this is how these systems work, and I don't foresee any changes in this behavior.

We will look at handling this error better in preflight, to avoid confusion.

One other question, are you submitting images to Red Hat nightly? If so this isn't supported and will cause issues, we do not even push Red Hat images nightly ie UBI images. Only images that are intended to be published should be pushed to Red Hat. You can still run preflight in your nightly build, but without the --submit flag, then your releases that are intended to be published can be ran with the --submit flag.

[jrsmroz]

Yeah, our pipeline pushes the images nightly. But those are our nightly images, build on top od released UBI image. The reasoning was doing the security scans often. However, we do not publish them, they stay in red hat scan registry. What is the problem with nightly images? So far it worked.

We also have a release pipeline that pushes the release images.

[acornett21]

We should only see images that you intend to publish, not every intermediate image, our systems were not designed for this type of use. If used this way, there will be a point where you will not be able to see or publish your images without manual intervention.

UBI images are not published nightly, they are only published when there is a vulnerability. If your concern is to address vulnerabilities when they happen, there are other processes that you can setup, happy to discuss that via other partner channels.

[jrsmroz]

Thanks for clarification.

[jfrancin]

One clarification on Adam's comment about OSPID values. For all projects created prior to July of 2021, the OSPID was derived from a process that created a value that had several dashes embedded and bore no resemblance to the Pyxis project ID.

For all projects created AFTER July 2021, the OSPID value is the same as the Pyxis database project ID.

There's an easy way to tell, besides the presence of dashes in the OSPID value. Pyxis project IDs for projects created before the cut-over will have a leading 5 in their value - hence 5eab1b8ea165648bd1353266 has an OSPID that doesn't match the project ID. However, if the project ID led with a 6 in the value - example 62c721d5ea2ed524ce4af71b - the OSPID value will match the project ID.