Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CVE reports incorrectly report images that are no longer deployed with CKF #674

Closed
i-chvets opened this issue Aug 23, 2023 · 13 comments
Closed

CVE reports incorrectly report images that are no longer deployed with CKF #674

i-chvets opened this issue Aug 23, 2023 · 13 comments
Labels
bug Something isn't working

Comments

@i-chvets
Copy link
Contributor

i-chvets commented Aug 23, 2023 •
edited
Loading

Bug Description

Current CVE reporting tools incorrectly report images that are not deployed in the final product. These images should be reviewed and removed from reporting.

To Reproduce

Run tools/get-images.sh scripts in each charm repository (on track branches) to get the list of images related to that charm. In some cases, this list might contain images that are no longer deployed/used. Example: https://github.com/canonical/seldon-core-operator/blob/track/1.15/tools/get-images.sh
Example of image in the report that is not actually deployed is docker.io/seldonio/engine:1.12.0. It it set in static list here. This should be removed from that script/list.

Environment

Charm repositories, tools/ directory on track branches.

Relevant Log Output

N/A

Additional Context

Related issues and PRs in charm repositories:
canonical/seldon-core-operator#197
canonical/seldon-core-operator#186

Related PRs in CI and bundle repositories:
#666
canonical/kubeflow-ci#105
@kimwnasptd
Copy link
Contributor

Thanks @i-chvets! Let's use this issue to add the full list of images that are currently reported, and should not.

So until now we have:
docker.io/seldonio/engine:1.12.0
kserve/models-web-app:v0.8.0

I'd propose that we should also consider including the images defined in canonical/katib-rocks#17 (comment). But before we commit to removing those, let's double check where they were used in the first place.

@i-chvets i-chvets added the bug Something isn't working label Aug 23, 2023
@i-chvets
Copy link
Contributor Author

@i-chvets to take a look at katib-operators images to identify not deployed images.

@i-chvets
Copy link
Contributor Author

i-chvets commented Aug 23, 2023 •
edited by kimwnasptd
Loading

In katib-controller on track/0.15 there are multiple ConfigMaps are deployed that contain a number of default container images.
On main those images are specified in the charm's code as default images due to recent changes that were done to support custom image lists. In addition to already identified images for Katib the following 3 seem to be used as default AND can possibly be deployed:

  • kubeflowkatib/enas-cnn-cifar10-cpu
  • kubeflowkatib/mxnet-mnist
  • kubeflowkatib/pytorch-mnist-cpu
    I've updated spreadsheet foir latest/edge
    @kimwnasptd @andreeamun ^^^

@i-chvets
Copy link
Contributor Author

i-chvets commented Aug 24, 2023 •
edited
Loading

@kimwnasptd kserve/models-web-app:v0.8.0 image is referenced in 1.7 branch for kserve (it is not in main). However, I don't think we deploy that application in 1.7. So, it looks like it is false positive and we can remove it from the list.

@kimwnasptd
Copy link
Contributor

Yes, let's remove it

@i-chvets
Copy link
Contributor Author

Another note. In order to remove incorrectly reported image docker.io/seldonio/engine:1.12.0 the following issue/PR need to be addressed.

canonical/seldon-core-operator#199
canonical/seldon-core-operator#186

i-chvets pushed a commit to canonical/kserve-operators that referenced this issue Aug 24, 2023
fix: modified to retrieve images only from one charm
b4fe6f4 
canonical/bundle-kubeflow#674

Summary of changes:
- Modified scriot to retrieve images only from kserve-controller,
  because kserve-web-app is not deployed in this release.
This was referenced Aug 24, 2023
Merged
Closed
@kimwnasptd
Copy link
Contributor

@i-chvets what's the overall state at this point? Could you add a follow-up comment with all the images that we need to remove?

@i-chvets
Copy link
Contributor Author

i-chvets commented Aug 25, 2023 •
edited
Loading

  1. Kserve: After fixing other issue in Kserve repo (PR approved this morning EST), last PR is in review fix: modified to retrieve images only from one charm kserve-operators#155 and after that it is all done - kserve images will be all good.
  2. Seldon: Requires the following issue/PR to be resolved in order to properly skip invalid image:
    Integration tests are failing in Github runners on track/1.15 seldon-core-operator#199
    fix: removed not supported static image seldon-core-operator#186
  3. Katib: No PR and it looks like we cannot get rid of all those example images after all on track/0.15. They are used as default images in trial templates. It will be different story for main, because we handling them differently and we will have ROCKs. Only 3 images will remain on main.

i-chvets added a commit to canonical/kserve-operators that referenced this issue Aug 25, 2023
@i-chvets
fix: modified to retrieve images only from one charm (#155)
9f5ff2e 
# Description
`kserve-web-app` is not deployed as part of 1.7 release. Get images script incorrectly retrieves image from `kserve-web-app` charm which incorrectly shows up on report of deployable images.
This changes will address the issue by only scanning charm that is deployed in 1.7 release.

More details are in canonical/bundle-kubeflow#674

Summary of changes:
- Modified script to retrieve images only from kserve-controller, because kserve-web-app is not deployed in this release.

NOTE: This change is only applicable to track/0.10
@i-chvets i-chvets mentioned this issue Aug 31, 2023
Closed
@i-chvets
Copy link
Contributor Author

Katib issues that need to be handled to ensure reliable handling of default images:
canonical/katib-operators#130
canonical/katib-operators#131

@i-chvets i-chvets mentioned this issue Sep 1, 2023
Closed
@kimwnasptd
Copy link
Contributor

While not directly related, I'll add a link here regarding our effort to also remove the default template images for Katib canonical/katib-operators#132

@kimwnasptd
Copy link
Contributor

Will close this one once we back-port the changes to CKF 1.7 as well

@kimwnasptd
Copy link
Contributor

Closing this issue since we merged both PRs

@syncronize-issues-to-jira Syncronize issues to Jira
Copy link

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5137.

This message was autogenerated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
MLOps Solution Issues
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kimwnasptd @i-chvets