Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Image gathering scripts for main branches #679

Closed
18 tasks done
kimwnasptd opened this issue Aug 27, 2023 · 7 comments
Closed
18 tasks done

Image gathering scripts for main branches #679

kimwnasptd opened this issue Aug 27, 2023 · 7 comments
Assignees
@i-chvets
Labels
23.10 Should be fixed by 23.10 enhancement New feature or request

Comments

@kimwnasptd
Copy link
Contributor

kimwnasptd commented Aug 27, 2023 •
edited
Loading

Following our effort to test CKF on airgapped environments #678 we'll need to make sure our image gathering scripts work for main branches as well. The goal is to be able to automatically gather all the images of a latest bundle.

Right now our scripts expect specific metadata to be present in a bundle
https://github.com/canonical/kubeflow-ci/blob/c0913a5962611eb3accd4d4ec174cd04fb78b76c/scripts/images/get-all-images.sh#L10

This information, right now, is only present in our 1.7 bundle and specifically in the stable risk

bundle-kubeflow/releases/1.7/stable/kubeflow/bundle.yaml

Line 10 in 47cfb9f

_github_repo_branch: track/1.7

We'll need to extend the rest of the bundle definitions, in latest to make sure we can grab images list from the edge bundles automatically

List of repositories and corresponding PRs:

Related issues:
Katib issues to resolve for images scripts:
canonical/katib-operators#130
canonical/katib-operators#131
canonical/katib-operators#132

Kserve issue to resolve for images scripts:
canonical/kserve-operators#161
canonical/kserve-operators#162
@kimwnasptd
Copy link
Contributor Author

Also, we'll need to ensure we copy the get-images.sh scripts, that leave in each charm repo, to the main branches.

Right now those scripts only live in the track branches:
https://github.com/canonical/notebook-operators/blob/track/1.7/tools/get-images.sh

This was referenced Aug 28, 2023
Open
Merged
@i-chvets i-chvets self-assigned this Aug 28, 2023
@i-chvets
Copy link
Contributor

i-chvets commented Aug 28, 2023 •
edited
Loading

For updating bundle with repository references and branch names plan is to update releases/latest/edge/bundle.yaml. Note that releases/latest/beta/bundle.yaml will be unaffected by this change. It will be updated at lates stage based on edge/bundle.yaml
Branch names in all cases will be either main or master depending on what repository is using (all CKF repositories are using main, but some charms from other teams might use different branch name).
Scan images workflow will be modified to include latest/edge and latest/beta.

i-chvets pushed a commit that referenced this issue Aug 28, 2023
feat: update bundle with branches
7d51643 
#679

Summary of changes:
- Added branches to latest/edge/bundle.yaml. This is required to enable
  image retrieval scripts to properly checkout source code.
- Updated MySQL K8S charm to point to 8.0/edge
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/knative-operators that referenced this issue Aug 28, 2023
feat: get images on main
4532839 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charms
  on this repository and/or by corresponding workloads. Image list is a
combindation of static and dynamic lists.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/admission-webhook-operator that referenced this issue Aug 28, 2023
feat: get images on main
505f044 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is dynamic list.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/argo-operators that referenced this issue Aug 28, 2023
feat: get images on main
8ab55b6 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
- Removed outdated get-images-1.7-stable.sh script.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/dex-auth-operator that referenced this issue Aug 28, 2023
feat: get images on main
9caf554 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/istio-operators that referenced this issue Aug 28, 2023
feat: get images on main
04bd95a 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charms in this repository and/or by corresponding workloads. Image list is a combination of static and dynamic lists.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/kfp-operators that referenced this issue Aug 28, 2023
feat: get images on main
eb93667 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/kubeflow-dashboard-operator that referenced this issue Aug 28, 2023
feat: get images on main
646230e 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/kubeflow-profiles-operator that referenced this issue Aug 28, 2023
feat: get images on main
478af34 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/kubeflow-volumes-operator that referenced this issue Aug 28, 2023
feat: get images on main
58cad6a 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/metacontroller-operator that referenced this issue Aug 28, 2023
feat: get images on main
afd3f08 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/minio-operator that referenced this issue Aug 28, 2023
feat: get images on main
5b660ea 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
@i-chvets i-chvets mentioned this issue Aug 28, 2023
Merged
i-chvets pushed a commit to canonical/oidc-gatekeeper-operator that referenced this issue Aug 28, 2023
feat: get images on main
46b217a 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
i-chvets added a commit to canonical/oidc-gatekeeper-operator that referenced this issue Aug 31, 2023
@i-chvets
feat: get images on main (#106)
5ee3773 
For more details refer to: canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.

NOTE: Script replaced outdated get-images-1.7-stable.sh script.
i-chvets added a commit to canonical/kubeflow-tensorboards-operator that referenced this issue Aug 31, 2023
@i-chvets
feat: get images on main (#90)
7132b6e 
For more details refer to: canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.

NOTE: Script replaced outdated get-images-1.7-stable.sh script.
@i-chvets i-chvets mentioned this issue Aug 31, 2023
Closed
i-chvets added a commit to canonical/dex-auth-operator that referenced this issue Sep 1, 2023
@i-chvets
feat: get images on main (#163)
e9a54e0 
For more details refer to the following issue: canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.

NOTE: Script replaced outdated get-images-1.7-stable.sh script.
i-chvets added a commit to canonical/admission-webhook-operator that referenced this issue Sep 1, 2023
@i-chvets
feat: get images on main (#106)
683f828 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is dynamic list.
i-chvets added a commit to canonical/kubeflow-profiles-operator that referenced this issue Sep 1, 2023
@i-chvets
feat: get images on main (#147)
9cabbac 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
@i-chvets i-chvets mentioned this issue Sep 1, 2023
Closed
@i-chvets
Copy link
Contributor

i-chvets commented Sep 1, 2023

Katib issues to resolve for images scripts:
canonical/katib-operators#130
canonical/katib-operators#131
canonical/katib-operators#132

Kserve issue to resolve for images scripts:
canonical/kserve-operators#161
canonical/kserve-operators#162

i-chvets added a commit to canonical/seldon-core-operator that referenced this issue Sep 2, 2023
@i-chvets
Kf 4233 feat get images script main (#204)
f15fd7a 
For more details refer to: canonical/bundle-kubeflow#679
    
Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.

NOTE: Script replaced outdated get-images-1.7-stable.sh script.
i-chvets added a commit to canonical/notebook-operators that referenced this issue Sep 2, 2023
@i-chvets @phoevos
feat: get images on main (#289)
7c91c69 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.

Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
i-chvets added a commit to canonical/minio-operator that referenced this issue Sep 2, 2023
@i-chvets @phoevos
feat: get images on main (#145)
42fef35 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.

Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
i-chvets added a commit to canonical/argo-operators that referenced this issue Sep 2, 2023
@i-chvets
feat: get images on main (#124)
cd6890c 
* feat: get images on main
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
- Removed outdated get-images-1.7-stable.sh script.
phoevos pushed a commit to canonical/training-operator that referenced this issue Sep 2, 2023
@phoevos
feat: get images on main
d902bef 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
i-chvets added a commit to canonical/training-operator that referenced this issue Sep 2, 2023
@i-chvets
feat: get images on main (#127)
36de342 
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
i-chvets added a commit to canonical/metacontroller-operator that referenced this issue Sep 2, 2023
@i-chvets
feat: get images on main (#79)
e1850b8 
* feat: get images on main
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
- Update tools/get-images.sh
---------
Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
@NohaIhab
Copy link
Contributor

NohaIhab commented Sep 4, 2023

This needs to be done for Katib as well. Instead of using a static list in the script as mentioned in #689 , we can create a similar strategy as kserve canonical/kserve-operators#163

@NohaIhab NohaIhab mentioned this issue Sep 4, 2023
Closed
phoevos pushed a commit to canonical/kfp-operators that referenced this issue Sep 4, 2023
@i-chvets
feat: get images on main (#312)
2227982 
Add script that produces a dynamic list of container images managed by charm in
this repository.

Refs canonical/bundle-kubeflow#679
orfeas-k added a commit to canonical/knative-operators that referenced this issue Sep 5, 2023
@i-chvets @orfeas-k
feat: get images on main (#141)
a22dd6e 
Add a script that produces list of container images managed by charms on this repository and/or by corresponding workloads.
Refs canonical/bundle-kubeflow#679
Closes #142 

Co-authored-by: Orfeas Kourkakis <orfeas.kourkakis@canonical.com>
@NohaIhab NohaIhab mentioned this issue Sep 5, 2023
Merged
@i-chvets i-chvets mentioned this issue Sep 5, 2023
Merged
@kimwnasptd
Copy link
Contributor Author

Marking this as done as we merged all scripts in main and generated the full list of images

misohu pushed a commit to canonical/seldon-core-operator that referenced this issue Sep 26, 2023
@i-chvets @misohu
Kf 4233 feat get images script main (#204)
c3439d9 
For more details refer to: canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.

NOTE: Script replaced outdated get-images-1.7-stable.sh script.
DnPlas pushed a commit to canonical/knative-operators that referenced this issue Sep 26, 2023
@i-chvets @orfeas-k @DnPlas
feat: get images on main (#141)
4e7c259 
Add a script that produces list of container images managed by charms on this repository and/or by corresponding workloads.
Refs canonical/bundle-kubeflow#679
Closes #142

Co-authored-by: Orfeas Kourkakis <orfeas.kourkakis@canonical.com>
orfeas-k pushed a commit to canonical/metacontroller-operator that referenced this issue Sep 26, 2023
@i-chvets @orfeas-k
feat: get images on main (#79)
208a83e 
* feat: get images on main
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
- Update tools/get-images.sh
---------
Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
orfeas-k pushed a commit to canonical/metacontroller-operator that referenced this issue Sep 26, 2023
@i-chvets @orfeas-k
feat: get images on main (#79)
4cef054 
* feat: get images on main
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
- Update tools/get-images.sh
---------
Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
misohu added a commit to canonical/seldon-core-operator that referenced this issue Sep 27, 2023
@misohu @i-chvets @phoevos
Kf 4446 Backport airgap support (#213)
a557cbb 
* Add customizable images from config

* Add config option for custom images for air gaped environment

* Kf 4233 feat get images script main (#204)

For more details refer to: canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.

NOTE: Script replaced outdated get-images-1.7-stable.sh script.

* fix: Parse custom images with ":" in their name (#206)

For some of the images, the charm attempts to separate the image name
from the tag, splitting the string in two using `:` as a separator.
However, this breaks the installation if the image name contains that
special character, which could be the case if a local container registry
address is provided for instance (e.g.
`172.17.0.2:5000/tensorflow/serving:2.1.0`).

Use `rsplit` to ensure that the rightmost part of the string (following
the last `:`) is parsed as the tag.

Signed-off-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>

---------

Signed-off-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
Co-authored-by: i-chvets <113444075+i-chvets@users.noreply.github.com>
Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
DnPlas pushed a commit to canonical/metacontroller-operator that referenced this issue Aug 16, 2024
@i-chvets @DnPlas
feat: get images on main (#79)
e044615 
* feat: get images on main
canonical/bundle-kubeflow#679

Summary of changes:
- Added script that produces list of container images managed by charm in this repository. Image list is a dynamic list.
- Update tools/get-images.sh
---------
Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@i-chvets i-chvets

Labels
23.10 Should be fixed by 23.10 enhancement New feature or request
Projects
MLOps Solution Issues
Status: Done
Milestone
No milestone
Development

No branches or pull requests
4 participants
@kimwnasptd @NohaIhab @phoevos @i-chvets