Kubernetes's multi-cloud image registry.
For more details on the design and implementation see the source repo: https://github.com/kubernetes/registry.k8s.io
Most of the deployment configuration is in infra/gcp/terraform/k8s-infra-oci-proxy (staging instance for development only) and infra/gcp/terraform/k8s-infra-oci-proxy-prod (production / end users).
This directory is for tools and things that are used to administer the GCR/AR repositories used to publish official container images for Kubernetes.
Kubernetes subprojects may use a dedicated staging GCP project to build and host container images. We refer to the GCR provided by each staging project as a staging repository. Images are promoted from staging repositories into the main Kubernetes image registry (registry.k8s.io).
Access to each staging project is governed by a Google Group, which grants the ability to manually trigger GCB or push container images in the event that automated builds via something like prow.k8s.io are not setup or are broken.
The rule of thumb is that staging repositories should be used to host artifacts produced by code that is part of the Kubernetes project, as defined by presence in one of the project-owned GitHub Organizations.
In other words, code that is not part of the Kubernetes project should not have its artifacts hosted in staging repos. SIG K8s Infra may make exceptions to this policy on a case-by-case basis.
For example:
- CRI-O is not part of the kubernetes project, it does not meet the requirements to get a staging repo
- While etcd and coredns are not part of the kubernetes project, we do bundle them with kubernetes as part of the release, so for this specific case are allowing a staging repo to host them (solely within the context of the kubernetes project)
-
Add the project name to the
infra.staging.projects
list defined ininfra/gcp/infra.yaml
-
One your PR merges:
- a postsubmit job will create the necessary google group
- whoever approved your PR will run the necessary bash script(s) to create the staging repo
Once your staging repo is up and running, you can enable automatic build and push. For more info, see the instructions here
NOTE: All sub-projects are strongly encouraged to use this mechanism, though it is not mandatory yet. Over time this will become the primary way to build and push images, and anything else will become exceptional.
-
Enable automatic builds and ensure and image has been published by Cloud Build to the staging repo.
-
Create 3 files:
images/k8s-staging-<project-name>/OWNERS
images/k8s-staging-<project-name>/images.yaml
manifests/k8s-staging-<project-name>/promoter-manifest.yaml
The promoter-manifest.yaml
file will house the credentials and other registry
metadata, whereas the images.yaml
file will hold only the image data. You can
look at the existing staging repos for examples.
NOTE: For images/k8s-staging-<project-name>/images.yaml
, it must not be empty as an image must be proposed
for promotion when you are opening this PR.
The separation between promoter-manifest.yaml
and images.yaml
is there to
prevent a single PR from modifying the source registry information as well as
the image information. Any changes to the manifest/...
folder is expected to
be 1-time only during project setup.
Be sure to add the project owners to the
images/k8s-staging-<project-name>/OWNERS
file to increase the number of
people who can approve new images for promotion for your project.
Image promotion roughly follows the following steps:
- Push your image to one of the above staging docker repos e.g., gcr.io/k8s-staging-coredns
- Fork this git repo
- Add the image into the promoter manifest
e.g., if you pushed gcr.io/k8s-staging-coredns/foo:1.3, then add a "foo"
image entry into the manifest in
images/k8s-staging-coredns/images.yaml
- Create a PR to this git repo for your changes
- The PR should trigger a
pull-k8sio-cip
job which will validate and dry-run your changes; check that thek8s-ci-robot
responds 'Job succeeded' for it. - Merge the PR. Your image will be promoted by one of two jobs:
post-k8sio-image-promo
is a postsubmit that runs immediately after mergeci-k8sio-cip
is a periodic job that runs every four hours in case there are transient failures of the postsubmit jobs
- Published images will be available at registry.k8s.io
We've written some tooling to simplify the creation of image promotion pull requests, which is described in detail here.