Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Start Publishing Staging Images to Artifact Registry #3961

Open
upodroid opened this issue Jul 12, 2022 · 17 comments · Fixed by #6905
Open

Start Publishing Staging Images to Artifact Registry #3961

upodroid opened this issue Jul 12, 2022 · 17 comments · Fixed by #6905
Assignees
@upodroid
Labels
area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects area/release-eng Issues or PRs related to the Release Engineering subproject priority/backlog Higher priority than priority/awaiting-more-evidence. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. sig/testing Categorizes an issue or PR as relevant to SIG Testing.

Comments

@upodroid
Copy link
Member

upodroid commented Jul 12, 2022 •
edited

Part of: #1343

Notes: https://gist.github.com/upodroid/a33723a7e1abc5e9c6fabc6b07e7aac0

When images are built after a PR is merged, they need to be pushed to Artifact Registry(AR).

Unlike Google Container Registry(GCR), you don't need a separate GCP project per staging project. The permission boundary is at the repository level instead of the project. This allows multiple registries to be created per project. Therefore, we will create a new project, start publishing images to it and delete the other projects after the transition.

We will need to do the following:
- create a new GCP project (k8s-artifacts-staging)
- create a docker image repository per project
- modify the jobs to push images to prow.
- Provision the infra with terraform. There is a separate issue open to deploy changes to terraform via Prow on merges.

Prod changes need to be done via the shell scripts which makes deploying staging via terraform kind of pointless.

Open questions:

  • Do we need to backfill the staging repository via gcrane?

@puerco

/area artifacts
/priority important-soon
/area release-eng
@upodroid upodroid added the sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. label Jul 12, 2022
@k8s-ci-robot k8s-ci-robot added area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. area/release-eng Issues or PRs related to the Release Engineering subproject labels Jul 12, 2022
@upodroid upodroid mentioned this issue Jul 13, 2022
Merged
@upodroid
Copy link
Member Author

After #3968 is merged, we need to do a few things:

  • repos that use ko to build images(k-sigs/bom,etc), modify KO_DOCKER_REPO variable to us-docker.pkg.dev/k8s-staging-bom/images/bom as an example
  • Modify image-builder to tweak the cloudbuild.yaml file

Bit unsure about other things.

@upodroid upodroid mentioned this issue Jul 13, 2022
Closed
@upodroid
Copy link
Member Author

Spoke about this at the sig-testing meeting. Will push it out to the future when GCR deprecation is actually announced by Google.

May want to standardize on Aritfact Registry eventually, but would require tracking down a number of staging registries and changing team push processes (with unclear payoff right now)

/priority backlog
/sig testing

@k8s-ci-robot k8s-ci-robot added priority/backlog Higher priority than priority/awaiting-more-evidence. sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Jul 26, 2022
@ameukam ameukam removed the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Aug 1, 2022
@ameukam
Copy link
Member

ameukam commented Aug 25, 2022

/milestone v1.26

@k8s-ci-robot k8s-ci-robot added this to the v1.26 milestone Aug 25, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 23, 2022
@ameukam
Copy link
Member

ameukam commented Dec 21, 2022

/remove-lifecycle stale
/milestone v1.27
/priority important-longterm

@k8s-ci-robot k8s-ci-robot added priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 21, 2022
@k8s-ci-robot k8s-ci-robot modified the milestones: v1.26, v1.27 Dec 21, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 21, 2023
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Apr 20, 2023
@ameukam
Copy link
Member

ameukam commented Jun 24, 2023

/remove-lifecycle rotten
/milestone v1.29

@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jun 24, 2023
@k8s-ci-robot k8s-ci-robot modified the milestones: v1.27, v1.29 Jun 24, 2023
@ameukam ameukam removed this from the v1.29 milestone Dec 8, 2023
@BenTheElder
Copy link
Member

Strawman:

  1. Create AR versions of all staging registries. These can be empty, but should have the same IAM access / names as the GCRs
  2. Ask subprojects to switch to the AR registry. They do so by:
    1. Switching new builds to push to AR (this will vary by how they're doing image push)
    2. Running a backfill task we provide (gcrane cp -R)
    3. Switching from GCR to AR in image promoter config

We should probably check in with release engineering about that last part.

We could alternately:

  1. Create AR versions of all staging registries.
  2. SIG K8s Infra spins up a job or jobs to continuously backfill from GCR to AR (grane cp -R ... potentially excessive API usage ...)
  3. SIG K8s Infra swaps over all the promoter manifests to promote from AR
  4. We ask subprojects to stop writing to GCR so we can wind down the grane sync for their repo.

The first plan is less resource intensive but more people-coordination heavy

@ameukam
Copy link
Member

ameukam commented Jan 18, 2024

Another alternative could be to transition existing GCR repos to AR repos: https://cloud.google.com/artifact-registry/docs/transition/setup-gcr-repo

The infrastructure changes would be minimal and there is little to be done by the community. The downsize of this approach is mostly cost. we will use multi-regional AR repositories.

@BenTheElder
Copy link
Member

I forgot about this! Nice! We should definitely estimate the cost delta of that approach ...

One other downside: skew versus newly created registries going forward.

@ameukam
Copy link
Member

ameukam commented Jan 18, 2024

One other downside: skew versus newly created registries going forward.

AR allow us to create AR repos using gcr.io (traffic is redirected from gcr.io to the AR repo) so I think we are fine until GCR is gone.

ameukam added a commit to ameukam/k8s.io that referenced this issue Feb 7, 2024
@ameukam
GCP: Ensure staging Artifact Registry repositories
9be7a53 
Related to:
  - kubernetes#3961

With the deprecation of GCR, we need to ensure we can provide Artifact
Registry repositories for new and existing subprojects in need of a
container registry repo.

Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
@ameukam ameukam mentioned this issue Feb 7, 2024
Merged
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 18, 2024
@BenTheElder
Copy link
Member

Maybe instead tracking in #1343

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 18, 2024
@upodroid
Copy link
Member Author

Not rotten

@dims dims mentioned this issue Jun 17, 2024
Merged
@dims
Copy link
Member

dims commented Jun 17, 2024

Need to go back and create etcd-manager staging bucket in #6897

slack conversation : https://kubernetes.slack.com/archives/CCK68P2Q2/p1718654806739639

@dims dims removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jun 17, 2024
This was referenced Jun 18, 2024
Merged
Merged
@ameukam ameukam reopened this Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@upodroid upodroid

Labels
area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects area/release-eng Issues or PRs related to the Release Engineering subproject priority/backlog Higher priority than priority/awaiting-more-evidence. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. sig/testing Categorizes an issue or PR as relevant to SIG Testing.
Projects
registry.k8s.io (SIG K8S Infra)
Status: Reporting, Audit & Logging
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Deploy new k8s-staging-images project borg-land/k8s.io
6 participants
@dims @BenTheElder @ameukam @k8s-ci-robot @upodroid @k8s-triage-robot