Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate non-Kubernetes repos off of prow.k8s.io #12863

Closed
13 of 15 tasks
cblecker opened this issue Jun 4, 2019 · 56 comments · Fixed by #32089
Closed
13 of 15 tasks

Migrate non-Kubernetes repos off of prow.k8s.io #12863

cblecker opened this issue Jun 4, 2019 · 56 comments · Fixed by #32089
Assignees
@BenTheElder
Labels
kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. sig/testing Categorizes an issue or PR as relevant to SIG Testing.
Projects
sig-k8s-infra sig-testing issues

Comments

@cblecker
Copy link
Member

cblecker commented Jun 4, 2019 •
edited by BenTheElder

Currently, the Kubernetes community prow instance (prow.k8s.io) is supporting a number of non-Kubernetes project repositories. We need to migrate these off the Kubernetes community infrastructure ahead of moving our instance onto project owned infrastructure.

We should also stop adding any new repos/orgs to the prow.k8s.io instance that aren't directly involved with the Kubernetes project.

Currently, I see the following repos/orgs with configuration in config.yaml/plugins.yaml:

cc: @kubernetes/k8s-infra-team @kubernetes/sig-testing @fejta @cjwagner @Katharine @krzyzacy @amwat @michelle192837
@cblecker cblecker added sig/testing Categorizes an issue or PR as relevant to SIG Testing. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. wg/k8s-infra labels Jun 4, 2019
@cblecker
Copy link
Member Author

cblecker commented Jun 4, 2019

cc @nikhita for client-go/unofficial-docs

@cblecker
Copy link
Member Author

cblecker commented Jun 4, 2019

cc @Random-Liu for containerd/cri

@nikhita
Copy link
Member

nikhita commented Jun 4, 2019 •
edited

cc @nikhita for client-go/unofficial-docs

This is not active and can be removed even right now. 👍

@krzyzacy
Copy link
Member

krzyzacy commented Jun 4, 2019

For google owned projects' owners: feel free to migrate to https://github.com/GoogleCloudPlatform/oss-test-infra instead

@cblecker
Copy link
Member Author

cblecker commented Jun 4, 2019

@krzyzacy Do we have owners/contacts for the google owned projects?

@krzyzacy
Copy link
Member

krzyzacy commented Jun 4, 2019

@cblecker you can check 😂 I think it can fall back to us, or maybe @spiffxp can also help coordinate?

@nikhita nikhita mentioned this issue Jun 4, 2019
Merged
@cblecker
Copy link
Member Author

cblecker commented Jun 4, 2019

Looking at the configs, we probably don't need to worry about cncf/apisnoop (CNCF project and only using meow/bark).

We also may want to keep helm/charts around as it is also a CNCF project and may provide valuable signal.

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 2, 2019
@cblecker
Copy link
Member Author

cblecker commented Sep 2, 2019

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 2, 2019
@cblecker cblecker added this to Needs Triage in sig-k8s-infra via automation Oct 20, 2019
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 1, 2019
@cblecker
Copy link
Member Author

cblecker commented Dec 2, 2019

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 2, 2019
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 1, 2020
@nikhita
Copy link
Member

nikhita commented Mar 1, 2020 via email

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 1, 2020
@spiffxp spiffxp mentioned this issue Mar 26, 2020
Closed
13 tasks
@spiffxp
Copy link
Member

spiffxp commented Mar 26, 2020

@clarketm posted the following over in #16974 (comment)

For each project we choose to migrate: a proposal to minimize risk of downtime is:

  • Phase 1: migrating project to a dedicated build cluster (e.g. Kubeflow Prow Migration)
  • Phase 2: setup bot, webhooks, transfigure, and port Prow config, plugins, kubeconfig and jobs to new instance (ideally with support from tools offered by Prow).
  • Phase 3: miscellaneous cleanup task: delete artifacts, redefine cluster, rerun, and bot permissions, and copy over historical job results.

Caveat: the above may not be comprehensive but it is what I have gathered from my experience with Prow migration thus far. Please provide input here or in doc.

@dims
Copy link
Member

dims commented Mar 26, 2020

/priority important-soon

@k8s-ci-robot k8s-ci-robot added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Mar 26, 2020
@dims dims mentioned this issue Apr 7, 2020
Merged
@dashpole dashpole mentioned this issue Apr 14, 2020
Closed
@riaankleinhans
Copy link

#26297 for https://github.com/GoogleCloudPlatform/k8s-multicluster-ingress

cc @Riaankl CNCF has a prow instance don't they, is there a reason apisnoop is on the k8s prow still?

This prow job is a release informing job that is not requiered at the moment. We can remove it.

@hh
Copy link
Member

hh commented May 26, 2022

The two cncf/apisnoop prow.k8s.io jobs are not part of the cncf/k8s-conformance Certification program, they are part of the Kubernetes #sig-arch Conformance Subproject to ensure #sig-release has CI-Signal to GA API promotions that are not covered by Conformance tests.

See https://github.com/kubernetes/community/tree/master/sig-architecture#conformance-definition-1

This underlying data is also used by apisnoop.cncf.io, which is not there for CNCF Conformance program, but for ensuring the Kubernetes technical debt collections/reductions continue to occur and that all new GA APIs include Conformance coverage.

It's my take that these are best served by the Kubernetes community prow.k8s.io instance. Anything related to the cncf.io/ck program and cncf/k8s-conformance are already run on prow.cncf.io.

@hh hh mentioned this issue May 26, 2022
Merged
@BenTheElder
Copy link
Member

The two cncf/apisnoop prow.k8s.io jobs are not part of the cncf/k8s-conformance Certification program, they are part of the Kubernetes #sig-arch Conformance Subproject to ensure #sig-release has CI-Signal to GA API promotions that are not covered by Conformance tests.

The apisnoop repo however runs on @k8s-ci-robot / prow.k8s.io, independent of jobs producing apisnoop data.

kubernetes-sigs/apisnoop#548 (comment)

@BenTheElder
Copy link
Member

BenTheElder commented May 26, 2022 •
edited

It is probably inadvisable to have k8s-ci-robot remain an org owner of github.com/cncf, IMHO, it already has the keys to a lot.

@hh
Copy link
Member

hh commented May 26, 2022

@BenTheElder removing all bot interactions other than trigger to support the K8s focus prow jobs here : https://github.com/kubernetes/test-infra/pull/26418/files#r883159775

@hh
Copy link
Member

hh commented May 26, 2022

Currently @k8s-ci-robot only admin perms on the cncf/apisnoop repo specifically not the cncf org.

We could reduce the perms lower than admin as we now only need a trigger for the jobs, not the bot interactions.

image

@BenTheElder
Copy link
Member

Why do we need trigger from the repo?
I don't see what these jobs are.

I found some image pushing jobs, but that also means that more generally this repo is dependent on SIG k8s infra infrastructure for GCB + image hosting.

Seems like you're saying this project is really a subproject of SIG architecture, is there a reason it's not in kubernetes-sigs?

@khareyash05
Copy link
Member

@BenTheElder is https://github.com/GoogleCloudPlatform/k8s-cluster-bundle open to work upon?

@BenTheElder
Copy link
Member

I'm not sure how much people who are not admins of the respective repos can work on this, you need admin access and support from go.k8s.io/oncall test-infra-oncall to handle the webhook transitions unfortunately.

@BenTheElder
Copy link
Member

BenTheElder commented Apr 27, 2023 •
edited

cc @aojea @mpherman2 @cjwagner @michelle192837

I think we should put a deadline on this and then just disable any remaining jobs after the deadline, they can be spun back up on a different prow later. I've filed bugs with all the remaining repos that have lingered for approximately 1 year now ...

We can even call it something generous like EOY 2023, but this really needs to be finished so we can look to migrate prow.k8s.io to the community infrastructure now that we have the GCP run rate more reasonable.

@mrbobbytables
Copy link
Member

+1 to deadline, and I think we should be more aggressive with the deadline than EOY. Like 3 months tops.

@BenTheElder
Copy link
Member

BenTheElder commented Apr 27, 2023 •
edited

I don't see prow.k8s.io migrating this year as we're still figuring out how to run CI on AWS and while the run rate is vastly improved we overspent earlier this year, so I'm ambivalent about how aggressive the deadline is. We have however had open bugs about this for about a year with each remaining project and this bug is nearly 4 years old.

I think we also need to revisit the exceptions permitted, if we're permitting apisnoop and containerd, then you might argue for cadvisor as a node dep.

@dims
Copy link
Member

dims commented Apr 27, 2023

+1 to document and have a process around exceptions.

@BenTheElder
Copy link
Member

Reached out to my best guess at current owners for cadvisor, rules_k8s, k8s-cluster-bundle.

ameukam added a commit to ameukam/test-infra that referenced this issue Jan 12, 2024
@ameukam
k8s-cluster-bundle: remove prowjobs
dc840e9 
Part of:
  - kubernetes#12863

Remove unused jobs

Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
@ameukam ameukam mentioned this issue Jan 12, 2024
Merged
@ameukam
Copy link
Member

ameukam commented Feb 2, 2024

Apisnoop is currently in transition to be Kubernetes sub-project:

@BenTheElder
Copy link
Member

Will be finishing this shortly https://groups.google.com/a/kubernetes.io/g/dev/c/p6PAML90ZOU

I think https://github.com/GoogleCloudPlatform/k8s-cluster-bundle is the main one remaining after api snoop, unless we change our minds about closely related projects like cAdvisor, containerd.

@BenTheElder BenTheElder self-assigned this Feb 27, 2024
@BenTheElder BenTheElder mentioned this issue Feb 27, 2024
Merged
@BenTheElder
Copy link
Member

#32089 should pretty much wrap this up

@BenTheElder BenTheElder added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. labels Feb 27, 2024
sig-k8s-infra automation moved this from Needs Triage to Done Feb 27, 2024
sig-testing issues automation moved this from Backlog to Done Feb 27, 2024
@BenTheElder
Copy link
Member

google/cadvisor#3116 is probably the one follow-up at this point, will track there.

@BenTheElder
Copy link
Member

Note: https://groups.google.com/a/kubernetes.io/g/dev/c/p6PAML90ZOU

@BenTheElder BenTheElder mentioned this issue Aug 12, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@BenTheElder BenTheElder

Labels
kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. sig/testing Categorizes an issue or PR as relevant to SIG Testing.
Projects
sig-k8s-infra
  
Done
sig-testing issues
  
Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove lingering prow config for non-Kubernetes repos BenTheElder/test-infra