Identify quota changes needed for ingress jobs, create pool of ingress projects #1093

spiffxp · 2020-07-31T20:58:44Z

This is like #851 but for ingress

The default quotas for an e2e project (eg: k8s-infra-e2e-gce-project) MAY be insufficient to run ci-kubernetes-e2e-gci-gce-ingress

Currently this job runs in the google.com k8s-prow-builds cluster, using a project from that boskos' ingress-project pool

~~Identify whether ci-kubernetes-e2e-gci-gce-ingress can run against a stock e2e project~~ (it can! so this is much lower priority, since we don't need ingress projects for release/merge-blocking job migration)
Identify what quotas are set that make ingress-projects differ from a default e2e project
Identify what jobs this will allow us to migrate
Identify pool size
Provision pool in kubernetes.io org
Add pool to k8s-infra-prow-build's boskos

spiffxp · 2020-08-04T21:57:48Z

$ for prj in k8s-ingress-boskos-01 k8s-infra-e2e-ingress-project; do
  gcloud compute project-info describe --project=$prj > $prj.compute-project-info.yaml
done
$ diff -yW80 k8s-infra-e2e-ingress-project.compute-project-info.yaml k8s-ingress-boskos-01.compute-project-info.yaml | grep -A3 "|.*limit"
- limit: 21.0			      |	- limit: 50.0
  metric: STATIC_ADDRESSES		  metric: STATIC_ADDRESSES
  usage: 0.0				  usage: 0.0
- limit: 250.0				- limit: 250.0
--
- limit: 45.0			      |	- limit: 100.0
  metric: FORWARDING_RULES		  metric: FORWARDING_RULES
  usage: 0.0				  usage: 0.0
- limit: 150.0				- limit: 150.0
--
- limit: 150.0			      |	- limit: 55.0
  metric: HEALTH_CHECKS			  metric: HEALTH_CHECKS
  usage: 0.0				  usage: 0.0
- limit: 69.0			      |	- limit: 100.0
  metric: IN_USE_ADDRESSES		  metric: IN_USE_ADDRESSES
  usage: 0.0				  usage: 0.0
- limit: 150.0				- limit: 150.0
--
- limit: 30.0			      |	- limit: 50.0
  metric: TARGET_HTTP_PROXIES		  metric: TARGET_HTTP_PROXIES
  usage: 0.0				  usage: 0.0
- limit: 30.0			      |	- limit: 50.0
  metric: URL_MAPS			  metric: URL_MAPS
  usage: 0.0				  usage: 0.0
- limit: 9.0			      |	- limit: 100.0
  metric: BACKEND_SERVICES		  metric: BACKEND_SERVICES
  usage: 0.0				  usage: 0.0
- limit: 300.0				- limit: 300.0
--
- limit: 30.0			      |	- limit: 50.0
  metric: TARGET_HTTPS_PROXIES		  metric: TARGET_HTTPS_PROXIES
  usage: 0.0				  usage: 0.0
- limit: 30.0			      |	- limit: 50.0
  metric: SSL_CERTIFICATES		  metric: SSL_CERTIFICATES
  usage: 0.0				  usage: 0.0
- limit: 175.0				- limit: 175.0
--
- limit: 100.0			      |	- limit: 200.0
  metric: SECURITY_POLICY_RULES		  metric: SECURITY_POLICY_RULES
  usage: 0.0				  usage: 0.0
- limit: 45.0				- limit: 45.0
--
- limit: 300.0			      |	- limit: 1000.0
  metric: NETWORK_ENDPOINT_GROUPS	  metric: NETWORK_ENDPOINT_GROUPS
  usage: 0.0				  usage: 0.0
- limit: 6.0				- limit: 6.0
--
- limit: 15.0			      |	- limit: 50.0
  metric: EXTERNAL_VPN_GATEWAYS		  metric: EXTERNAL_VPN_GATEWAYS
  usage: 0.0				  usage: 0.0
- limit: 1.0				- limit: 1.0
--
- limit: 1024.0			      |	- limit: 128.0
  metric: STATIC_BYOIP_ADDRESSES	  metric: STATIC_BYOIP_ADDRESSES
  usage: 0.0				  usage: 0.0

spiffxp · 2020-08-04T22:25:53Z

Jobs that use this project type:

~~ci-cri-containerd-e2e-gci-gce-ingress~~
ci-ingress-gce-e2e
~~ci-ingress-gce-e2e-multi-zone~~
~~ci-ingress-gce-e2e-release-1-6~~
~~ci-kubemci-ingress-conformance~~
ci-kubernetes-e2e-gce-cos-k8sbeta-ingress
ci-kubernetes-e2e-gce-cos-k8sstable1-ingress
ci-kubernetes-e2e-gce-cos-k8sstable2-ingress
ci-kubernetes-e2e-gce-cos-k8sstable3-ingress
ci-kubernetes-e2e-gci-gce-ingress
ci-kubernetes-e2e-gci-gce-ingress-manual-network

spiffxp · 2020-08-04T22:30:24Z

/assign

spiffxp · 2020-08-04T22:44:29Z

Will use kubernetes/test-infra#18663 as a canary job, should be able to let it schedule (or re-run it) to see if I've gotten quotas correct

I'm starting with no increase in quotas at all

spiffxp · 2020-08-07T19:42:46Z

https://testgrid.k8s.io/sig-testing-canaries#gci-gce-ingress&width=20&graph-metrics=test-duration-minutes - so it, uh, looks like the release-blocking ingress job runs just fine with the default quotas

https://testgrid.k8s.io/sig-release-master-blocking#gci-gce-ingress&width=20&graph-metrics=test-duration-minutes - and comparing to the sig-release-master-blocking runs, it performs basically the same as well

So this task isn't strictly necessary to migrate release-blocking jobs. Are there any presubmits I'm missing?

spiffxp · 2020-08-31T21:54:57Z

There are no presubmits that use this

Remaining jobs that use this project type

ci-cri-containerd-e2e-gci-gce-ingress (containerd/cri + kubernetes periodic)
ci-ingress-gce-e2e (kubernetes periodic)
ci-ingress-gce-e2e-multi-zone (kubernetes periodic)
ci-ingress-gce-e2e-release-1-6 (kubernetes periodic)
ci-kubemci-ingress-conformance (GoogleCloudPlatform/k8s-multicluster-ingress)
ci-kubernetes-e2e-gci-gce-ingress-manual-network (kubernetes periodic)

Duplicate of ci-ingress-gce-e2e that validates the job can run on k8s-infra-prow-build. Ref: kubernetes/k8s.io#1093 Part of : kubernetes#18549 Signed-off-by: Arnaud Meukam <ameukam@gmail.com>

fejta-bot · 2020-11-29T22:32:16Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot · 2020-12-29T23:16:50Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

spiffxp · 2021-01-15T19:34:07Z

/remove-lifecycle rotten
We still don't know whether we need ingress projects. These are the jobs we should try migrating to confirm whether we do: #1093 (comment)

The approach I would recommend is switching a job to run on k8s-infra-prow-build, and switch from boskos (--gcp-project-type=) to manual (--gcp-project=k8s-infra-e2e-ingress-project`). If the job is high visibility-enough, consider a canary job instead. If it doesn't run due to quota issues, great, we've identified that we do need special ingress projects, and we can use the failure to determine what needs to be customized.

fejta-bot · 2021-04-22T23:22:14Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

ameukam · 2021-04-23T05:18:33Z

/remove-lifecycle stale

spiffxp · 2021-10-01T19:37:51Z

/remove-sig testing
/sig network

ameukam · 2021-12-14T22:09:39Z

/milestone clear

k8s-triage-robot · 2022-03-14T22:12:17Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

ameukam · 2022-03-15T05:19:03Z

/remove-lifecycle stale
/lifecycle frozen

ameukam · 2024-03-03T14:04:05Z

All ingress-gce related jobs have been migrated.

See:

Request: project information needed for ingress-gce jobs when migrating to k8s-infra-prow-build cluster test-infra#31757

/close
/milestone v1.30
/lifecycle clear

k8s-ci-robot · 2024-03-03T14:04:10Z

@ameukam: Closing this issue.

In response to this:

All ingress-gce related jobs have been migrated.

See:

Request: project information needed for ingress-gce jobs when migrating to k8s-infra-prow-build cluster test-infra#31757

/close
/milestone v1.30
/lifecycle clear

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

spiffxp added sig/release Categorizes an issue or PR as relevant to SIG Release. sig/testing Categorizes an issue or PR as relevant to SIG Testing. wg/k8s-infra area/prow Setting up or working with prow in general, prow.k8s.io, prow build clusters labels Aug 1, 2020

spiffxp added this to Backlog in CI Policy Improvements via automation Aug 1, 2020

k8s-ci-robot assigned spiffxp Aug 4, 2020

spiffxp mentioned this issue Aug 4, 2020

Add job to test out k8s-infra-e2e-ingress-project kubernetes/test-infra#18663

Merged

spiffxp moved this from Backlog to In Progress in CI Policy Improvements Aug 4, 2020

spiffxp mentioned this issue Oct 6, 2020

RFC: Move boskos testing projects pool to kubernetes.io #390

Closed

ameukam mentioned this issue Oct 29, 2020

Add canary job for ci-ingress-gce-e2e kubernetes/test-infra#19762

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 29, 2020

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 29, 2020

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jan 15, 2021

spiffxp added this to Needs Triage in sig-k8s-infra Jan 20, 2021

spiffxp added the priority/backlog Higher priority than priority/awaiting-more-evidence. label Jan 22, 2021

spiffxp moved this from Needs Triage to Backlog (infra to migrate) in sig-k8s-infra Feb 19, 2021

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 22, 2021

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 23, 2021

ameukam added this to the v1.23 milestone Jul 16, 2021

spiffxp added this to To Triage in sig-testing issues Jul 27, 2021

k8s-ci-robot added sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. and removed wg/k8s-infra labels Sep 29, 2021

k8s-ci-robot added sig/network Categorizes an issue or PR as relevant to SIG Network. and removed sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Oct 1, 2021

spiffxp removed this from To Triage in sig-testing issues Nov 24, 2021

k8s-ci-robot removed this from the v1.23 milestone Dec 14, 2021

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 14, 2022

k8s-ci-robot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Mar 15, 2022

ameukam mentioned this issue Sep 22, 2023

don't use custom projects for sig-network jobs kubernetes/test-infra#30791

Merged

k8s-ci-robot added this to the v1.30 milestone Mar 3, 2024

k8s-ci-robot closed this as completed Mar 3, 2024

sig-k8s-infra automation moved this from Backlog (infra to migrate) to Done Mar 3, 2024

CI Policy Improvements automation moved this from In Progress to Closed Mar 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Identify quota changes needed for ingress jobs, create pool of ingress projects #1093

Identify quota changes needed for ingress jobs, create pool of ingress projects #1093

spiffxp commented Jul 31, 2020 •

edited

Loading

spiffxp commented Aug 4, 2020

spiffxp commented Aug 4, 2020 •

edited by ameukam

Loading

spiffxp commented Aug 4, 2020

spiffxp commented Aug 4, 2020

spiffxp commented Aug 7, 2020

spiffxp commented Aug 31, 2020

fejta-bot commented Nov 29, 2020

fejta-bot commented Dec 29, 2020

spiffxp commented Jan 15, 2021

fejta-bot commented Apr 22, 2021

ameukam commented Apr 23, 2021

spiffxp commented Oct 1, 2021

ameukam commented Dec 14, 2021

k8s-triage-robot commented Mar 14, 2022

ameukam commented Mar 15, 2022

ameukam commented Mar 3, 2024

k8s-ci-robot commented Mar 3, 2024

Identify quota changes needed for ingress jobs, create pool of ingress projects #1093

Identify quota changes needed for ingress jobs, create pool of ingress projects #1093

Comments

spiffxp commented Jul 31, 2020 • edited Loading

spiffxp commented Aug 4, 2020

spiffxp commented Aug 4, 2020 • edited by ameukam Loading

spiffxp commented Aug 4, 2020

spiffxp commented Aug 4, 2020

spiffxp commented Aug 7, 2020

spiffxp commented Aug 31, 2020

fejta-bot commented Nov 29, 2020

fejta-bot commented Dec 29, 2020

spiffxp commented Jan 15, 2021

fejta-bot commented Apr 22, 2021

ameukam commented Apr 23, 2021

spiffxp commented Oct 1, 2021

ameukam commented Dec 14, 2021

k8s-triage-robot commented Mar 14, 2022

ameukam commented Mar 15, 2022

ameukam commented Mar 3, 2024

k8s-ci-robot commented Mar 3, 2024

spiffxp commented Jul 31, 2020 •

edited

Loading

spiffxp commented Aug 4, 2020 •

edited by ameukam

Loading