Request: cgroup v2 node pool in community clusters

[AkihiroSuda]

Currently k8s-infra-prow-build and eks-prow-build-cluster are still using cgroup v1, although many users should have already switched to cgroup v2.
It is still possible to create a custom cluster with Boskos, but this is often complicated and hard to test on laptop.

Any chance to have cgroup v2 node pool in community clusters, or just have a new cluster dedicated for cgroup v2?

[AkihiroSuda]

/sig testing

[ameukam]

cc @BenTheElder

[kannon92]

In a sig-node meeting a few weeks back, we did talk about making cgroups v2 the default. cc @dchen1107 @SergeyKanzhelev

It is possible to get a cgroupv2 cluster but you may have to target it specifically. We have examples of cgroupv2 jobs.

[BenTheElder]

This is also
/sig k8s-infra

We have kubernetes/k8s.io#5276

In a sig-node meeting a few weeks back, we did talk about making cgroups v2 the default. cc @dchen1107 @SergeyKanzhelev

It is possible to get a cgroupv2 cluster but you may have to target it specifically. We have examples of cgroupv2 jobs.

This issue is about the cluster on which we run the prowjob pods, so that nested local clusters (like KIND, hack/local-up-cluster.sh) run on cgroupsv2.

We could create a cgroupsv2 node-pool or cluster on GKE and run some jobs there ¹, someone else from k8s infra will have to chime in for the EKS clusters. @xmudrii

Footnotes

1. https://cloud.google.com/kubernetes-engine/docs/how-to/node-system-config#cgroup-mode-options ↩

[dims]

fyi all the ci jobs wiht al2023 in their names are cgroupsv2

• https://testgrid.k8s.io/amazon-ec2-al2023
• https://testgrid.k8s.io/amazon-ec2-node

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[BenTheElder]

/remove-lifecycle stale

We're still going to want to sort this out at some point, eventually this will be the default even ...

[dims]

we have a lot of cgroupv2 jobs now ... https://prow.k8s.io/?job=*cgroupv2*

[BenTheElder]

we have a lot of cgroupv2 jobs now ... https://prow.k8s.io/?job=*cgroupv2*

Yes, and that's probably more than enough for now, but at some point the prow clusters will also wind up migrating, and we could make sure kind etc are tested that way.

For now, I guess the cloudy E2E are enough and we can track this kubernetes/k8s.io#5276 instead