Tasks to bump to Kubernetes v1.29

[killianmuldoon]

This issue is tracking the tasks that should be implemented after the Kubernetes minor release has been released.

Tasks

Prerequisites:

• Decide which Cluster API release series will support the new Kubernetes version
  • If feasible we usually cherry-pick the changes back to the latest release series.

Supporting managing and running on the new Kubernetes version

This section contains tasks to update our book, e2e testing and CI to use and test the new Kubernetes version
as well as changes to Cluster API that we might have to make to support the new Kubernetes version. All of these
changes should be cherry-picked to all release series that will support the new Kubernetes version.

• Modify quickstart and CAPD to use the new Kubernetes release:
  • Bump the Kubernetes version in:
    • test/*: search for occurrences of the previous Kubernetes version
    • Tiltfile
  • Ensure the latest available kind version is used (including the latest images for this kind release)
    • Add new images in the kind mapper.go.
      • See the kind releases page for the list of released images.
    • Set new default image for the test framework
  • Verify the quickstart manually
  • Bump InitWithKubernetesVersion and WorkloadKubernetesVersion in clusterctl_upgrade_test.go
    • Note: Only bump for Cluster API versions that will support the new Kubernetes release.
  • Prior art: ✨ v1.28: Prepare quickstart, capd and tests for the new release including kind bump #9160
• Ensure the jobs are adjusted to provide test coverage according to our support policy:
  • For the main branch:
    • periodics:
      • Drop the oldest upgrade job as the oldest Kubernetes minor version is now out of support.
      • Add new upgrade job which upgrades from the previous to the new Kubernetes version.
    • periodics & presubmits:
      • Bump KUBERNETES_VERSION_MANAGEMENT of the e2e-mink8s job to the new minimum supported management cluster version.
      • Bump KUBEBUILDER_ENVTEST_KUBERNETES_VERSION of the test-mink8s jobs to the new minimum supported management cluster version.
      • Adjust the -latest upgrade job to upgrade from the new Kubernetes to the next Kubernetes version.
  • For the release branch of the latest supported Cluster API minor release:
    • periodics & presubmits:
      • Adust the -latest upgrade jobs to upgrade to the new Kubernetes version instead of latest.
  • Note: Also check if ETCD_VERSION_UPGRADE_TO or COREDNS_VERSION_UPGRADE_TO needs to change for the upgrades jobs to the new or next Kubernetes version.
    • For etcd, see the DefaultEtcdVersion kubeadm constant: e.g. for v1.28.0
    • For coredns, see the CoreDNSVersion kubeadm constant:e.g. for v1.28.0
  • Prior art: CAPI: bump jobs for v1.28 kubernetes/test-infra#30347 CAPI: fix tabnames and release-1.4 upgrade presubmit kubernetes/test-infra#30406 CAPI fix main mink8s presubmit to also refer the correct KUBERNETES_VERSION_MANAGEMENT kubernetes/test-infra#30407
• Update book:
  • Update supported versions in versions.md
  • Update job documentation in jobs.md
  • Prior art: 📖 book: Update versions doc adding Kubernetes 1.28 + prowjob documentation #9161
• Issues specific to the Kubernetes minor release:
  • RBAC kubeadm:cluster-admins must be applied before CAPI triggers the upgrade to kubeadm 1.29 machines. (assuming change is merged for 1.29. 🌱 Support admin config for Kubeadm v1.29 #9682

Using new Kubernetes dependencies

This section contains tasks to update Cluster API to use the latest Kubernetes Go dependencies and related topics
like using the right Go version and build images. These changes are only made on the main branch. We don't
need them in older releases as they are not necessary to manage workload clusters of the new Kubernetes version or
run the Cluster API controllers on the new Kubernetes version.

• Ensure there is a new controller-runtime minor release which uses the new Kubernetes Go dependencies.

• Update our Prow jobs for the main branch to use the correct kubekins-e2e image

  • It is recommended to have one PR for presubmit and one for periodic jobs to reduce the risk of breaking the periodic jobs.
  • PRs: CAPI: bump main presubmits kubekins image to v1.29 kubernetes/test-infra#31450 CAPI: bump main presubmits kubekins image to v1.29 kubernetes/test-infra#31450
  • Prior art: presubmit jobs: Upgrade cluster-api kubekins-e2e version (v20220829-d5650c79bf-1.24 -… kubernetes/test-infra#27311
  • Prior art: periodic jobs: Upgrade cluster-api kubekins-e2e version for periodics-main (v2022082… kubernetes/test-infra#27326

• Bump the Go version in Cluster API: (if Kubernetes is using a new Go minor version)

  • Search for the currently used Go version across the repository and update it
  • We have to at least modify it in: hack/ensure-go.sh, .golangci.yml, cloudbuild*.yaml, go.mod, Makefile, netlify.toml, Tiltfile
  • PR: 🌱 Bump Go to 1.21.5 #9900
  • Prior art: 🌱 Upgrade golang version (1.18.3 -> 1.19.0) #7135

• Bump controller-runtime ⚠️ Bump to controller-runtime v0.17.0 & k8s.io v0.29 #9964

• Bump controller-tools 🌱 Bump controller-tools to v0.14 #9987

• Bump the Kubernetes version used in integration tests via KUBEBUILDER_ENVTEST_KUBERNETES_VERSION in Makefile

  • Note: This PR should be cherry-picked as well. It is part of this section as it depends on kubebuilder/controller-runtime
    releases and is not strictly necessary for Supporting managing and running on the new Kubernetes version.
  • PRs: 🌱 Bump kubebuilder envtest to 1.29.0 #10013 [release-1.6] 🌱 Bump kubebuilder envtest to 1.29.0 #10014
  • Prior art: 🌱 Update kubebuilder envtest (1.24.2 -> 1.25.0) #7193

• Bump conversion-gen via CONVERSION_GEN_VER in Makefile

  • PR: 🌱 Bump conversion-gen to v0.29.0 #10012
  • Prior art: 🌱 Update conversion-gen version (v0.23.1 -> v0.25.0) #7118

• ensure versions.md on main & release-1.6 is up-to-date after 🌱 Provide faster support for new k8s minor releases #9971

  • PR: [release-1.6] 📖 Sync version and release docs with main #10024

[killianmuldoon]

Created now mainly to keep track of kubernetes/enhancements#4302

/triage accepted

[chrischdi]

/assign

[furkatgofurov7]

@chrischdi Hey, one question: am I correct to assume that once this is completed, we will be supporting k8s 1.29 in the release-1.6 branch, i.e starting from v1.6.x, although there is a note in the book:

When a new Kubernetes minor release is available, we will try to support it in an upcoming Cluster API patch release (although only in the latest supported Cluster API minor release). But this depends on the changes made in Kubernetes, if the corresponding required changes in Cluster API are too invasive we won’t backport the support and users have to wait for the next Cluster API minor release.

meaning it might not be the case always?

[chrischdi]

Yes it is. With this issue we should also add the necessary jobs for the v1.6 release to get test coverage.

Especially via this part:

• For the release branch of the latest supported Cluster API minor release:
  • periodics & presubmits:
    • Adust the -latest upgrade jobs to upgrade to the new Kubernetes version instead of latest.

If any fixes are required they should get cherry-picked and released with the next round of patch releases (although normally there is no fix necessary).

[chrischdi]

Also note:

All of these changes should be cherry-picked to all release series that will support the new Kubernetes version.

[kashifest]

@chrischdi is there any timeline for the support for k8s v1.29 in CAPI, I think k8s v1.29 lands tomorrow, so can we expect the support in CAPI branches before the xmas or after? And also if theres any other blocker for example controller-runtime release? Meanwhile if any support is needed for the subtasks listed here in the issue, we can help.

[chrischdi]

@chrischdi is there any timeline for the support for k8s v1.29 in CAPI, I think k8s v1.29 lands tomorrow, so can we expect the support in CAPI branches before the xmas or after? And also if theres any other blocker for example controller-runtime release? Meanwhile if any support is needed for the subtasks listed here in the issue, we can help.

I want this to land (if possible) before xmas, however a supporting release will be v1.6.1 which is planned for January afaik.

[chrischdi]

Prerequisites:

• Decide which Cluster API release series will support the new Kubernetes version
  • If feasible we usually cherry-pick the changes back to the latest release series.

All work will be cherry-picked to release-1.6 so hopefully the upcoming release v1.6.1 will be the first one officially supporting Kubernetes v1.29

[chrischdi]

Sumarizing the above open tasks:

Supporting managing and running on the new Kubernetes version

• (@chrischdi) Modify quickstart and CAPD to use the new Kubernetes release:

  • Waiting for kubernetes release and kind images
  • Waiting for 🌱 Prep main branch for v1.7 development #9799
  • PR: ✨ v1.29: Prepare quickstart, capd and tests for the new release including kind bump #9872
  • PR release-1.6: ✨ [release-1.6] v1.29: Prepare quickstart, capd and tests for the new release including kind bump #9890

• (@chrischdi) Ensure the jobs are adjusted to provide test coverage according to our support policy:

  • Waiting for (above) ✨ v1.29: Prepare quickstart, capd and tests for the new release including kind bump #9872
  • Draft PR: CAPI: bump jobs for v1.29 kubernetes/test-infra#31441

• (@chrischdi) Update book:

  • Waiting for 📖 Update CAPI support and guarantees for v1.7 #9867
  • Waiting for CAPI: bump jobs for v1.29 kubernetes/test-infra#31441
  • 📖 book: Update versions doc adding Kubernetes 1.29 and prowjob documentation #9897

• Issues specific to the Kubernetes minor release:

  • no open todo's currently, but we will see when the tests are running

Using new Kubernetes dependencies

After release of v1.6.1 (and if v1.6.1 supports Kubernetes v1.29):

• bump WorkloadKubernetesVersion for test When testing clusterctl upgrades using ClusterClass (v1.6=>current) [ClusterClass]

After release-28 kubekins image includes go 1.21:

• [-] bump golang on release-1.6 and release-1.5 branches
  • Moved to: Bump release-1.6 & release-1.5 to Go 1.21 #10025

[kashifest]

@chrischdi are waiting for controller-runtime and controller-tools release on this ?

[sbueringer]

Yes.

[kashifest]

Do we have any expected timeline ?

[sbueringer]

No

[sbueringer]

Do we have any expected timeline ?

@kashifest CR has been released and bump PR is now ready for review #9964

[sbueringer]

Looks like we didn't add 1.29 to versions.md on release-1.6 yet (probably intentional).

We should make sure versions.md is updated once #9971 merges on main

(added a sub-task)

[sbueringer]

@chrischdi I think now we only have one sub-task for docs and "Bump golang on release-1.6 and release-1.5 branches"

I would move the latter to a separate issue so we can close the current one, WDYT?

[chrischdi]

Sounds good 👍

[sbueringer]

Moved to #10025

Otherwise we're done. Thx everyone :)

/close

[k8s-ci-robot]

@sbueringer: Closing this issue.

In response to this:

Moved to #10025

Otherwise we're done. Thx everyone :)

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.