v1.5.0-beta.0
Pre-release🚨 This is a beta release. Use it only for testing purposes. If you find any bugs, file an issue.
Highlights
Some of the hightlights in this release are:
- MS preflight checks to improve cluster stability (behind feature gate).
- Support for concurrent MachineDeployment upgrades in classy clusters.
- Support for add-on providers in clusterctl.
- Performance improvements when using CAPI at scale, new dev-tools and a new in-memory infrastructure provider for scale testing.
⚠️ Deprecation warning
The API version v1alpha3 is no longer served in this release. API version v1alpha4 is deprecated and will be removed in v1.6.
Cluster API v1.4 compared to v1.5
See the Cluster-API book for an overview over relevant changes between Cluster API v1.4 and v1.5 for maintainers of providers and consumers of our Go API.
More details about the release
👌 Kubernetes version support
- Management Cluster: v1.24.x -> v1.27.x
- Workload Cluster: v1.22.x -> v1.27.x
Changes since v1.4.0
📈 Overview
- 285 new commits merged
- 4 breaking changes
⚠️ - 11 feature additions ✨
- 60 bugs fixed 🐛
⚠️ Breaking Changes
- CAPD: Use k8s v1.27.0 in quickstart docs and CAPD (#8518)
- clusterctl: Bump to CR v0.15.0 (#8007)
- clusterctl: Follow XDG Directory standard for config/data/... files (#6913)
- clusterctl: Stop serving v1alpha3 API types (#8549)
✨ New Features
- CAPBK: Use caching read for bootstrap config owner (#8867)
- ClusterClass: add support or concurrent MD upgrades in classy clusters (#8432)
- Devtools: hack/observability: Add Parca, Bump observability tools, Add Grafana scale dashboard (#8754)
- IPAM: Implements Getter interface for IPAddressClaim object (#8374)
- KCP/Core/CAPD: Add flags for configuring rate limits (#8579)
- MachineSet: MS preflight checks to improve cluster stability (#8595)
- clusterctl: Add in memory provider (#8799)
- clusterctl: Add support for add-on providers in clusterctl (#8472)
- clusterctl: feat: accept resource mutators in Move operation (#7966)
- e2e: ci: collect debug information about leftover processes (#8734)
- util: Move mergeMap to util (#8377)
🐛 Bug Fixes
- API: Ensure ownerReference apiVersions are always up to date (#8256)
- CAPBK: ignition: start kubeadm after network.target (#8772)
- CAPBK: set uninitialized taint only on worker nodes (#8358)
- CAPD: Add kind mapper (#8880)
- CAPD: CAPD: Implement watch filter (#8789)
- CAPD: CAPD: change the haproxy entrypoint to prevent getting stopped immediately after start (#8685)
- CAPD: Delegate CAPD port selection to the container runtime (#8642)
- CAPD: test/capd: fix kind mapper entry for v1.25.11 (#8914)
- CAPD: test/e2e fix fail-swap-on=false flag not being part of kind images anymore (#8767)
- CAPIM: fix cluster deletion in the in-memory API server (#8818)
- CAPIM: fix inmemory provider docker build (#8822)
- CAPIM: test/e2e/in-memory: set providerID after VM is provisioned (#8879)
- CI: Always build Kind images for upgrade tests (#8859)
- CI: Ensure ALWAYS_BUILD_KIND_IMAGES is defaulted correctly in ci script (#8873)
- CI: verifier: use pull_request instead of pull_request_target (#8635)
- ClusterCacheTracker: ClusterCacheTracker: Stop pod caching when checking workload cluster (#8850)
- ClusterClass: Drop uid from error messages (#8333)
- ClusterClass: Validate variables in defaulting webhook (#8332)
- ClusterClass: avoid errors when MHC and upgrade occur together in classy clusters (#8464)
- ClusterClass: topology controller should avoid unnecessary rollouts during upgrades (#8628)
- ClusterClass: update TopologyReconciled condition on cluster deletion (#8422)
- ClusterResourceSet: : Make sure the Kubernetes API Server service already created on remote cluster before applying ClusterResourceSets (#8048)
- ClusterResourceSet: ClusterResourceSet: continue applying when apply for a single cluster failed (#8611)
- Dependency: Update kindnetd and kindest/haproxy (#8469)
- Devtools: Update clean-charts make target for new paths (#8904)
- Devtools: tilt: set CR logger in tilt-prepare, fix allowed contexts (#8919)
- IPAM: ipam: fix gateway being required for IPAddress (#8506)
- KCP: Allow machine rollout if cert reconcile fails (#8711)
- KCP: Ensure nil-pointer check in KCP syncMachines (#8481)
- KCP: Prevent KCP to create many private keys for each reconcile (#8617)
- KCP: enable updates from CloudInit (mostly ubuntu) based KCPs to Ignition (flatcar) based KCPs (#8339)
- Machine: bug fix to set node interruptible label (#8297)
- Machine: fix node label propagation (#8427)
- MachinePool: Add node watcher to MachinePool controller (#8443)
- MachinePool: Fix MachinePool node taint patching (#8462)
- MachinePool: MachinePool: always patch owned conditions, fix GetTypedPhase, doc fixes (#8627)
- MachinePool: Update MachinePool bootstrap dataSecretName when bootstrap config changes (#8667)
- Testing: Catch failure when fetching component URLs (#8455)
- Testing: Fix flake in machine phases test (#8370)
- Testing: Fix flaky wait in CRS test (#8334)
- Testing: Use CreateAndWait in MP ScaleFromZero test (#8927)
- Testing: internal/machine-controller fix flakes in phases test (#8576)
- Testing: machine-controller: fix phase tests race condition in tests on lastUpdated field (#8484)
- clusterctl: Add soft ownership from clusters to ClusterResourceSetBinding (#8318)
- clusterctl: Ensure all GitHub releases are fetched when searching provider versions (#8240)
- clusterctl: Return error on infra cluster and control plane discovery (#8604)
- clusterctl: Use local kustomize version in create-local-repository.py (#8414)
- clusterctl: clusterctl: return early if release for latest tag does not exist yet (#8253)
- clusterctl: handle nil pointer in clusterctl describe (#8724)
- e2e: Adjust machinepool helper e2e timeout (#8739)
- e2e: Fix length check in MD helper (#8352)
- e2e: Ignore cert annotation in rollout test (#8761)
- e2e: Increase timeout for clusterclass rollout test (#8897)
- e2e: Increased assertClusterObjects timeout (#8758)
- e2e: Pin kindest/node images to known good versions in clusterctl upgrade tests (#8860)
- e2e: Remove non-functioning ginkgo.no-color flag (#8703)
- e2e: Revert "Add calico as e2e test CNI" (#8575)
- e2e: test/e2e check for machines being ready after provisioning on Runtime SDK test (#8625)
- e2e: test/e2e: use topology cluster-template for clusterctl upgrade mgmt cluster (#8311)
- e2e: test/framework fix docker pod log collector (#8634)
🌱 Others
- API: Deprecate v1alpha3 and v1alpha4 in CRDs (#8687)
- CAPBK: replace reflect deepequal in bootstrap kubeadm (#8266)
- CAPD: Automatically set kubelet args for capd (#8881)
- CAPD: CAPD: only ignore necessary kubeadm preflight errors (#7911)
- CAPIM: Add startup timeout to the in memory provider (#8831)
- CAPIM: Add watch to in-memory server multiplexer (#8851)
- CAPIM: CAPIM: Enable update for coreDNS and kube-proxy (#8899)
- CAPIM: Fixups for watch in in-memory apiServer (#8898)
- CAPIM: Improve etcd management in CAPIM (#8906)
- CAPIM: add ClusterClass support for in-memory provider (#8807)
- CAPIM: fix readme for in-memory provider (#8905)
- CAPIM: test/e2e/in-memory: enable unit tests (#8886)
- CAPIM: test/e2e/in-memory: improve performance by fixing locking issues (#8895)
- CAPIM: test/e2e/in-memory: sync in-memory provider ClusterClass with test (#8892)
- CAPIM: test/infra/inmemory: rename controller files (#8830)
- CI: Add area/dependency label for dependabot when bumping gomod (#8410)
- CI: Add dependency label to github actions update (#8845)
- CI: Switch cloudbuild.yaml to e2 machine type (#8428)
- CI: verifier: change back to pull_request_target to make it work again (#8637)
- ClusterCacheTracker: ClusterCacheTracker: improve error when workload cluster is not reachable (#8801)
- ClusterCacheTracker: Deprecate DefaultIndex usage and remove where not needed (#8855)
- ClusterCacheTracker: Use ClusterCacheTracker consistently (instead of NewClusterClient) (#8744)
- ClusterCacheTracker: Use rest config from ClusterCacheTracker consistently (#8894)
- ClusterClass: cluster/topology: use cached MD list in get current state (#8922)
- ClusterClass: deprecate rolloutAfter in cluster topology (#8324)
- ClusterClass: upgrading control plane should only be blocked if MD are upgrading (not just rolling out) (#8658)
- Core: Cache unstructured in Cluster, MD and MS controller (#8916)
- Core: Remove unnecessary requeues (#8743)
- Dependency: Bump EndBug/add-and-commit from 9.1.1 to 9.1.2 (#8584)
- Dependency: Bump EndBug/add-and-commit from 9.1.2 to 9.1.3 (#8621)
- Dependency: Bump actions/checkout from 3.3.0 to 3.4.0 (#8321)
- Dependency: Bump actions/checkout from 3.4.0 to 3.5.0 (#8389)
- Dependency: Bump actions/checkout from 3.5.0 to 3.5.2 (#8540)
- Dependency: Bump actions/checkout from 3.5.2 to 3.5.3 (#8837)
- Dependency: Bump actions/setup-go from 3.5.0 to 4.0.1 (#8664)
- Dependency: Bump docker/distribution to v2.8.2 (#8645)
- Dependency: Bump gcb-docker-gcloud from v20230424-910a2a439d to v20230522-312425ae46 (#8770)
- Dependency: Bump gcb-docker-gcloud image (#8570)
- Dependency: Bump github.com/go-logr/logr from 1.2.3 to 1.2.4 (#8461)
- Dependency: Bump github.com/onsi/ginkgo/v2 from 2.10.0 to 2.11.0 (#8891)
- Dependency: Bump github.com/onsi/ginkgo/v2 from 2.9.2 to 2.9.4 (#8622)
- Dependency: Bump github.com/onsi/ginkgo/v2 from 2.9.4 to 2.9.5 (#8666)
- Dependency: Bump github.com/onsi/ginkgo/v2 from 2.9.5 to 2.9.7 (#8792)
- Dependency: Bump github.com/onsi/ginkgo/v2 from 2.9.7 to 2.10.0 (#8839)
- Dependency: Bump github.com/onsi/gomega from 1.27.4 to 1.27.5 (#8390)
- Dependency: Bump github.com/onsi/gomega from 1.27.6 to 1.27.7 (#8715)
- Dependency: Bump github.com/onsi/gomega from 1.27.7 to 1.27.8 (#8841)
- Dependency: Bump github.com/prometheus/client_golang from 1.14.0 to 1.15.0 (#8541)
- Dependency: Bump github.com/prometheus/client_golang from 1.15.0 to 1.15.1 (#8623)
- Dependency: Bump github.com/prometheus/client_golang from 1.15.1 to 1.16.0 (#8890)
- Dependency: Bump github.com/spf13/cobra from 1.6.1 to 1.7.0 (#8502)
- Dependency: Bump github.com/spf13/viper from 1.15.0 to 1.16.0 (#8791)
- Dependency: Bump golang version (1.20.3 -> 1.20.4) (#8749)
- Dependency: Bump golang.org/x/oauth2 from 0.6.0 to 0.7.0 (#8503)
- Dependency: Bump golang.org/x/oauth2 from 0.7.0 to 0.8.0 (#8665)
- Dependency: Bump golang.org/x/oauth2 from 0.8.0 to 0.9.0 (#8889)
- Dependency: Bump golang.org/x/text from 0.8.0 to 0.9.0 (#8504)
- Dependency: Bump golang.org/x/text from 0.9.0 to 0.10.0 (#8840)
- Dependency: Bump golangci-lint to 1.52.1 and fix findings (#8331)
- Dependency: Bump golangci-lint to v1.51.2 (#8312)
- Dependency: Bump golangci/golangci-lint-action from 3.4.0 to 3.5.0 (#8790)
- Dependency: Bump golangci/golangci-lint-action from 3.5.0 to 3.6.0 (#8838)
- Dependency: Bump gomodules.xyz/jsonpatch/v2 from 2.2.0 to 2.3.0 (#8716)
- Dependency: Bump kind to v0.19.0 (#8681)
- Dependency: Bump kindnet and haproxy images to latest (#8676)
- Dependency: Update KIND to v0.20.0 (#8908)
- Dependency: Update cert manager to v1.11.1 (#8529)
- Dependency: Update cert-manager to v1.11.2 (#8638)
- Dependency: Update cert-manager to v1.12.0 (#8702)
- Dependency: Update cert-manager to v1.12.1 (#8748)
- Dependency: Update controller runtime to 0.14.6 (#8405)
- Dependency: Update controller tools to v0.11.4 (#8547)
- Dependency: Update conversion-gen version (v0.26.0 -> v0.27.1) (#8573)
- Dependency: Update docker to v20.10.24 (#8475)
- Dependency: Update kind to v0.18.0 (#8433)
- Dependency: Update kpromo to v3.5.2 (#8487)
- Dependency: Update kpromo to v3.6.0 (#8680)
- Dependency: Update kubebuilder envtest (1.26.0 -> 1.27.1) (#8599)
- Dependency: Upgrade golang version (1.19.6 -> 1.20.3) (#8527)
- Devtools: Add go.work to .gitignore and .dockerignore (#8155)
- Devtools: Align kind registry script with upstream (#8731)
- Devtools: Pin delve to match go minor version (#8725)
- Devtools: bump containerd to 1.6.18 in hack/tools (#8523)
- Devtools: hack/observability: Add Grafana state dashboard, improve metrics (#8834)
- Devtools: hack/observability: Bump Loki, Grafana and promtail charts (#7182)
- Devtools: hack/observability: add tempo (for traces) (#8920)
- Devtools: hack/observability: disable unused loki-gateway (#8794)
- Devtools: hack/observability: improve Grafana dashboards (#8861)
- Devtools: hack/observability: move charts so they are ignored by go mod (#8795)
- IPAM: Make IPAddressClaim.Status.AddressRef optional (#8493)
- KCP: Change endpoints to endpoint in the etcd client (#8909)
- KCP: Avoid panic if etcd server misbehaves (#8862)
- KCP: Fix noisy error log triggered by missing patch helper (#8863)
- KCP: Improve error log on scale up when Machine doesn't have a nodeRef (#8564)
- KCP: Use one workload cluster for reconcile (#8900)
- KCP: Remove v1alpha2 adoption logic from KCP (#8328)
- KCP: cache unstructured (#8913)
- KCP: drop redundant get machines (#8912)
- Machine: controller/machine: use unstructured caching client (#8896)
- Machine: optimize
reconcileInterruptibleNodeLabel
of machine controller (#8852) - Machine: use providerID string as-is (#8577)
- MachineDeployment: Drop the first return value from FindOldMachineSets (#8415)
- MachineSet: surface failed preflight checks on MachineSet in
MachinesCreated
condition (#8669) - Release: Add area label lookup for prefixes to release notes tool (#8780)
- Release: Add runtime extension components to release (#8438)
- Release: Bump kpromo to v3.5.1 (#8301)
- Release: Improve release notes formatting (#8826)
- Release: Make release notes generation concurrent (#8817)
- Runtime SDK: Update handler name validation rules (#8280)
- Testing: Add calico as e2e test CNI (#8420)
- Testing: Add dualstack e2e test (#8517)
- Testing: Add unit test for health check during creation of a new cluster accessor (#8269)
- Testing: Allow overriding gingko args in e2e tests (#8365)
- Testing: Fix typos in phrase "to exist" (#8688)
- Testing: Merge ownerReference tests with quick start (#8264)
- Testing: Update e2e upgrade test versions (#8361)
- Testing: Update version matrix for github workflows (#8398)
- Testing: Validating the warnings in tests (#8778)
- clusterctl: Add move annotation on objects for cluster move operation (#8322)
- clusterctl: Bump controller-tools to v0.12 (#8581)
- clusterctl: CAPIM: use consistent spelling for release artifacts (#8853)
- clusterctl: Fix spelling mistake in clusterctl comment (#8344)
- clusterctl: Update cert-manager to v1.12.2 (#8883)
- clusterctl: clusterctl: Add labels to OWNERS file (#8342)
- clusterctl: prepare main branch for v1.5 development (#8430)
- e2e: Add Scale e2e - development only (#8833)
- e2e: Disable fail-fast by default for e2e tests (#8548)
- e2e: E2E: autoscaler e2e test (#8514)
- e2e: E2e: Unhold CI packages before installing (#7910)
- e2e: Improve gomega fail handling in clusterClass rollout (#8769)
- e2e: Improve logging for MachinePool upgrade test (#8728)
- e2e: Move ClusterClass quickstart test from e2e-informing to e2e-main (#8765)
- e2e: Pin cgroup driver used in v0.3 and v0.4 templates (#8684)
- e2e: add E2E test for MachineSet Preflight checks (#8698)
- e2e: assert E2E error responses when waiting for MD nodes (#8501)
- e2e: e2e: log leftover processes to eventually detect zombies (#8662)
- e2e: modify scale e2e test defaults (#8902)
- e2e: test/e2e: Fixup dump kube-system pods (#8809)
- e2e: test/e2e: also dump workload cluster kube-system pods (#8800)
- e2e: test/e2e: cleanup test files (#8557)
- e2e: test/framework: add functions to collect infrastructure logs in tests (#8675)
- util: Rename internal/util/ssa util functions for better naming consistency (#8423)
- util: util: cache list calls in cluster to objects mapper (#8918)
📖 Additionally, there have been 67 contributions to our documentation and book. (#8901, #8924, #8854, #8843, #8782, #8802, #8812, #8805, #8760, #8798, #8787, #8779, #8781, #8775, #8763, #8762, #8753, #8740, #8729, #8719, #8701, #8712, #8686, #8699, #8673, #8661, #8651, #8632, #8630, #8612, #8596, #8587, #8593, #8580, #8597, #8613, #8552, #8559, #8554, #8521, #8520, #8509, #8508, #8511, #8510, #8454, #8279, #8446, #8447, #8293, #8439, #8419, #8383, #8355, #8309, #8416, #8375, #8397, #8308, #8327, #8363, #8319, #8284, #8351, #8307, #8252, #8288)
Thanks to all our contributors! 😊