-
Notifications
You must be signed in to change notification settings - Fork 904
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Use policy/v1 instead of policy/v1beta1 #2354
Conversation
✅ Deploy Preview for karpenter-docs-prod canceled.
|
we can remove 1.19 from our testing since it is out of the EKS support timeframe. 1.20 is still technically supported until November 2022. https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#:~:text=.-,Amazon%20EKS%20Kubernetes%20release%20calendar,-Note |
Isn't Karpenter independent from EKS? Being held back to EKS versions is not ideal for other providers and e.g Load Balancer Controller forked in order to support newer versions than EKS does. AWS CSI drivers, Cloud Controller Manager etc all keep up with the latest Kubernetes versions. How do we approach this? |
We're not specifically tied to EKS, but a lot of users on Karpenter today are using EKS so we need to be deliberate on trying not to break current users. |
I appreciate that. But if they stay on current Karpenter version that will continue to work. And that software keeps requiring newish versions of Kubernetes is nothing new. And as mention, you have the option to fork like Load Balancer Controller did. But kOps is looking to release 1.25 in the coming weeks, ideally without Karpenter behind a feature flag. And that required K8s 1.25 support. So how can we achieve this? |
If Kubernetes itself does not allow these API versions to coexist, I think likely we need to stay up to date with the latest Kubernetes version on principles of neutrality -- we can't allow EKS to block Kops adoption of Karpenter. Agree with @olemarkus that customers can still run older versions of Karpenter with older versions of Kubernetes. Essentially, we already have an implicit version matrix, and this is the first time we've run into it. |
Thanks. Yeah policy/v1beta1 is no longer served in 1.25 and no longer exists as part of the k8s.io/api package. See https://kubernetes.io/docs/reference/using-api/deprecation-guide/ Do you want me to do more regarding e.g documentation in this PR? |
kOps 1.25 is entering beta phase now and this is really blocking. How can we progress this PR? |
That's a good idea. I can add that matrix somewhere after this has merged and we know which specific karpenter version this goes into. |
881880f
to
54f06a0
Compare
54f06a0
to
6881766
Compare
Required for Karpenter to work on kubernetes 1.25+ Also bumps k8s libs to 1.22.13 because Evictions did not exists in policy/v1 before k8s 1.22
6881766
to
ea5fc35
Compare
How can we progress on this? |
Hey @olemarkus, sorry for the slow progress here. We've been having some internal discussions about this problem generally and are working on some longer term solutions to avoid this problem in the future. For the short term, we're decided to let this problem solve itself with time given the short timeframe, though we recognize that the underlying issue of version compatibility is unresolved and this may recur. On November 1 (https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html), this version of Kubernetes is deprecated. For customers scrambling to upgrade their 1.20 clusters, we want to minimize potential impact of an incompatible Karpenter version hindering the upgrade process and ultimately slowing this deprecation. Let's hold this PR until that deprecation occurs, and get it in as soon as possible afterwards. |
Any update? :) |
We're looking at taking this relatively soon but most likely won't get to the 1.25 upgrade until after the holidays since so many people are out of the office. This is definitely a priority when the majority of the team is back online in the new year now that 1.20 is formally deprecated by EKS |
Rebased over here: kubernetes-sigs/karpenter#150. @olemarkus, wanna toss a commit on so you get credit? |
@ellistarn too late I guess. |
Description
Switch from
policy/v1beta
topolicy/v1
to support Kubernetes 1.25+.Kubernetes 1.25 does no longer serve the v1beta1 API and breaks consolidation.
Also bumps Kubernetes libs to 0.22 as 0.21 does not have the types for the entire v1 API. This also brings in upgrade of controller-runtime and knative.
Since policy/v1 requires kubernetes 1.21, users will have to upgrade Kubernetes before upgrading Karpenter.
How was this change tested?
Does this change impact docs?
Release Note
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.