Support Ubuntu 22.04 AMIs for EKS 1.29+

[jmdeal]

Description

Update:
Canonical has released 20.04 AMIs for Ubuntu 1.29 alongside the 22.04 AMIs. Users may continue to use the Ubuntu AMI family without pinning AMIs on 1.29 with 20.04. There is still ongoing discussion on how Karpenter should support 22.04.

What problem are you trying to solve?
Canonical has switched to Ubuntu 22.04 for their EKS optimized AMI for EKS 1.29+. Karpenter currently will only auto-discover 20.04 AMIs due to the new AMIs using a new SSM parameter. While it would be trivial to use the new SSM parameter for EKS 1.29+ and fallback to the 20.04 parameter for previous versions, this would force major Ubuntu version upgrades on users relying on Karpenter for AMI auto-discovery when they upgrade to EKS 1.29.

Currently, users that rely on AMI auto-discovery with the Ubuntu AMIFamily that upgrade 1.29 will fail to create new nodes since AMIs can't be discovered. If you are one of these users and want to upgrade before Karpenter introduces a long term strategy, you can pin your current AMI using amiSelectorTerms.

Example:

apiVersion: karpenter.k8s.aws/v1beta1
kind: EC2NodeClass
spec:
  amiFamily: Ubuntu
  # ...
  amiSelectorTerms:
    # The latest 20.04 amd64 and arm64 AMIs for EKS 1.28 in us-west-2 at time of writing
    - id: ami-04d00324ebf262399 
    - id: ami-03513159c48355642

Open Questions

• Are the changes between major versions minimal enough that an auto-upgrade can be considered safe? Or does a new AMIFamily and migration strategy need to be introduced?

How important is this feature to you?

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[jmdeal]

Discussed with some folks at Canonical, they'll be extending 20.04 (focal) support to EKS 1.29. Once these new AMIs are released Ubuntu users should be able to update to 1.29 without any modifications to their EC2NodeClass. At that point they can take their time migrating workloads from 20.04 to 22.04. I'll leave this issue open for tracking purposes until a release is made.

edit: forgot AMI family decisions were also in scope here, still discussing internally but currently leaning toward a new AMI family. Migration, unless you're willing to pin AMIs (in which case the current Ubuntu family will still work), will need to wait for a resolution here.

[erulabs]

@jmdeal is there a timeline or a github issue we can track for this? We're currently pinning AMIs but would love to move back to amiFamily = Ubuntu when possible. Thank you!

[jmdeal]

The 20.04 images for 1.29 have been released, you should be able to use the Ubuntu AMI family. There's still ongoing discussion among the maintainers on how best to support Ubuntu 22.04.

[dex4er]

#5861 was rejected; we still do not have support for Ubuntu 22.04 (nor incoming 24.04). New Karpenter 0.36 doesn't bring anything for that. Can you write when we can expect any solution, please?

[jonathan-innis]

New Karpenter 0.36 doesn't bring anything for that. Can you write when we can expect any solution, please

Our general recommendation at this point is to directly pin the AMI version for the latest ubuntu images and using the Custom AMIFamily to be able to have full control over the configuration -- it's still unclear whether we want to have direct support for the Ubuntu AMIFamilies going forward due to the number of times that the latest version of the Ubuntu image has broken people -- and for extended periods of time.

We're currently pinning AMIs but would love to move back to amiFamily = Ubuntu when possible

We generally recommend users to pin AMIs (particularly in production). If we can get SSM support in (#3657) and get userData templating (#5134), I wonder if there's even a need anymore to have dedicated families for different userData formats and SSM paths.

[jmdeal]

With the release of v1.0.0 Karpenter has dropped native support for Ubuntu AMI discovery. Ubuntu AMIs can still be used with Karpenter, but will need to be selected via AMI selector terms. There's more details in the docs about the migration path.