Add experimental Flatcar Linux arm64 support on AWS

* Add `arch` variable to Flatcar Linux AWS `kubernetes` and `workers` modules. Accept `amd64` (default) or `arm64` to support native arm64/aarch64 clusters or mixed/hybrid clusters with arm64 workers * Requires `flannel` or `cilium` CNI Similar to #875
poseidon · Jan 14, 2022 · beb9f14 · beb9f14
1 parent f544a9c
commit beb9f14
Show file tree

Hide file tree

Showing 9 changed files with 94 additions and 12 deletions.
diff --git a/CHANGES.md b/CHANGES.md
@@ -8,6 +8,15 @@ Notable changes between versions.
 
 * Switch Kubernetes Container Runtime from `docker` to `containerd` ([#1101](https://github.com/poseidon/typhoon/pull/1101))
 
+### Flatcar Linux
+
+#### AWS
+
+* Add experimental Flatcar Linux ARM64 support
+  * Add `arch` variable to AWS `kubernetes` and `workers` modules
+  * Allow arm64 full-cluster or mixed/hybrid cluster with arm64 workers
+  * Requires `flannel` or `cilium` CNI provider
+
 ## v1.23.1
 
 * Kubernetes [v1.23.1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#v1231)

diff --git a/README.md b/README.md
@@ -45,6 +45,10 @@ Typhoon is available for [Flatcar Linux](https://www.flatcar-linux.org/releases/
 | DigitalOcean | Flatcar Linux  | [digital-ocean/flatcar-linux/kubernetes](digital-ocean/flatcar-linux/kubernetes) | beta |
 | Google Cloud  | Flatcar Linux  | [google-cloud/flatcar-linux/kubernetes](google-cloud/flatcar-linux/kubernetes) | beta |
 
+| Platform      | Operating System | Terraform Module | Status |
+|---------------|------------------|------------------|--------|
+| AWS           | Flatcar Linux (ARM64) | [aws/flatcar-linux/kubernetes](aws/flatcar-linux/kubernetes) | alpha |
+
 ## Documentation
 
 * [Docs](https://typhoon.psdn.io)

diff --git a/aws/flatcar-linux/kubernetes/ami.tf b/aws/flatcar-linux/kubernetes/ami.tf
@@ -1,7 +1,7 @@
 locals {
   # Pick a Flatcar Linux AMI
   # flatcar-stable -> Flatcar Linux AMI
-  ami_id  = data.aws_ami.flatcar.image_id
+  ami_id  = var.arch == "arm64" ? data.aws_ami.flatcar-arm64[0].image_id : data.aws_ami.flatcar.image_id
   channel = split("-", var.os_image)[1]
 }
 
@@ -25,3 +25,25 @@ data "aws_ami" "flatcar" {
   }
 }
 
+data "aws_ami" "flatcar-arm64" {
+  count = var.arch == "arm64" ? 1 : 0
+
+  most_recent = true
+  owners      = ["075585003325"]
+
+  filter {
+    name   = "architecture"
+    values = ["arm64"]
+  }
+
+  filter {
+    name   = "virtualization-type"
+    values = ["hvm"]
+  }
+
+  filter {
+    name   = "name"
+    values = ["Flatcar-${local.channel}-*"]
+  }
+}
+
diff --git a/aws/flatcar-linux/kubernetes/variables.tf b/aws/flatcar-linux/kubernetes/variables.tf
@@ -160,6 +160,17 @@ variable "cluster_domain_suffix" {
   default     = "cluster.local"
 }
 
+variable "arch" {
+  type        = string
+  description = "Container architecture (amd64 or arm64)"
+  default     = "amd64"
+
+  validation {
+    condition     = var.arch == "amd64" || var.arch == "arm64"
+    error_message = "The arch must be amd64 or arm64."
+  }
+}
+
 variable "daemonset_tolerations" {
   type        = list(string)
   description = "List of additional taint keys kube-system DaemonSets should tolerate (e.g. ['custom-role', 'gpu-role'])"

diff --git a/aws/flatcar-linux/kubernetes/workers.tf b/aws/flatcar-linux/kubernetes/workers.tf
@@ -9,6 +9,7 @@ module "workers" {
   worker_count    = var.worker_count
   instance_type   = var.worker_type
   os_image        = var.os_image
+  arch            = var.arch
   disk_size       = var.disk_size
   spot_price      = var.worker_price
   target_groups   = var.worker_target_groups

diff --git a/aws/flatcar-linux/kubernetes/workers/ami.tf b/aws/flatcar-linux/kubernetes/workers/ami.tf
@@ -1,7 +1,7 @@
 locals {
   # Pick a Flatcar Linux AMI
   # flatcar-stable -> Flatcar Linux AMI
-  ami_id  = data.aws_ami.flatcar.image_id
+  ami_id  = var.arch == "arm64" ? data.aws_ami.flatcar-arm64[0].image_id : data.aws_ami.flatcar.image_id
   channel = split("-", var.os_image)[1]
 }
 
@@ -25,3 +25,24 @@ data "aws_ami" "flatcar" {
   }
 }
 
+data "aws_ami" "flatcar-arm64" {
+  count = var.arch == "arm64" ? 1 : 0
+
+  most_recent = true
+  owners      = ["075585003325"]
+
+  filter {
+    name   = "architecture"
+    values = ["arm64"]
+  }
+
+  filter {
+    name   = "virtualization-type"
+    values = ["hvm"]
+  }
+
+  filter {
+    name   = "name"
+    values = ["Flatcar-${local.channel}-*"]
+  }
+}
diff --git a/aws/flatcar-linux/kubernetes/workers/variables.tf b/aws/flatcar-linux/kubernetes/workers/variables.tf
@@ -119,3 +119,16 @@ variable "node_taints" {
   description = "List of initial node taints"
   default     = []
 }
+
+# unofficial, undocumented, unsupported
+
+variable "arch" {
+  type        = string
+  description = "Container architecture (amd64 or arm64)"
+  default     = "amd64"
+
+  validation {
+    condition     = var.arch == "amd64" || var.arch == "arm64"
+    error_message = "The arch must be amd64 or arm64."
+  }
+}
diff --git a/docs/advanced/arm64.md b/docs/advanced/arm64.md
@@ -1,12 +1,9 @@
 # ARM64
 
-!!! warning
-    ARM64 support is experimental
-
-Typhoon has experimental support for ARM64 with Fedora CoreOS on AWS. Full clusters can be created with ARM64 controller and worker nodes. Or worker pools of ARM64 nodes can be attached to an AMD64 cluster to create a hybrid/mixed architecture cluster.
+Typhoon has experimental support for ARM64 on AWS, with Fedora CoreOS or Flatcar Linux. Clusters can be created with ARM64 controller and worker nodes. Or worker pools of ARM64 nodes can be attached to an AMD64 cluster to create a hybrid/mixed architecture cluster.
 
 !!! note
-    Currently, CNI networking must be set to flannel or Cilium.
+    Currently, CNI networking must be set to `flannel` or `cilium`.
 
 ## Cluster
 
@@ -35,14 +32,14 @@ module "gravitas" {
 }
 ```
 
-Verify the cluster has only arm64 (`aarch64`) nodes.
+Verify the cluster has only arm64 (`aarch64`) nodes. For Flatcar Linux, describe nodes.
 
 ```
 $ kubectl get nodes -o wide
-NAME             STATUS   ROLES    AGE    VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                          KERNEL-VERSION            CONTAINER-RUNTIME
-ip-10-0-12-178   Ready    <none>   101s   v1.23.1   10.0.12.178   <none>        Fedora CoreOS 32.20201104.dev.0   5.8.17-200.fc32.aarch64   docker://19.3.11
-ip-10-0-18-93    Ready    <none>   102s   v1.23.1   10.0.18.93    <none>        Fedora CoreOS 32.20201104.dev.0   5.8.17-200.fc32.aarch64   docker://19.3.11
-ip-10-0-90-10    Ready    <none>   104s   v1.23.1   10.0.90.10    <none>        Fedora CoreOS 32.20201104.dev.0   5.8.17-200.fc32.aarch64   docker://19.3.11
+NAME             STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                        KERNEL-VERSION            CONTAINER-RUNTIME
+ip-10-0-21-119   Ready    <none>   77s   v1.23.1   10.0.21.119   <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
+ip-10-0-32-166   Ready    <none>   80s   v1.23.1   10.0.32.166   <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
+ip-10-0-5-79     Ready    <none>   77s   v1.23.1   10.0.5.79     <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
 ```
 
 ## Hybrid

diff --git a/docs/index.md b/docs/index.md
@@ -45,6 +45,10 @@ Typhoon is available for [Flatcar Linux](https://www.flatcar-linux.org/releases/
 | DigitalOcean | Flatcar Linux  | [digital-ocean/flatcar-linux/kubernetes](flatcar-linux/digitalocean.md) | beta |
 | Google Cloud  | Flatcar Linux  | [google-cloud/flatcar-linux/kubernetes](flatcar-linux/google-cloud.md) | beta |
 
+| Platform      | Operating System | Terraform Module | Status |
+|---------------|------------------|------------------|--------|
+| AWS           | Flatcar Linux (ARM64) | [aws/flatcar-linux/kubernetes](advanced/arm64.md) | alpha |
+
 ## Documentation
 
 * Architecture [concepts](architecture/concepts.md) and [operating-systems](architecture/operating-systems.md)