From 94d06e3d4b28dbdf214c44b760de1073793b1a77 Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 14:59:11 -0700 Subject: [PATCH 01/13] docs: Add helm-release document to explain the different options available --- docs/helm-release.md | 95 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 docs/helm-release.md diff --git a/docs/helm-release.md b/docs/helm-release.md new file mode 100644 index 00000000..ca7cab15 --- /dev/null +++ b/docs/helm-release.md @@ -0,0 +1,95 @@ +# Helm Release Add-ons + +Starting with [EKS Blueprints v5](https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/main/docs/v4-to-v5/motivation.md) we have made a decision to only support the provisioning of a certain core set of [add-ons](./addons/). On an going basis, we will evaluate the current list to see if more add-ons need to be supported via this repo. Typically you can expect that any AWS created add-on that is not yet available via the [Amazon EKS add-ons](./amazon-eks-addons.md) will be prioritized to be provisioned through this repository. + +In addition to these AWS add-ons, we will also support the provisioning of certain OSS add-ons that we think customers will benefit from. These are selected based on customer demand (e.g. [metrics-server](./addons/metrics-server.md)) and certain patterns ([gitops](./addons/argocd.md)) that are foundational elements a complete blueprint of an EKS cluster. + +One of the reasons customers pick kubernetes is because of its strong commercial and open-source software ecosystem and would like to provision add-ons that are not necessarily supported by EKS Blueprints. For such add-ons the options are as following: + +## With `helm_release` Terraform Resource + +The [helm_release](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) resource is the most basic way to provision a helm chart via terraform. + +Use this resource, if you need to control the lifecycle add-ons down to level of each add-on resource. + +## With `helm_releases` Variable + +You can use the `helm_releases` variable in [EKS Blueprints Add-ons](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addons/aws/latest?tab=inputs) to provide a map of add-ons and their respective helm configuration. Under the hood, we just iterate through the provided map and pass it to the terraform [helm_release](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) resource. + +E.g. + +```hcl +module "addons" { + source = "aws-ia/eks-blueprints-addon/aws" + version = "1.1.0" + + cluster_name = "" + cluster_endpoint = "" + cluster_version = "" + oidc_provider_arn = "" + + # EKS add-ons + eks_addons = { + coredns = {} + vpc-cni = {} + kube-proxy = {} + } + + # Blueprints add-ons + enable_aws_efs_csi_driver = true + enable_aws_cloudwatch_metrics = true + enable_cert_manager = true + ... + + # Pass in any number of Helm charts to be created for those that are not natively supported + helm_releases = { + prometheus-adapter = { + description = "A Helm chart for k8s prometheus adapter" + namespace = "prometheus-adapter" + create_namespace = true + chart = "prometheus-adapter" + chart_version = "4.2.0" + repository = "https://prometheus-community.github.io/helm-charts" + values = [ + <<-EOT + replicas: 2 + podDisruptionBudget: + enabled: true + EOT + ] + } + gpu-operator = { + description = "A Helm chart for NVIDIA GPU operator" + namespace = "gpu-operator" + create_namespace = true + chart = "gpu-operator" + chart_version = "v23.3.2" + repository = "https://nvidia.github.io/gpu-operator" + values = [ + <<-EOT + operator: + defaultRuntime: containerd + EOT + ] + } + } + + tags = local.tags +} +``` + +With this pattern, the lifecycle of all your add-ons is tied to that of the `addons` module. This allows you to easily target the addon module in your terraform apply and destroy commands. E.g. + +```sh +terraform apply -target=module.addons + +terraform destroy -target=module.addons +``` + +## With EKS Blueprints Add-on Module + +If you have any add-on that requires setting up of an IAM Role for Service Account (IRSA), we have created a new terraform module [terraform-aws-eks-blueprints-addon](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addon/aws/latest) that can help provision a helm chart along with an IAM role and policies with permissions required for the add-on to function properly. We use this module for all of the add-ons that are provisioned by EKS Blueprints Add-ons today. + +You can optionally use this module for add-ons that do not need IRSA or even just to create the IAM resources for IRSA and skip the helm release. Detailed usage of how to consume this module can be found in its [readme](https://github.com/aws-ia/terraform-aws-eks-blueprints-addon#readme). + +This pattern can be used to create a terraform module with a set of add-ons that are not supported in the EKS Blueprints Add-ons today and wrap them in the same module definition. An example of this is the [ACK add-ons repository](https://github.com/aws-ia/terraform-aws-eks-ack-addons) which is a collection of ACK helm chart deployments with IRSA for each of the ACK controllers. From f4cfa45cbb6cf1342b452c42a0ecb9858efefcce Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 15:01:01 -0700 Subject: [PATCH 02/13] pre-commit run --- docs/helm-release.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index ca7cab15..46144062 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -1,6 +1,6 @@ # Helm Release Add-ons -Starting with [EKS Blueprints v5](https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/main/docs/v4-to-v5/motivation.md) we have made a decision to only support the provisioning of a certain core set of [add-ons](./addons/). On an going basis, we will evaluate the current list to see if more add-ons need to be supported via this repo. Typically you can expect that any AWS created add-on that is not yet available via the [Amazon EKS add-ons](./amazon-eks-addons.md) will be prioritized to be provisioned through this repository. +Starting with [EKS Blueprints v5](https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/main/docs/v4-to-v5/motivation.md) we have made a decision to only support the provisioning of a certain core set of [add-ons](./addons/). On an going basis, we will evaluate the current list to see if more add-ons need to be supported via this repo. Typically you can expect that any AWS created add-on that is not yet available via the [Amazon EKS add-ons](./amazon-eks-addons.md) will be prioritized to be provisioned through this repository. In addition to these AWS add-ons, we will also support the provisioning of certain OSS add-ons that we think customers will benefit from. These are selected based on customer demand (e.g. [metrics-server](./addons/metrics-server.md)) and certain patterns ([gitops](./addons/argocd.md)) that are foundational elements a complete blueprint of an EKS cluster. @@ -40,7 +40,7 @@ module "addons" { enable_aws_cloudwatch_metrics = true enable_cert_manager = true ... - + # Pass in any number of Helm charts to be created for those that are not natively supported helm_releases = { prometheus-adapter = { @@ -84,11 +84,11 @@ With this pattern, the lifecycle of all your add-ons is tied to that of the `add terraform apply -target=module.addons terraform destroy -target=module.addons -``` +``` ## With EKS Blueprints Add-on Module -If you have any add-on that requires setting up of an IAM Role for Service Account (IRSA), we have created a new terraform module [terraform-aws-eks-blueprints-addon](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addon/aws/latest) that can help provision a helm chart along with an IAM role and policies with permissions required for the add-on to function properly. We use this module for all of the add-ons that are provisioned by EKS Blueprints Add-ons today. +If you have any add-on that requires setting up of an IAM Role for Service Account (IRSA), we have created a new terraform module [terraform-aws-eks-blueprints-addon](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addon/aws/latest) that can help provision a helm chart along with an IAM role and policies with permissions required for the add-on to function properly. We use this module for all of the add-ons that are provisioned by EKS Blueprints Add-ons today. You can optionally use this module for add-ons that do not need IRSA or even just to create the IAM resources for IRSA and skip the helm release. Detailed usage of how to consume this module can be found in its [readme](https://github.com/aws-ia/terraform-aws-eks-blueprints-addon#readme). From 990c40b89bd24283c066dc3481921b7ad34011e1 Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 15:03:18 -0700 Subject: [PATCH 03/13] update index --- docs/.pages | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/.pages b/docs/.pages index 43042130..0820b69a 100644 --- a/docs/.pages +++ b/docs/.pages @@ -3,4 +3,5 @@ nav: - Architectures: architectures.md - Amazon EKS Addons: amazon-eks-addons.md - AWS Partner Addons: aws-partner-addons.md + - Helm Releases: helm-release.md - Addons: addons From 363badbf04224a2544dd339db984ba6c66501820 Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:06:05 -0700 Subject: [PATCH 04/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index 46144062..da6ea0ad 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -2,7 +2,7 @@ Starting with [EKS Blueprints v5](https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/main/docs/v4-to-v5/motivation.md) we have made a decision to only support the provisioning of a certain core set of [add-ons](./addons/). On an going basis, we will evaluate the current list to see if more add-ons need to be supported via this repo. Typically you can expect that any AWS created add-on that is not yet available via the [Amazon EKS add-ons](./amazon-eks-addons.md) will be prioritized to be provisioned through this repository. -In addition to these AWS add-ons, we will also support the provisioning of certain OSS add-ons that we think customers will benefit from. These are selected based on customer demand (e.g. [metrics-server](./addons/metrics-server.md)) and certain patterns ([gitops](./addons/argocd.md)) that are foundational elements a complete blueprint of an EKS cluster. +In addition to these AWS add-ons, we will also support the provisioning of certain OSS add-ons that we think customers will benefit from. These are selected based on customer demand (e.g. [metrics-server](./addons/metrics-server.md)) and certain patterns ([gitops](./addons/argocd.md)) that are foundational elements for a complete blueprint of an EKS cluster. One of the reasons customers pick kubernetes is because of its strong commercial and open-source software ecosystem and would like to provision add-ons that are not necessarily supported by EKS Blueprints. For such add-ons the options are as following: From 880d704bbb4ea383b090231aecec907ff609c198 Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:06:12 -0700 Subject: [PATCH 05/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index da6ea0ad..77a6287a 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -4,7 +4,7 @@ Starting with [EKS Blueprints v5](https://github.com/aws-ia/terraform-aws-eks-bl In addition to these AWS add-ons, we will also support the provisioning of certain OSS add-ons that we think customers will benefit from. These are selected based on customer demand (e.g. [metrics-server](./addons/metrics-server.md)) and certain patterns ([gitops](./addons/argocd.md)) that are foundational elements for a complete blueprint of an EKS cluster. -One of the reasons customers pick kubernetes is because of its strong commercial and open-source software ecosystem and would like to provision add-ons that are not necessarily supported by EKS Blueprints. For such add-ons the options are as following: +One of the reasons customers pick Kubernetes is because of its strong commercial and open-source software ecosystem and would like to provision add-ons that are not necessarily supported by EKS Blueprints. For such add-ons the options are as following: ## With `helm_release` Terraform Resource From baf35682e97a89b14f6f7adb8f42281053336453 Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:06:21 -0700 Subject: [PATCH 06/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index 77a6287a..02dc1ed0 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -8,7 +8,7 @@ One of the reasons customers pick Kubernetes is because of its strong commercial ## With `helm_release` Terraform Resource -The [helm_release](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) resource is the most basic way to provision a helm chart via terraform. +The [helm_release](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) resource is the most fundamental way to provision a helm chart via Terraform. Use this resource, if you need to control the lifecycle add-ons down to level of each add-on resource. From dfe95aa37c49e624ca9ede4659871dcc2d02aecc Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:06:36 -0700 Subject: [PATCH 07/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index 02dc1ed0..e0e34946 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -14,7 +14,7 @@ Use this resource, if you need to control the lifecycle add-ons down to level of ## With `helm_releases` Variable -You can use the `helm_releases` variable in [EKS Blueprints Add-ons](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addons/aws/latest?tab=inputs) to provide a map of add-ons and their respective helm configuration. Under the hood, we just iterate through the provided map and pass it to the terraform [helm_release](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) resource. +You can use the `helm_releases` variable in [EKS Blueprints Add-ons](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addons/aws/latest?tab=inputs) to provide a map of add-ons and their respective Helm configuration. Under the hood, we just iterate through the provided map and pass each configuration to the Terraform [helm_release](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) resource. E.g. From 17861716bdd91f2f5ef865cd87ca268f3e2372fe Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:06:48 -0700 Subject: [PATCH 08/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index e0e34946..5e63c348 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -20,7 +20,7 @@ E.g. ```hcl module "addons" { - source = "aws-ia/eks-blueprints-addon/aws" + source = "aws-ia/eks-blueprints-addons/aws" version = "1.1.0" cluster_name = "" From 64f4de0c6b51d4e67ab7c468103385519f8db9fc Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:06:57 -0700 Subject: [PATCH 09/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index 5e63c348..77270a66 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -21,7 +21,7 @@ E.g. ```hcl module "addons" { source = "aws-ia/eks-blueprints-addons/aws" - version = "1.1.0" + version = "~> 1.0" cluster_name = "" cluster_endpoint = "" From 92897223e247b75fa9201a9235bdaceba8a65ba9 Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:07:16 -0700 Subject: [PATCH 10/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index 77270a66..bcfbd7c2 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -78,7 +78,7 @@ module "addons" { } ``` -With this pattern, the lifecycle of all your add-ons is tied to that of the `addons` module. This allows you to easily target the addon module in your terraform apply and destroy commands. E.g. +With this pattern, the lifecycle of all your add-ons is tied to that of the `addons` module. This allows you to easily target the addon module in your Terraform apply and destroy commands. E.g. ```sh terraform apply -target=module.addons From 52790e527540077091c75025d5f8dd6347f5677a Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:10:07 -0700 Subject: [PATCH 11/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index bcfbd7c2..9072aec6 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -88,7 +88,7 @@ terraform destroy -target=module.addons ## With EKS Blueprints Add-on Module -If you have any add-on that requires setting up of an IAM Role for Service Account (IRSA), we have created a new terraform module [terraform-aws-eks-blueprints-addon](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addon/aws/latest) that can help provision a helm chart along with an IAM role and policies with permissions required for the add-on to function properly. We use this module for all of the add-ons that are provisioned by EKS Blueprints Add-ons today. +If you have an add-on that requires an IAM Role for Service Account (IRSA), we have created a new Terraform module [terraform-aws-eks-blueprints-addon](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addon/aws/latest) that can help provision a Helm chart along with an IAM role and policies with permissions required for the add-on to function properly. We use this module for all of the add-ons that are provisioned by EKS Blueprints Add-ons today. You can optionally use this module for add-ons that do not need IRSA or even just to create the IAM resources for IRSA and skip the helm release. Detailed usage of how to consume this module can be found in its [readme](https://github.com/aws-ia/terraform-aws-eks-blueprints-addon#readme). From 07a5c0f713fb88916188a6f377ceb577fd7aec85 Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:13:41 -0700 Subject: [PATCH 12/13] Update docs/helm-release.md Co-authored-by: Bryant Biggs --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index 9072aec6..f6bd0f65 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -92,4 +92,4 @@ If you have an add-on that requires an IAM Role for Service Account (IRSA), we h You can optionally use this module for add-ons that do not need IRSA or even just to create the IAM resources for IRSA and skip the helm release. Detailed usage of how to consume this module can be found in its [readme](https://github.com/aws-ia/terraform-aws-eks-blueprints-addon#readme). -This pattern can be used to create a terraform module with a set of add-ons that are not supported in the EKS Blueprints Add-ons today and wrap them in the same module definition. An example of this is the [ACK add-ons repository](https://github.com/aws-ia/terraform-aws-eks-ack-addons) which is a collection of ACK helm chart deployments with IRSA for each of the ACK controllers. +This pattern can be used to create a Terraform module with a set of add-ons that are not supported in the EKS Blueprints Add-ons today and wrap them in the same module definition. An example of this is the [ACK add-ons repository](https://github.com/aws-ia/terraform-aws-eks-ack-addons) which is a collection of ACK helm chart deployments with IRSA for each of the ACK controllers. From 5487c65facc2193ce3de51b72682e888c7bef7c4 Mon Sep 17 00:00:00 2001 From: Apoorva Kulkarni Date: Wed, 7 Jun 2023 16:14:53 -0700 Subject: [PATCH 13/13] remove hyphen for now --- docs/helm-release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/helm-release.md b/docs/helm-release.md index f6bd0f65..28054569 100644 --- a/docs/helm-release.md +++ b/docs/helm-release.md @@ -86,7 +86,7 @@ terraform apply -target=module.addons terraform destroy -target=module.addons ``` -## With EKS Blueprints Add-on Module +## With EKS Blueprints Addon Module If you have an add-on that requires an IAM Role for Service Account (IRSA), we have created a new Terraform module [terraform-aws-eks-blueprints-addon](https://registry.terraform.io/modules/aws-ia/eks-blueprints-addon/aws/latest) that can help provision a Helm chart along with an IAM role and policies with permissions required for the add-on to function properly. We use this module for all of the add-ons that are provisioned by EKS Blueprints Add-ons today.