Moving autoscaling from beta to GA

README.md

@@ -39,6 +39,7 @@ module "gke" {
     {
       name               = "default-node-pool"
       machine_type       = "e2-medium"
+      node_locations     = "us-central1-b,us-central1-c"
       min_count          = 1
       max_count          = 100
       local_ssd_count    = 0
@@ -77,6 +78,18 @@ module "gke" {
     }
   }
 
+  node_pools_taints = {
+    all = []
+
+    default-node-pool = [
+      {
+        key    = "default-node-pool"
+        value  = true
+        effect = "PREFER_NO_SCHEDULE"
+      },
+    ]
+  }
+
   node_pools_tags = {
     all = []
 
@@ -103,6 +116,7 @@ Then perform the following commands on the root folder:
 | add\_cluster\_firewall\_rules | Create additional firewall rules | bool | `"false"` | no |
 | basic\_auth\_password | The password to be used with Basic Authentication. | string | `""` | no |
 | basic\_auth\_username | The username to be used with Basic Authentication. An empty value will disable Basic Authentication, which is the recommended configuration. | string | `""` | no |
+| cluster\_autoscaling | Cluster autoscaling configuration. See [more details](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#clusterautoscaling) | object | `<map>` | no |
 | cluster\_ipv4\_cidr | The IP address range of the kubernetes pods in this cluster. Default is an automatically assigned CIDR. | string | `"null"` | no |
 | cluster\_resource\_labels | The GCE resource labels (a map of key/value pairs) to be applied to the cluster | map(string) | `<map>` | no |
 | configure\_ip\_masq | Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server. | string | `"false"` | no |
@@ -201,17 +215,22 @@ The node_pools variable takes the following parameters:
 | auto_upgrade | Whether the nodes will be automatically upgraded | true (if cluster is regional) | Optional |
 | disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
 | disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
+| effect | Effect for the taint | | Required |
 | image_type | The image type to use for this node. Note that changing the image type will delete and recreate all nodes in the node pool | COS | Optional |
 | initial_node_count | The initial number of nodes for the pool. In regional or multi-zonal clusters, this is the number of nodes per zone. Changing this will force recreation of the resource. Defaults to the value of min_count | " " | Optional |
+| key | The key required for the taint | | Required |
 | local_ssd_count | The amount of local SSD disks that will be attached to each cluster node | 0 | Optional |
 | machine_type | The name of a Google Compute Engine machine type | e2-medium | Optional |
 | max_count | Maximum number of nodes in the NodePool. Must be >= min_count | 100 | Optional |
 | min_count | Minimum number of nodes in the NodePool. Must be >=0 and <= max_count. Should be used when autoscaling is true | 1 | Optional |
 | name | The name of the node pool |  | Required |
 | node_count | The number of nodes in the nodepool when autoscaling is false. Otherwise defaults to 1. Only valid for non-autoscaling clusers |  | Required |
+| node_locations | The list of zones in which the cluster's nodes are located. Nodes must be in the region of their regional cluster or in the same region as their cluster's zone for zonal clusters. Defaults to cluster level node locations if nothing is specified | " " | Optional |
+| node_metadata | Options to expose the node metadata to the workload running on the node | | Optional |
 | preemptible | A boolean that represents whether or not the underlying node VMs are preemptible | false | Optional |
 | service_account | The service account to be used by the Node VMs | " " | Optional |
 | tags | The list of instance tags applied to all nodes | | Required |
+| value | The value for the taint | | Required |
 | version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |
 
 
@@ -231,7 +250,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
 - [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
 #### Terraform and Plugins
 - [Terraform](https://www.terraform.io/downloads.html) 0.12
-- [Terraform Provider for GCP][terraform-provider-google] v2.9
+- [Terraform Provider for GCP][terraform-provider-google] v3.41
 
 ### Configure a Service Account
 In order to execute this module you must have a Service Account with the

autogen/main/README.md

@@ -79,9 +79,7 @@ module "gke" {
     {
       name               = "default-node-pool"
       machine_type       = "e2-medium"
-      {% if beta_cluster %}
       node_locations     = "us-central1-b,us-central1-c"
-      {% endif %}
       min_count          = 1
       max_count          = 100
       local_ssd_count    = 0
@@ -119,7 +117,6 @@ module "gke" {
       node-pool-metadata-custom-value = "my-node-pool"
     }
   }
-  {% if beta_cluster %}
 
   node_pools_taints = {
     all = []
@@ -132,7 +129,6 @@ module "gke" {
       },
     ]
   }
-  {% endif %}
 
   node_pools_tags = {
     all = []
@@ -169,14 +165,10 @@ The node_pools variable takes the following parameters:
 | auto_upgrade | Whether the nodes will be automatically upgraded | true (if cluster is regional) | Optional |
 | disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
 | disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
-{% if beta_cluster %}
 | effect | Effect for the taint | | Required |
-{% endif %}
 | image_type | The image type to use for this node. Note that changing the image type will delete and recreate all nodes in the node pool | COS | Optional |
 | initial_node_count | The initial number of nodes for the pool. In regional or multi-zonal clusters, this is the number of nodes per zone. Changing this will force recreation of the resource. Defaults to the value of min_count | " " | Optional |
-{% if beta_cluster %}
 | key | The key required for the taint | | Required |
-{% endif %}
 | local_ssd_count | The amount of local SSD disks that will be attached to each cluster node | 0 | Optional |
 | machine_type | The name of a Google Compute Engine machine type | e2-medium | Optional |
 | max_count | Maximum number of nodes in the NodePool. Must be >= min_count | 100 | Optional |
@@ -188,19 +180,15 @@ The node_pools variable takes the following parameters:
 | min_count | Minimum number of nodes in the NodePool. Must be >=0 and <= max_count. Should be used when autoscaling is true | 1 | Optional |
 | name | The name of the node pool |  | Required |
 | node_count | The number of nodes in the nodepool when autoscaling is false. Otherwise defaults to 1. Only valid for non-autoscaling clusers |  | Required |
-{% if beta_cluster %}
 | node_locations | The list of zones in which the cluster's nodes are located. Nodes must be in the region of their regional cluster or in the same region as their cluster's zone for zonal clusters. Defaults to cluster level node locations if nothing is specified | " " | Optional |
 | node_metadata | Options to expose the node metadata to the workload running on the node | | Optional |
-{% endif %}
 | preemptible | A boolean that represents whether or not the underlying node VMs are preemptible | false | Optional |
 {% if beta_cluster %}
 | sandbox_type | Sandbox to use for pods in the node pool | | Required |
 {% endif %}
 | service_account | The service account to be used by the Node VMs | " " | Optional |
 | tags | The list of instance tags applied to all nodes | | Required |
-{% if beta_cluster %}
 | value | The value for the taint | | Required |
-{% endif %}
 | version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |
 
 
@@ -221,9 +209,9 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
 #### Terraform and Plugins
 - [Terraform](https://www.terraform.io/downloads.html) 0.12
 {% if beta_cluster %}
-- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
+- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
 {% else %}
-- [Terraform Provider for GCP][terraform-provider-google] v2.9
+- [Terraform Provider for GCP][terraform-provider-google] v3.41
 {% endif %}
 
 ### Configure a Service Account

autogen/main/cluster.tf.tmpl

@@ -65,10 +65,11 @@ resource "google_container_cluster" "primary" {
   logging_service    = var.logging_service
   monitoring_service = var.monitoring_service
 
-{% if beta_cluster %}
   cluster_autoscaling {
     enabled             = var.cluster_autoscaling.enabled
+{% if beta_cluster %}
     autoscaling_profile = var.cluster_autoscaling.autoscaling_profile != null ? var.cluster_autoscaling.autoscaling_profile : "BALANCED"
+{% endif %}
     dynamic "resource_limits" {
       for_each = local.autoscalling_resource_limits
       content {
@@ -78,7 +79,6 @@ resource "google_container_cluster" "primary" {
       }
     }
   }
-{% endif %}
 
   default_max_pods_per_node = var.default_max_pods_per_node
 
@@ -389,10 +389,8 @@ resource "google_container_node_pool" "pools" {
   {% endif %}
   project  = var.project_id
   location = local.location
-  {% if beta_cluster %}
   // use node_locations if provided, defaults to cluster level node_locations if not specified
   node_locations = lookup(each.value, "node_locations", "") != "" ? split(",", each.value["node_locations"]) : null
-  {% endif %}
 
   cluster = google_container_cluster.primary.name
 

autogen/main/main.tf.tmpl

@@ -51,7 +51,6 @@ locals {
   node_pools      = zipmap(local.node_pool_names, tolist(toset(var.node_pools)))
 
   release_channel = var.release_channel != null ? [{ channel : var.release_channel }] : []
-{% if beta_cluster %}
 
   autoscalling_resource_limits = var.cluster_autoscaling.enabled ? [{
     resource_type = "cpu"
@@ -63,8 +62,6 @@ locals {
     maximum       = var.cluster_autoscaling.max_memory_gb
   }] : []
 
-{% endif %}
-
 
   custom_kube_dns_config      = length(keys(var.stub_domains)) > 0
   upstream_nameservers_config = length(var.upstream_nameservers) > 0

autogen/main/variables.tf.tmpl

@@ -207,27 +207,31 @@ variable "enable_kubernetes_alpha" {
   description = "Whether to enable Kubernetes Alpha features for this cluster. Note that when this option is enabled, the cluster cannot be upgraded and will be automatically deleted after 30 days."
   default     = false
 }
+{% endif %}
 
 variable "cluster_autoscaling" {
   type = object({
     enabled             = bool
+{% if beta_cluster %}
     autoscaling_profile = string
+{% endif %}
     min_cpu_cores       = number
     max_cpu_cores       = number
     min_memory_gb       = number
     max_memory_gb       = number
   })
   default = {
     enabled             = false
+{% if beta_cluster %}
     autoscaling_profile = "BALANCED"
+{% endif %}
     max_cpu_cores       = 0
     min_cpu_cores       = 0
     max_memory_gb       = 0
     min_memory_gb       = 0
   }
   description = "Cluster autoscaling configuration. See [more details](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#clusterautoscaling)"
 }
-{% endif %}
 
 variable "node_pools_taints" {
   type        = map(list(object({ key = string, value = string, effect = string })))

cluster.tf

@@ -56,6 +56,17 @@ resource "google_container_cluster" "primary" {
   logging_service    = var.logging_service
   monitoring_service = var.monitoring_service
 
+  cluster_autoscaling {
+    enabled = var.cluster_autoscaling.enabled
+    dynamic "resource_limits" {
+      for_each = local.autoscalling_resource_limits
+      content {
+        resource_type = lookup(resource_limits.value, "resource_type")
+        minimum       = lookup(resource_limits.value, "minimum")
+        maximum       = lookup(resource_limits.value, "maximum")
+      }
+    }
+  }
 
   default_max_pods_per_node = var.default_max_pods_per_node
 
@@ -181,6 +192,8 @@ resource "google_container_node_pool" "pools" {
   name     = each.key
   project  = var.project_id
   location = local.location
+  // use node_locations if provided, defaults to cluster level node_locations if not specified
+  node_locations = lookup(each.value, "node_locations", "") != "" ? split(",", each.value["node_locations"]) : null
 
   cluster = google_container_cluster.primary.name
 

examples/node_pool/main.tf

@@ -24,7 +24,7 @@ provider "google-beta" {
 }
 
 module "gke" {
-  source                            = "../.."
+  source                            = "../../modules/beta-public-cluster/"
   project_id                        = var.project_id
   name                              = "${local.cluster_type}-cluster${var.cluster_name_suffix}"
   region                            = var.region

main.tf

@@ -48,6 +48,16 @@ locals {
 
   release_channel = var.release_channel != null ? [{ channel : var.release_channel }] : []
 
+  autoscalling_resource_limits = var.cluster_autoscaling.enabled ? [{
+    resource_type = "cpu"
+    minimum       = var.cluster_autoscaling.min_cpu_cores
+    maximum       = var.cluster_autoscaling.max_cpu_cores
+    }, {
+    resource_type = "memory"
+    minimum       = var.cluster_autoscaling.min_memory_gb
+    maximum       = var.cluster_autoscaling.max_memory_gb
+  }] : []
+
 
   custom_kube_dns_config      = length(keys(var.stub_domains)) > 0
   upstream_nameservers_config = length(var.upstream_nameservers) > 0

modules/beta-private-cluster-update-variant/README.md

@@ -315,7 +315,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
 - [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
 #### Terraform and Plugins
 - [Terraform](https://www.terraform.io/downloads.html) 0.12
-- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
+- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
 
 ### Configure a Service Account
 In order to execute this module you must have a Service Account with the

modules/beta-private-cluster-update-variant/main.tf

@@ -59,7 +59,6 @@ locals {
   }] : []
 
 
-
   custom_kube_dns_config      = length(keys(var.stub_domains)) > 0
   upstream_nameservers_config = length(var.upstream_nameservers) > 0
   network_project_id          = var.network_project_id != "" ? var.network_project_id : var.project_id

modules/beta-private-cluster/README.md

@@ -293,7 +293,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
 - [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
 #### Terraform and Plugins
 - [Terraform](https://www.terraform.io/downloads.html) 0.12
-- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
+- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
 
 ### Configure a Service Account
 In order to execute this module you must have a Service Account with the

modules/beta-private-cluster/main.tf

@@ -59,7 +59,6 @@ locals {
   }] : []
 
 
-
   custom_kube_dns_config      = length(keys(var.stub_domains)) > 0
   upstream_nameservers_config = length(var.upstream_nameservers) > 0
   network_project_id          = var.network_project_id != "" ? var.network_project_id : var.project_id

modules/beta-public-cluster-update-variant/README.md

@@ -302,7 +302,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
 - [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
 #### Terraform and Plugins
 - [Terraform](https://www.terraform.io/downloads.html) 0.12
-- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
+- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
 
 ### Configure a Service Account
 In order to execute this module you must have a Service Account with the

modules/beta-public-cluster-update-variant/main.tf

@@ -59,7 +59,6 @@ locals {
   }] : []
 
 
-
   custom_kube_dns_config      = length(keys(var.stub_domains)) > 0
   upstream_nameservers_config = length(var.upstream_nameservers) > 0
   network_project_id          = var.network_project_id != "" ? var.network_project_id : var.project_id

modules/beta-public-cluster/README.md

@@ -280,7 +280,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
 - [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
 #### Terraform and Plugins
 - [Terraform](https://www.terraform.io/downloads.html) 0.12
-- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
+- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
 
 ### Configure a Service Account
 In order to execute this module you must have a Service Account with the

modules/beta-public-cluster/main.tf

@@ -59,7 +59,6 @@ locals {
   }] : []
 
 
-
   custom_kube_dns_config      = length(keys(var.stub_domains)) > 0
   upstream_nameservers_config = length(var.upstream_nameservers) > 0
   network_project_id          = var.network_project_id != "" ? var.network_project_id : var.project_id