Skip to content

Commit

Permalink
feat: Promote previously beta features to GA modules (#709)
Browse files Browse the repository at this point in the history 
* fix: promoting release_channel, database_encryption, workload_identity_config, and enable_shielded_nodes from beta to GA

* feat: allow passing roles to created Workload Identity service account (#708)

* Moving autoscaling from beta to GA

* Moving binary authorization from beta to GA

Co-authored-by: Mmadu Manasseh <mmadumanasseh@gmail.com>
Co-authored-by: Bharath KKB <bharathkrishnakb@gmail.com>
  • Loading branch information
@marko7460 @MeNsaaH @bharathkkb
3 people committed Oct 14, 2020
1 parent e761dce commit 2cb4fae
Show file tree
Hide file tree
Showing 49 changed files with 824 additions and 323 deletions.
29 changes: 28 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ module "gke" {
{
name = "default-node-pool"
machine_type = "e2-medium"
node_locations = "us-central1-b,us-central1-c"
min_count = 1
max_count = 100
local_ssd_count = 0
Expand Down Expand Up @@ -77,6 +78,18 @@ module "gke" {
}
}
node_pools_taints = {
all = []
default-node-pool = [
{
key = "default-node-pool"
value = true
effect = "PREFER_NO_SCHEDULE"
},
]
}
node_pools_tags = {
all = []
Expand All @@ -103,22 +116,27 @@ Then perform the following commands on the root folder:
| add\_cluster\_firewall\_rules | Create additional firewall rules | bool | `"false"` | no |
| basic\_auth\_password | The password to be used with Basic Authentication. | string | `""` | no |
| basic\_auth\_username | The username to be used with Basic Authentication. An empty value will disable Basic Authentication, which is the recommended configuration. | string | `""` | no |
| cluster\_autoscaling | Cluster autoscaling configuration. See [more details](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#clusterautoscaling) | object | `<map>` | no |
| cluster\_ipv4\_cidr | The IP address range of the kubernetes pods in this cluster. Default is an automatically assigned CIDR. | string | `"null"` | no |
| cluster\_resource\_labels | The GCE resource labels (a map of key/value pairs) to be applied to the cluster | map(string) | `<map>` | no |
| configure\_ip\_masq | Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server. | string | `"false"` | no |
| create\_service\_account | Defines if service account specified to run nodes should be created. | bool | `"true"` | no |
| database\_encryption | Application-layer Secrets Encryption settings. The object format is {state = string, key_name = string}. Valid values of state are: "ENCRYPTED"; "DECRYPTED". key_name is the name of a CloudKMS key. | object | `<list>` | no |
| default\_max\_pods\_per\_node | The maximum number of pods to schedule per node | string | `"110"` | no |
| description | The description of the cluster | string | `""` | no |
| disable\_legacy\_metadata\_endpoints | Disable the /0.1/ and /v1beta1/ metadata server endpoints on the node. Changing this value will cause all node pools to be recreated. | bool | `"true"` | no |
| enable\_binary\_authorization | Enable BinAuthZ Admission controller | string | `"false"` | no |
| enable\_network\_egress\_export | Whether to enable network egress metering for this cluster. If enabled, a daemonset will be created in the cluster to meter network egress traffic. | bool | `"false"` | no |
| enable\_resource\_consumption\_export | Whether to enable resource consumption metering on this cluster. When enabled, a table will be created in the resource export BigQuery dataset to store resource consumption data. The resulting table can be joined with the resource usage table or with BigQuery billing export. | bool | `"true"` | no |
| enable\_shielded\_nodes | Enable Shielded Nodes features on all nodes in this cluster | bool | `"true"` | no |
| firewall\_inbound\_ports | List of TCP ports for admission/webhook controllers | list(string) | `<list>` | no |
| firewall\_priority | Priority rule for firewall rules | number | `"1000"` | no |
| gcloud\_skip\_download | Whether to skip downloading gcloud (assumes gcloud is already available outside the module) | bool | `"true"` | no |
| gcloud\_upgrade | Whether to upgrade gcloud at runtime | bool | `"false"` | no |
| grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer role. | bool | `"false"` | no |
| horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | bool | `"true"` | no |
| http\_load\_balancing | Enable httpload balancer addon | bool | `"true"` | no |
| identity\_namespace | Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`) | string | `"enabled"` | no |
| initial\_node\_count | The number of nodes to create in this cluster's default node pool. | number | `"0"` | no |
| ip\_masq\_link\_local | Whether to masquerade traffic to the link-local prefix (169.254.0.0/16). | bool | `"false"` | no |
| ip\_masq\_resync\_interval | The interval at which the agent attempts to sync its ConfigMap file from the disk. | string | `"60s"` | no |
Expand All @@ -135,6 +153,7 @@ Then perform the following commands on the root folder:
| network\_policy | Enable network policy addon | bool | `"true"` | no |
| network\_policy\_provider | The network policy provider. | string | `"CALICO"` | no |
| network\_project\_id | The project ID of the shared VPC's host (for shared vpc support) | string | `""` | no |
| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | string | `"GKE_METADATA_SERVER"` | no |
| node\_pools | List of maps containing node pools | list(map(string)) | `<list>` | no |
| node\_pools\_labels | Map of maps containing node labels by node-pool name | map(map(string)) | `<map>` | no |
| node\_pools\_metadata | Map of maps containing node metadata by node-pool name | map(map(string)) | `<map>` | no |
Expand All @@ -146,6 +165,7 @@ Then perform the following commands on the root folder:
| region | The region to host the cluster in (optional if zonal cluster / required if regional) | string | `"null"` | no |
| regional | Whether is a regional cluster (zonal cluster if set false. WARNING: changing this after cluster creation is destructive!) | bool | `"true"` | no |
| registry\_project\_id | Project holding the Google Container Registry. If empty, we use the cluster project. If grant_registry_access is true, storage.objectViewer role is assigned on this project. | string | `""` | no |
| release\_channel | The release channel of this cluster. Accepted values are `UNSPECIFIED`, `RAPID`, `REGULAR` and `STABLE`. Defaults to `UNSPECIFIED`. | string | `"null"` | no |
| remove\_default\_node\_pool | Remove default node pool while setting up the cluster | bool | `"false"` | no |
| resource\_usage\_export\_dataset\_id | The ID of a BigQuery Dataset for using BigQuery as the destination of resource usage export. | string | `""` | no |
| service\_account | The service account to run nodes as if not overridden in `node_pools`. The create_service_account variable default value (true) will cause a cluster-specific service account to be created. | string | `""` | no |
Expand All @@ -163,6 +183,7 @@ Then perform the following commands on the root folder:
| endpoint | Cluster endpoint |
| horizontal\_pod\_autoscaling\_enabled | Whether horizontal pod autoscaling enabled |
| http\_load\_balancing\_enabled | Whether http load balancing enabled |
| identity\_namespace | Workload Identity namespace |
| location | Cluster location (region if regional cluster, zone if zonal cluster) |
| logging\_service | Logging service used |
| master\_authorized\_networks\_config | Networks from which access to master is permitted |
Expand All @@ -174,6 +195,7 @@ Then perform the following commands on the root folder:
| node\_pools\_names | List of node pools names |
| node\_pools\_versions | List of node pools versions |
| region | Cluster region |
| release\_channel | The release channel of this cluster |
| service\_account | The service account to default running nodes as if not overridden in `node_pools`. |
| type | Cluster type (regional / zonal) |
| zones | List of zones in which the cluster resides |
Expand All @@ -194,17 +216,22 @@ The node_pools variable takes the following parameters:
| auto_upgrade | Whether the nodes will be automatically upgraded | true (if cluster is regional) | Optional |
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
| effect | Effect for the taint | | Required |
| image_type | The image type to use for this node. Note that changing the image type will delete and recreate all nodes in the node pool | COS | Optional |
| initial_node_count | The initial number of nodes for the pool. In regional or multi-zonal clusters, this is the number of nodes per zone. Changing this will force recreation of the resource. Defaults to the value of min_count | " " | Optional |
| key | The key required for the taint | | Required |
| local_ssd_count | The amount of local SSD disks that will be attached to each cluster node | 0 | Optional |
| machine_type | The name of a Google Compute Engine machine type | e2-medium | Optional |
| max_count | Maximum number of nodes in the NodePool. Must be >= min_count | 100 | Optional |
| min_count | Minimum number of nodes in the NodePool. Must be >=0 and <= max_count. Should be used when autoscaling is true | 1 | Optional |
| name | The name of the node pool | | Required |
| node_count | The number of nodes in the nodepool when autoscaling is false. Otherwise defaults to 1. Only valid for non-autoscaling clusers | | Required |
| node_locations | The list of zones in which the cluster's nodes are located. Nodes must be in the region of their regional cluster or in the same region as their cluster's zone for zonal clusters. Defaults to cluster level node locations if nothing is specified | " " | Optional |
| node_metadata | Options to expose the node metadata to the workload running on the node | | Optional |
| preemptible | A boolean that represents whether or not the underlying node VMs are preemptible | false | Optional |
| service_account | The service account to be used by the Node VMs | " " | Optional |
| tags | The list of instance tags applied to all nodes | | Required |
| value | The value for the taint | | Required |
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |


Expand All @@ -224,7 +251,7 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
- [kubectl](https://github.com/kubernetes/kubernetes/releases) 1.9.x
#### Terraform and Plugins
- [Terraform](https://www.terraform.io/downloads.html) 0.12
- [Terraform Provider for GCP][terraform-provider-google] v2.9
- [Terraform Provider for GCP][terraform-provider-google] v3.41

### Configure a Service Account
In order to execute this module you must have a Service Account with the
Expand Down
16 changes: 2 additions & 14 deletions autogen/main/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,7 @@ module "gke" {
{
name = "default-node-pool"
machine_type = "e2-medium"
{% if beta_cluster %}
node_locations = "us-central1-b,us-central1-c"
{% endif %}
min_count = 1
max_count = 100
local_ssd_count = 0
Expand Down Expand Up @@ -119,7 +117,6 @@ module "gke" {
node-pool-metadata-custom-value = "my-node-pool"
}
}
{% if beta_cluster %}
node_pools_taints = {
all = []
Expand All @@ -132,7 +129,6 @@ module "gke" {
},
]
}
{% endif %}
node_pools_tags = {
all = []
Expand Down Expand Up @@ -169,14 +165,10 @@ The node_pools variable takes the following parameters:
| auto_upgrade | Whether the nodes will be automatically upgraded | true (if cluster is regional) | Optional |
| disk_size_gb | Size of the disk attached to each node, specified in GB. The smallest allowed disk size is 10GB | 100 | Optional |
| disk_type | Type of the disk attached to each node (e.g. 'pd-standard' or 'pd-ssd') | pd-standard | Optional |
{% if beta_cluster %}
| effect | Effect for the taint | | Required |
{% endif %}
| image_type | The image type to use for this node. Note that changing the image type will delete and recreate all nodes in the node pool | COS | Optional |
| initial_node_count | The initial number of nodes for the pool. In regional or multi-zonal clusters, this is the number of nodes per zone. Changing this will force recreation of the resource. Defaults to the value of min_count | " " | Optional |
{% if beta_cluster %}
| key | The key required for the taint | | Required |
{% endif %}
| local_ssd_count | The amount of local SSD disks that will be attached to each cluster node | 0 | Optional |
| machine_type | The name of a Google Compute Engine machine type | e2-medium | Optional |
| max_count | Maximum number of nodes in the NodePool. Must be >= min_count | 100 | Optional |
Expand All @@ -188,19 +180,15 @@ The node_pools variable takes the following parameters:
| min_count | Minimum number of nodes in the NodePool. Must be >=0 and <= max_count. Should be used when autoscaling is true | 1 | Optional |
| name | The name of the node pool | | Required |
| node_count | The number of nodes in the nodepool when autoscaling is false. Otherwise defaults to 1. Only valid for non-autoscaling clusers | | Required |
{% if beta_cluster %}
| node_locations | The list of zones in which the cluster's nodes are located. Nodes must be in the region of their regional cluster or in the same region as their cluster's zone for zonal clusters. Defaults to cluster level node locations if nothing is specified | " " | Optional |
| node_metadata | Options to expose the node metadata to the workload running on the node | | Optional |
{% endif %}
| preemptible | A boolean that represents whether or not the underlying node VMs are preemptible | false | Optional |
{% if beta_cluster %}
| sandbox_type | Sandbox to use for pods in the node pool | | Required |
{% endif %}
| service_account | The service account to be used by the Node VMs | " " | Optional |
| tags | The list of instance tags applied to all nodes | | Required |
{% if beta_cluster %}
| value | The value for the taint | | Required |
{% endif %}
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |


Expand All @@ -221,9 +209,9 @@ The [project factory](https://github.com/terraform-google-modules/terraform-goog
#### Terraform and Plugins
- [Terraform](https://www.terraform.io/downloads.html) 0.12
{% if beta_cluster %}
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v2.9
- [Terraform Provider for GCP Beta][terraform-provider-google-beta] v3.41
{% else %}
- [Terraform Provider for GCP][terraform-provider-google] v2.9
- [Terraform Provider for GCP][terraform-provider-google] v3.41
{% endif %}

### Configure a Service Account
Expand Down
23 changes: 6 additions & 17 deletions autogen/main/cluster.tf.tmpl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,13 @@ resource "google_container_cluster" "primary" {
}
}

{% if beta_cluster %}
dynamic "release_channel" {
for_each = local.release_channel

content {
channel = release_channel.value.channel
}
}
{% endif %}

subnetwork = "projects/${local.network_project_id}/regions/${var.region}/subnetworks/${var.subnetwork}"

Expand All @@ -62,19 +60,16 @@ resource "google_container_cluster" "primary" {
disabled = var.disable_default_snat
}
{% endif %}
{% if beta_cluster %}
min_master_version = var.release_channel != null ? null : local.master_version
{% else %}
min_master_version = local.master_version
{% endif %}

logging_service = var.logging_service
monitoring_service = var.monitoring_service

{% if beta_cluster %}
cluster_autoscaling {
enabled = var.cluster_autoscaling.enabled
{% if beta_cluster %}
autoscaling_profile = var.cluster_autoscaling.autoscaling_profile != null ? var.cluster_autoscaling.autoscaling_profile : "BALANCED"
{% endif %}
dynamic "resource_limits" {
for_each = local.autoscalling_resource_limits
content {
Expand All @@ -84,14 +79,13 @@ resource "google_container_cluster" "primary" {
}
}
}
{% endif %}

default_max_pods_per_node = var.default_max_pods_per_node

{% if beta_cluster %}
enable_shielded_nodes = var.enable_shielded_nodes
enable_binary_authorization = var.enable_binary_authorization
{% if beta_cluster %}
enable_intranode_visibility = var.enable_intranode_visibility
enable_shielded_nodes = var.enable_shielded_nodes
enable_kubernetes_alpha = var.enable_kubernetes_alpha

vertical_pod_autoscaling {
Expand Down Expand Up @@ -224,7 +218,6 @@ resource "google_container_cluster" "primary" {

node_config {
service_account = lookup(var.node_pools[0], "service_account", local.service_account)
{% if beta_cluster %}

dynamic "workload_metadata_config" {
for_each = local.cluster_node_metadata_config
Expand All @@ -233,7 +226,6 @@ resource "google_container_cluster" "primary" {
node_metadata = workload_metadata_config.value.node_metadata
}
}
{% endif %}
}
}

Expand Down Expand Up @@ -278,7 +270,6 @@ resource "google_container_cluster" "primary" {
{% endif %}

remove_default_node_pool = var.remove_default_node_pool
{% if beta_cluster %}

dynamic "database_encryption" {
for_each = var.database_encryption
Expand All @@ -297,6 +288,7 @@ resource "google_container_cluster" "primary" {
}
}

{% if beta_cluster %}
dynamic "authenticator_groups_config" {
for_each = local.cluster_authenticator_security_group
content {
Expand Down Expand Up @@ -397,10 +389,8 @@ resource "google_container_node_pool" "pools" {
{% endif %}
project = var.project_id
location = local.location
{% if beta_cluster %}
// use node_locations if provided, defaults to cluster level node_locations if not specified
node_locations = lookup(each.value, "node_locations", "") != "" ? split(",", each.value["node_locations"]) : null
{% endif %}

cluster = google_container_cluster.primary.name

Expand Down Expand Up @@ -501,7 +491,6 @@ resource "google_container_node_pool" "pools" {
count = guest_accelerator["count"]
}
]
{% if beta_cluster %}

dynamic "workload_metadata_config" {
for_each = local.cluster_node_metadata_config
Expand All @@ -510,7 +499,7 @@ resource "google_container_node_pool" "pools" {
node_metadata = lookup(each.value, "node_metadata", workload_metadata_config.value.node_metadata)
}
}

{% if beta_cluster %}
dynamic "sandbox_config" {
for_each = local.cluster_sandbox_enabled

Expand Down
Loading

0 comments on commit 2cb4fae

Please sign in to comment.