Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GCE PD CSI Driver beta support #497

Merged
merged 8 commits into from
Apr 28, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions autogen/main/cluster.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,10 @@ resource "google_container_cluster" "primary" {
dns_cache_config {
enabled = var.dns_cache
}

gce_persistent_disk_csi_driver_config {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make this a dynamic block, as I've observed consistent issues with the behavior of addon blocks in disabled states.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, i based this on the initial DNS cache PR. Did you want me to fix that addon in this PR too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-  cluster_gce_pd_csi_config = var.gce_pd_csi_driver ? [{ enabled = true }] : []
+  cluster_gce_pd_csi_config = var.gce_pd_csi_driver ? [{ enabled = true }] : [{ enabled = false }]

I couldn't disable this addon with the initial dynamic block approach (based on istio addon), one difference with the older addons is that newer addon config blocks (dns/csi) use enabled rather then disabled states. When trying to plan it doesn't indicate a change needs to be made when trying to flip back to disabled

I was only able to disable when forcing a dynamic block by proving a explicit [{ enabled = false }]. So the dynamic block is always going to trigger now, do you still want to keep this approach then?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, that's odd. It might indeed by driven by the defaults.

Can you try importing an existing cluster (with this disabled) using the static block approach? If import is successful, then I'm fine with using that approach.

enabled = var.gce_pd_csi_driver
}
{% endif %}
}

Expand Down
1 change: 1 addition & 0 deletions autogen/main/main.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ locals {
cluster_istio_enabled = ! local.cluster_output_istio_disabled
cluster_cloudrun_enabled = var.cloudrun
cluster_dns_cache_enabled = var.dns_cache
cluster_gce_pd_csi_driver_enabled = var.gce_pd_csi_driver
cluster_pod_security_policy_enabled = local.cluster_output_pod_security_policy_enabled
cluster_intranode_visibility_enabled = local.cluster_output_intranode_visbility_enabled
cluster_vertical_pod_autoscaling_enabled = local.cluster_output_vertical_pod_autoscaling_enabled
Expand Down
5 changes: 5 additions & 0 deletions autogen/main/outputs.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,11 @@ output "dns_cache_enabled" {
value = local.cluster_dns_cache_enabled
}

output "cluster_gce_pd_csi_driver_enabled" {
description = "Whether GCE Persistent Disk CSI Driver enabled"
Dev25 marked this conversation as resolved.
Show resolved Hide resolved
value = local.cluster_gce_pd_csi_driver_enabled
Dev25 marked this conversation as resolved.
Show resolved Hide resolved
}

output "pod_security_policy_enabled" {
description = "Whether pod security policy is enabled"
value = local.cluster_pod_security_policy_enabled
Expand Down
6 changes: 6 additions & 0 deletions autogen/main/variables.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,12 @@ variable "dns_cache" {
default = false
}

variable "gce_pd_csi_driver" {
type = bool
description = "(Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver."
default = false
}

variable "database_encryption" {
description = "Application-layer Secrets Encryption settings. The object format is {state = string, key_name = string}. Valid values of state are: \"ENCRYPTED\"; \"DECRYPTED\". key_name is the name of a CloudKMS key."
type = list(object({ state = string, key_name = string }))
Expand Down
2 changes: 1 addition & 1 deletion autogen/main/versions.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ terraform {

required_providers {
{% if beta_cluster %}
google-beta = ">= 3.16, <4.0.0"
google-beta = ">= 3.19, <4.0.0"
{% else %}
google = ">= 3.16, <4.0.0"
{% endif %}
Expand Down
2 changes: 1 addition & 1 deletion examples/node_pool/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ locals {
}

provider "google-beta" {
version = "~> 3.16.0"
version = "~> 3.19.0"
region = var.region
}

Expand Down
2 changes: 1 addition & 1 deletion examples/node_pool_update_variant_beta/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ locals {
}

provider "google-beta" {
version = "~> 3.16.0"
version = "~> 3.19.0"
credentials = file(var.credentials_path)
region = var.region
}
Expand Down
2 changes: 1 addition & 1 deletion examples/safer_cluster/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ provider "google" {
}

provider "google-beta" {
version = "~> 3.16.0"
version = "~> 3.19.0"
}

module "gke" {
Expand Down
1 change: 1 addition & 0 deletions examples/simple_regional_beta/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This example illustrates how to create a simple cluster with beta features.
| database\_encryption | Application-layer Secrets Encryption settings. The object format is {state = string, key_name = string}. Valid values of state are: "ENCRYPTED"; "DECRYPTED". key_name is the name of a CloudKMS key. | object | `<list>` | no |
| dns\_cache | (Beta) The status of the NodeLocal DNSCache addon. | bool | `"false"` | no |
| enable\_binary\_authorization | Enable BinAuthZ Admission controller | string | `"false"` | no |
| gce\_pd\_csi\_driver | (Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver. | bool | `"false"` | no |
| ip\_range\_pods | The secondary ip range to use for pods | string | n/a | yes |
| ip\_range\_services | The secondary ip range to use for services | string | n/a | yes |
| istio | Boolean to enable / disable Istio | string | `"true"` | no |
Expand Down
3 changes: 2 additions & 1 deletion examples/simple_regional_beta/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ locals {
}

provider "google-beta" {
version = "~> 3.16.0"
version = "~> 3.19.0"
region = var.region
}

Expand All @@ -39,6 +39,7 @@ module "gke" {
istio = var.istio
cloudrun = var.cloudrun
dns_cache = var.dns_cache
gce_pd_csi_driver = var.gce_pd_csi_driver
node_metadata = var.node_metadata
sandbox_enabled = var.sandbox_enabled
remove_default_node_pool = var.remove_default_node_pool
Expand Down
6 changes: 6 additions & 0 deletions examples/simple_regional_beta/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,12 @@ variable "dns_cache" {
default = false
}

variable "gce_pd_csi_driver" {
type = bool
description = "(Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver."
default = false
}

variable "node_metadata" {
description = "Specifies how node metadata is exposed to the workload running on the node"
default = "SECURE"
Expand Down
9 changes: 5 additions & 4 deletions examples/simple_regional_private_beta/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ provider "google" {
}

provider "google-beta" {
version = "~> 3.16.0"
version = "~> 3.19.0"
region = var.region
}

Expand Down Expand Up @@ -56,9 +56,10 @@ module "gke" {
},
]

istio = var.istio
cloudrun = var.cloudrun
dns_cache = var.dns_cache
istio = var.istio
cloudrun = var.cloudrun
dns_cache = var.dns_cache
gce_pd_csi_driver = var.gce_pd_csi_driver
}

data "google_client_config" "default" {
Expand Down
6 changes: 6 additions & 0 deletions examples/simple_regional_private_beta/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,9 @@ variable "dns_cache" {
description = "Boolean to enable / disable NodeLocal DNSCache "
default = false
}

variable "gce_pd_csi_driver" {
type = bool
description = "(Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver."
default = false
}
2 changes: 1 addition & 1 deletion examples/workload_metadata_config/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ locals {
}

provider "google-beta" {
version = "~> 3.16.0"
version = "~> 3.19.0"
region = var.region
}

Expand Down
2 changes: 2 additions & 0 deletions modules/beta-private-cluster-update-variant/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,7 @@ Then perform the following commands on the root folder:
| enable\_vertical\_pod\_autoscaling | Vertical Pod Autoscaling automatically adjusts the resources of pods controlled by it | bool | `"false"` | no |
| firewall\_inbound\_ports | List of TCP ports for admission/webhook controllers | list(string) | `<list>` | no |
| firewall\_priority | Priority rule for firewall rules | number | `"1000"` | no |
| gce\_pd\_csi\_driver | (Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver. | bool | `"false"` | no |
| grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer role. | bool | `"false"` | no |
| horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | bool | `"true"` | no |
| http\_load\_balancing | Enable httpload balancer addon | bool | `"true"` | no |
Expand Down Expand Up @@ -238,6 +239,7 @@ Then perform the following commands on the root folder:
|------|-------------|
| ca\_certificate | Cluster ca certificate (base64 encoded) |
| cloudrun\_enabled | Whether CloudRun enabled |
| cluster\_gce\_pd\_csi\_driver\_enabled | Whether GCE Persistent Disk CSI Driver enabled |
| dns\_cache\_enabled | Whether DNS Cache enabled |
| endpoint | Cluster endpoint |
| horizontal\_pod\_autoscaling\_enabled | Whether horizontal pod autoscaling enabled |
Expand Down
4 changes: 4 additions & 0 deletions modules/beta-private-cluster-update-variant/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@ resource "google_container_cluster" "primary" {
dns_cache_config {
enabled = var.dns_cache
}

gce_persistent_disk_csi_driver_config {
enabled = var.gce_pd_csi_driver
}
}

ip_allocation_policy {
Expand Down
1 change: 1 addition & 0 deletions modules/beta-private-cluster-update-variant/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ locals {
cluster_istio_enabled = ! local.cluster_output_istio_disabled
cluster_cloudrun_enabled = var.cloudrun
cluster_dns_cache_enabled = var.dns_cache
cluster_gce_pd_csi_driver_enabled = var.gce_pd_csi_driver
cluster_pod_security_policy_enabled = local.cluster_output_pod_security_policy_enabled
cluster_intranode_visibility_enabled = local.cluster_output_intranode_visbility_enabled
cluster_vertical_pod_autoscaling_enabled = local.cluster_output_vertical_pod_autoscaling_enabled
Expand Down
5 changes: 5 additions & 0 deletions modules/beta-private-cluster-update-variant/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,11 @@ output "dns_cache_enabled" {
value = local.cluster_dns_cache_enabled
}

output "cluster_gce_pd_csi_driver_enabled" {
description = "Whether GCE Persistent Disk CSI Driver enabled"
value = local.cluster_gce_pd_csi_driver_enabled
}

output "pod_security_policy_enabled" {
description = "Whether pod security policy is enabled"
value = local.cluster_pod_security_policy_enabled
Expand Down
6 changes: 6 additions & 0 deletions modules/beta-private-cluster-update-variant/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -410,6 +410,12 @@ variable "dns_cache" {
default = false
}

variable "gce_pd_csi_driver" {
type = bool
description = "(Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver."
default = false
}

variable "database_encryption" {
description = "Application-layer Secrets Encryption settings. The object format is {state = string, key_name = string}. Valid values of state are: \"ENCRYPTED\"; \"DECRYPTED\". key_name is the name of a CloudKMS key."
type = list(object({ state = string, key_name = string }))
Expand Down
2 changes: 1 addition & 1 deletion modules/beta-private-cluster-update-variant/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ terraform {
required_version = "~> 0.12.6"

required_providers {
google-beta = ">= 3.16, <4.0.0"
google-beta = ">= 3.19, <4.0.0"
}
}
2 changes: 2 additions & 0 deletions modules/beta-private-cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@ Then perform the following commands on the root folder:
| enable\_vertical\_pod\_autoscaling | Vertical Pod Autoscaling automatically adjusts the resources of pods controlled by it | bool | `"false"` | no |
| firewall\_inbound\_ports | List of TCP ports for admission/webhook controllers | list(string) | `<list>` | no |
| firewall\_priority | Priority rule for firewall rules | number | `"1000"` | no |
| gce\_pd\_csi\_driver | (Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver. | bool | `"false"` | no |
| grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer role. | bool | `"false"` | no |
| horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | bool | `"true"` | no |
| http\_load\_balancing | Enable httpload balancer addon | bool | `"true"` | no |
Expand Down Expand Up @@ -216,6 +217,7 @@ Then perform the following commands on the root folder:
|------|-------------|
| ca\_certificate | Cluster ca certificate (base64 encoded) |
| cloudrun\_enabled | Whether CloudRun enabled |
| cluster\_gce\_pd\_csi\_driver\_enabled | Whether GCE Persistent Disk CSI Driver enabled |
| dns\_cache\_enabled | Whether DNS Cache enabled |
| endpoint | Cluster endpoint |
| horizontal\_pod\_autoscaling\_enabled | Whether horizontal pod autoscaling enabled |
Expand Down
4 changes: 4 additions & 0 deletions modules/beta-private-cluster/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@ resource "google_container_cluster" "primary" {
dns_cache_config {
enabled = var.dns_cache
}

gce_persistent_disk_csi_driver_config {
enabled = var.gce_pd_csi_driver
}
}

ip_allocation_policy {
Expand Down
1 change: 1 addition & 0 deletions modules/beta-private-cluster/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ locals {
cluster_istio_enabled = ! local.cluster_output_istio_disabled
cluster_cloudrun_enabled = var.cloudrun
cluster_dns_cache_enabled = var.dns_cache
cluster_gce_pd_csi_driver_enabled = var.gce_pd_csi_driver
cluster_pod_security_policy_enabled = local.cluster_output_pod_security_policy_enabled
cluster_intranode_visibility_enabled = local.cluster_output_intranode_visbility_enabled
cluster_vertical_pod_autoscaling_enabled = local.cluster_output_vertical_pod_autoscaling_enabled
Expand Down
5 changes: 5 additions & 0 deletions modules/beta-private-cluster/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,11 @@ output "dns_cache_enabled" {
value = local.cluster_dns_cache_enabled
}

output "cluster_gce_pd_csi_driver_enabled" {
description = "Whether GCE Persistent Disk CSI Driver enabled"
value = local.cluster_gce_pd_csi_driver_enabled
}

output "pod_security_policy_enabled" {
description = "Whether pod security policy is enabled"
value = local.cluster_pod_security_policy_enabled
Expand Down
6 changes: 6 additions & 0 deletions modules/beta-private-cluster/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -410,6 +410,12 @@ variable "dns_cache" {
default = false
}

variable "gce_pd_csi_driver" {
type = bool
description = "(Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver."
default = false
}

variable "database_encryption" {
description = "Application-layer Secrets Encryption settings. The object format is {state = string, key_name = string}. Valid values of state are: \"ENCRYPTED\"; \"DECRYPTED\". key_name is the name of a CloudKMS key."
type = list(object({ state = string, key_name = string }))
Expand Down
2 changes: 1 addition & 1 deletion modules/beta-private-cluster/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ terraform {
required_version = "~> 0.12.6"

required_providers {
google-beta = ">= 3.16, <4.0.0"
google-beta = ">= 3.19, <4.0.0"
}
}
2 changes: 2 additions & 0 deletions modules/beta-public-cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ Then perform the following commands on the root folder:
| enable\_vertical\_pod\_autoscaling | Vertical Pod Autoscaling automatically adjusts the resources of pods controlled by it | bool | `"false"` | no |
| firewall\_inbound\_ports | List of TCP ports for admission/webhook controllers | list(string) | `<list>` | no |
| firewall\_priority | Priority rule for firewall rules | number | `"1000"` | no |
| gce\_pd\_csi\_driver | (Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver. | bool | `"false"` | no |
| grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer role. | bool | `"false"` | no |
| horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | bool | `"true"` | no |
| http\_load\_balancing | Enable httpload balancer addon | bool | `"true"` | no |
Expand Down Expand Up @@ -194,6 +195,7 @@ Then perform the following commands on the root folder:
|------|-------------|
| ca\_certificate | Cluster ca certificate (base64 encoded) |
| cloudrun\_enabled | Whether CloudRun enabled |
| cluster\_gce\_pd\_csi\_driver\_enabled | Whether GCE Persistent Disk CSI Driver enabled |
| dns\_cache\_enabled | Whether DNS Cache enabled |
| endpoint | Cluster endpoint |
| horizontal\_pod\_autoscaling\_enabled | Whether horizontal pod autoscaling enabled |
Expand Down
4 changes: 4 additions & 0 deletions modules/beta-public-cluster/cluster.tf
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@ resource "google_container_cluster" "primary" {
dns_cache_config {
enabled = var.dns_cache
}

gce_persistent_disk_csi_driver_config {
enabled = var.gce_pd_csi_driver
}
}

ip_allocation_policy {
Expand Down
1 change: 1 addition & 0 deletions modules/beta-public-cluster/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@ locals {
cluster_istio_enabled = ! local.cluster_output_istio_disabled
cluster_cloudrun_enabled = var.cloudrun
cluster_dns_cache_enabled = var.dns_cache
cluster_gce_pd_csi_driver_enabled = var.gce_pd_csi_driver
cluster_pod_security_policy_enabled = local.cluster_output_pod_security_policy_enabled
cluster_intranode_visibility_enabled = local.cluster_output_intranode_visbility_enabled
cluster_vertical_pod_autoscaling_enabled = local.cluster_output_vertical_pod_autoscaling_enabled
Expand Down
5 changes: 5 additions & 0 deletions modules/beta-public-cluster/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,11 @@ output "dns_cache_enabled" {
value = local.cluster_dns_cache_enabled
}

output "cluster_gce_pd_csi_driver_enabled" {
description = "Whether GCE Persistent Disk CSI Driver enabled"
value = local.cluster_gce_pd_csi_driver_enabled
}

output "pod_security_policy_enabled" {
description = "Whether pod security policy is enabled"
value = local.cluster_pod_security_policy_enabled
Expand Down
6 changes: 6 additions & 0 deletions modules/beta-public-cluster/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -386,6 +386,12 @@ variable "dns_cache" {
default = false
}

variable "gce_pd_csi_driver" {
type = bool
description = "(Beta) Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver."
default = false
}

variable "database_encryption" {
description = "Application-layer Secrets Encryption settings. The object format is {state = string, key_name = string}. Valid values of state are: \"ENCRYPTED\"; \"DECRYPTED\". key_name is the name of a CloudKMS key."
type = list(object({ state = string, key_name = string }))
Expand Down
2 changes: 1 addition & 1 deletion modules/beta-public-cluster/versions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ terraform {
required_version = "~> 0.12.6"

required_providers {
google-beta = ">= 3.16, <4.0.0"
google-beta = ">= 3.19, <4.0.0"
}
}
2 changes: 2 additions & 0 deletions test/fixtures/beta_cluster/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ module "this" {

dns_cache = true

gce_pd_csi_driver = true

enable_binary_authorization = true

pod_security_policy_config = [{
Expand Down
3 changes: 3 additions & 0 deletions test/integration/beta_cluster/controls/gcloud.rb
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,9 @@
"dnsCacheConfig" => {
"enabled" => true,
}
"gcePersistentDiskCsiDriverConfig" => {
"enabled" => true,
}
})
end

Expand Down