Disabling a container cluster's network policy enforcement with an assignement results in endless changes detected

[pdecat]

Terraform Version

Same behavior with 1.5.0 and recent master@cc5f77e3:

# terraform version
Terraform v0.11.2
+ provider.google v1.5.0

Affected Resource(s)

• google_container_cluster

Terraform Configuration Files

1. configuration to enable network policy enforcement:

resource "google_container_cluster" "gke_cluster" {
  name               = "mycluster"
  zone               = "europe-west1-b"
  initial_node_count = "1"
  network            = "mynetwork"
  subnetwork         = "mysubnet"

  master_auth {
    username = "username"
    password = "password"
  }

  node_config {
    machine_type = "n1-standard-2"
  }

  network_policy {
    enabled  = true
    provider = "CALICO"
  }
}

2. configuration to disable network policy enforcement:

resource "google_container_cluster" "gke_cluster" {
  name               = "mycluster"
  zone               = "europe-west1-b"
  initial_node_count = "1"
  network            = "mynetwork"
  subnetwork         = "mysubnet"

  master_auth {
    username = "username"
    password = "password"
  }

  node_config {
    machine_type = "n1-standard-2"
  }

  network_policy {}
}

Notes: replacing network_policy {} by network_policy = [] has the same behavior with a slightly different diff:

  ~ google_container_cluster.gke_cluster                                                                                                                              
      network_policy.#:          "1" => "0"                                                                                                                                               
      network_policy.0.provider: "" => "PROVIDER_UNSPECIFIED"                                                                                                                             

Update: removing the network_policy assignment eliminates the diff but it prevents defining it from a variable and more importantly does not update the cluster.

Expected Behavior

After applying configuration 1 to enable network policy, then configuration 2 to disable it, the plan should no longer detect changes.

Actual Behavior

After applying configuration 1 to enable network policy, then configuration 2 to disable it, the plan still detects changes.

Steps to Reproduce

Enable network policy with configuration 1 then disable it with configuration 2 :

# terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ google_container_cluster.gke_cluster
      network_policy.0.provider: "" => "PROVIDER_UNSPECIFIED"


Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

google_container_cluster.gke_cluster: Modifying... (ID: mycluster)
  network_policy.0.provider: "" => "PROVIDER_UNSPECIFIED"
google_container_cluster.gke_cluster: Modifications complete after 2s (ID: mycluster)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

# terraform plan
[...]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ google_container_cluster.gke_cluster
      network_policy.0.provider: "" => "PROVIDER_UNSPECIFIED"


Plan: 0 to add, 1 to change, 0 to destroy.

Important Factoids

At some point, I passed the following to disable network policy on that cluster:

  network_policy {
    enabled = false
  }

This may have resulted in a special state on GKE API side as it returns an empty map for networkPolicy instead of nothing at all:

  
{
 "name": "mycluster",
 "initialNodeCount": 3,
 "nodeConfig": {
  "machineType": "n1-standard-2",
  "diskSizeGb": 100,
  "oauthScopes": [
   "https://www.googleapis.com/auth/compute",
   "https://www.googleapis.com/auth/devstorage.read_only",
   "https://www.googleapis.com/auth/service.management",
   "https://www.googleapis.com/auth/servicecontrol",
   "https://www.googleapis.com/auth/logging.write",
   "https://www.googleapis.com/auth/monitoring"
  ],
  "imageType": "COS",
  "serviceAccount": "default"
 },
 "masterAuth": {
  "username": "username",
  "password": "password",
  "clusterCaCertificate": "****",
  "clientCertificate": "****",
  "clientKey": "****"
 },
 "loggingService": "logging.googleapis.com",
 "monitoringService": "monitoring.googleapis.com",
 "network": "mynetwork",
 "clusterIpv4Cidr": "10.184.0.0/14",
 "addonsConfig": {
  "kubernetesDashboard": {
   "disabled": true
  },
  "networkPolicyConfig": {
  }
 },
 "subnetwork": "mysubnet",
 "nodePools": [
  {
   "name": "pool-1",
   "config": {
    "machineType": "n1-standard-2",
    "diskSizeGb": 100,
    "oauthScopes": [
     "https://www.googleapis.com/auth/compute",
     "https://www.googleapis.com/auth/devstorage.read_only",
     "https://www.googleapis.com/auth/service.management",
     "https://www.googleapis.com/auth/servicecontrol",
     "https://www.googleapis.com/auth/logging.write",
     "https://www.googleapis.com/auth/monitoring"
    ],
    "imageType": "COS",
    "serviceAccount": "default"
   },
   "initialNodeCount": 3,
   "autoscaling": {
   },
   "management": {
    "autoRepair": true
   },
   "selfLink": "https://container.googleapis.com/v1/projects/myproject/zones/europe-west1-b/clusters/mycluster/nodePools/pool-1",
   "version": "1.8.5-gke.0",
   "instanceGroupUrls": [
    "https://www.googleapis.com/compute/v1/projects/myproject/zones/europe-west1-b/instanceGroupManagers/gke-myproject-europe-w-pool-1-*****-grp"
   ],
   "status": "RUNNING"
  }
 ],
 "locations": [
  "europe-west1-b"
 ],
 "labelFingerprint": "*****",
 "legacyAbac": {
  "enabled": true
 },
 "networkPolicy": {
 },
 "maintenancePolicy": {
  "window": {
   "dailyMaintenanceWindow": {
    "startTime": "06:00",
    "duration": "PT4H0M0S"
   }
  }
 },
 "selfLink": "https://container.googleapis.com/v1/projects/myproject/zones/europe-west1-b/clusters/mycluster",
 "zone": "europe-west1-b",
 "endpoint": "******",
 "initialClusterVersion": "1.4.5",
 "currentMasterVersion": "1.8.5-gke.0",
 "currentNodeVersion": "1.8.5-gke.0",
 "createTime": "2016-11-03T10:35:06+00:00",
 "status": "RUNNING",
 "nodeIpv4CidrSize": 24,
 "servicesIpv4Cidr": "10.187.240.0/20",
 "instanceGroupUrls": [
  "https://www.googleapis.com/compute/v1/projects/myproject/zones/europe-west1-b/instanceGroupManagers/gke-myproject-europe-w-pool-1-*****-grp"
 ],
 "currentNodeCount": 3
}

My ultimate goal is to be able to pass a variable to modify network_policy with:

  network_policy = ["${var.gke_network_policy}"]

or:

  network_policy = "${var.gke_network_policy}"

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!