Workload Identity gets recreated everytime a change at GKE happens

[jona7o]

Hi, i am using the identity-workload module for several namespaces. When i change something in the main GKE module (for example remove a node-pool or anything else) all workload identity modules wants to delete and recreate everything because the value of the data is not known. This is a small part of the planed change when i add a node pool. If i confirm the action everything is fine at the end, but when i add another thing in the gke module the same happens again and again. I am using version 16.0.1 and i upgraded everything from 12.0.X with Terraform 1.0.4.

  # module.komro[0].module.app-workload-identity.data.google_service_account.cluster_service_account will be read during apply
  # (config refers to values not yet known)
 <= data "google_service_account" "cluster_service_account"  {
      ~ display_name = "GCP SA bound to K8S SA komroapp" -> (known after apply)
      ~ email        = "komroapp@my-project.iam.gserviceaccount.com" -> (known after apply)
      ~ id           = "projects/my-project/serviceAccounts/komroapp@my-project.iam.gserviceaccount.com" -> (known after apply)
      ~ name         = "projects/my-project/serviceAccounts/komroapp@my-project.iam.gserviceaccount.com" -> (known after apply)
      ~ unique_id    = "112212820338593187232" -> (known after apply)
        # (2 unchanged attributes hidden)
    }

  # module.komro[0].module.app-workload-identity.google_service_account_iam_member.main must be replaced
-/+ resource "google_service_account_iam_member" "main" {
      ~ etag               = "BwXI435kut0=" -> (known after apply)
      ~ id                 = "projects/my-project/serviceAccounts/komroapp@my-project.iam.gserviceaccount.com/roles/iam.workloadIdentityUser/serviceAccount:my-project.svc.id.goog[komro/komroapp]" -> (known after apply)
      ~ service_account_id = "projects/my-project/serviceAccounts/komroapp@my-project.iam.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.komro[0].module.app-workload-identity.kubernetes_service_account.main[0] will be updated in-place
  ~ resource "kubernetes_service_account" "main" {
        id                              = "komro/komroapp"
        # (2 unchanged attributes hidden)

      ~ metadata {
          ~ annotations      = {
              - "iam.gke.io/gcp-service-account" = "komroapp@my-project.iam.gserviceaccount.com"
            } -> (known after apply)
            name             = "komroapp"
            # (5 unchanged attributes hidden)
        }
    }

Plan: 61 to add, 18 to change, 62 to destroy.

[morgante]

Do you mind sharing your Terraform configuration? I'm wondering what is causing Terraform to attempt to refresh the data source, since presumably your service account isn't actually changing.

[jona7o]

I think i can identity the problem much better now.

We are using google provider v3.78.0 and i tried upgrading to version 16.01 as described in my issue.

In 16.01 the following data block was added and i think this is the root course of the problem. The data block gets evaluated every time, and all depending module get reinitialized in the second run. If i take the workload-identity module from version 15.02 everything is fine and fixed. Not sure if this a problem of the gcp provider?

terraform-google-kubernetes-engine/modules/workload-identity/main.tf

Lines 32 to 36 in 8764b76

	data "google_service_account" "cluster_service_account" {
	# This will cause Terraform to block looking up details until the service account is created
	account_id = var.use_existing_gcp_sa ? local.gcp_given_name : google_service_account.cluster_service_account[0].account_id
	project = var.project_id
	}