Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to enable private nodes when creating nodes with specified pod ip range #1493

Closed
Shelby-Jiao opened this issue Dec 9, 2022 · 4 comments · Fixed by #1514
Closed

Unable to enable private nodes when creating nodes with specified pod ip range #1493

Shelby-Jiao opened this issue Dec 9, 2022 · 4 comments · Fixed by #1514
Labels
bug Something isn't working

Comments

@Shelby-Jiao
Copy link

TL;DR

We are trying to create private nodes with our specified pod ip range
So we set the variable enable_private_nodes to true.
After the change, the terraform plan is broken. It seems the provider kubernetes couldn't make a request with error
Error: error creating NodePool: googleapi: Error 400: EnablePrivateNodes must be enabled for private clusters with valid masterIpv4Cidr., badRequest

I find there's no variables for changing this in resource "google_container_node_pool" "pools"

Expected behavior

a new node should be created as described below:
module.gke.google_container_node_pool.pools["default-node-pool"] will be created with such network config
resource "google_container_node_pool" "pools"

network_config {
enable_private_nodes = (known after apply)
pod_ipv4_cidr_block = (known after apply)
pod_range = "blablabla"
}

Observed behavior

Error: error creating NodePool: googleapi: Error 400: EnablePrivateNodes must be enabled for private clusters with valid masterIpv4Cidr., badRequest with module.gke.google_container_node_pool.pools["default-node-pool"], on .terraform/modules/gke/modules/beta-private-cluster/cluster.tf line 416, in resource "google_container_node_pool" "pools": 416: resource "google_container_node_pool" "pools" {

Terraform Configuration

We use the module beta-private-cluster just like the example in README.md.

# google_client_config and kubernetes provider must be explicitly specified like the following.
data "google_client_config" "default" {}

provider "kubernetes" {
  host                   = "https://${module.gke.endpoint}"    # Here we reference the endpoint of the cluster using the output of module gke
  token                  = data.google_client_config.default.access_token
  cluster_ca_certificate = base64decode(module.gke.ca_certificate)
}

module "gke" {
  source                     = "terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster"
  project_id                 = "<PROJECT ID>"
  name                       = "gke-test-1"
  region                     = "us-central1"
  zones                      = ["us-central1-a", "us-central1-b", "us-central1-f"]
  network                    = "vpc-01"
  subnetwork                 = "us-central1-01"
  ip_range_pods              = "us-central1-01-gke-01-pods"
  ip_range_services          = "us-central1-01-gke-01-services"
  http_load_balancing        = false
  network_policy             = false
  horizontal_pod_autoscaling = true
  filestore_csi_driver       = false
  enable_private_endpoint    = var.enable_private_endpoint      # var.enable_private_endpoint set to true to enable the private endpoint
  enable_private_nodes       = true
  master_ipv4_cidr_block     = "10.0.0.0/28"
  istio                      = true
  cloudrun                   = true
  dns_cache                  = false
  master_authorized_networks = var.enable_private_endpoint ? [] : concat(var.master_authorized_networks, var.additional_master_authorized_networks)      # remove master_authorized_networks once we enable the private endpoint
  node_pools = [
    {
      name               = "blabla"
      machine_type       = "e2-standard-2"
      min_count          = 0
      max_count          = 100
      disk_size_gb       = 100
      disk_type          = "pd-ssd"
      image_type         = "COS_CONTAINERD"
      auto_repair        = true
      auto_upgrade       = true
      preemptible        = false
      spot               = false
      initial_node_count = 1
      max_pods_per_node  = 110
      node_locations     = "asia-southeast1-a,asia-southeast1-b"
      pod_range          = "SPECIFIED POD RANGE"
    }
]

Terraform Version

$ terraform --version
Terraform v1.0.9
on linux_amd64

Additional information

No response
@Shelby-Jiao Shelby-Jiao added the bug Something isn't working label Dec 9, 2022
@mjhuber
Copy link

mjhuber commented Dec 19, 2022

Experiencing the same issue.

Terraform Version

❯ terraform version
Terraform v1.0.0
on darwin_amd64
+ provider registry.terraform.io/hashicorp/external v2.2.3
+ provider registry.terraform.io/hashicorp/google v3.90.1
+ provider registry.terraform.io/hashicorp/google-beta v4.46.0
+ provider registry.terraform.io/hashicorp/kubernetes v2.16.1
+ provider registry.terraform.io/hashicorp/null v3.2.1
+ provider registry.terraform.io/hashicorp/random v3.4.3

Terraform Configuration

module "gke" {
  source                          = "terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster-update-variant"
  version                         = "24.1.0"
  project_id                      = <reacted>
  name                            = "staging"
  region                          = "us-east4"
  regional                        = true
  network                         = "stage"
  ip_range_pods                   = "gke-pods"
  ip_range_services               = "gke-services"
  subnetwork                      = google_compute_subnetwork.gke_network.name
  kubernetes_version              = "1.23.13-gke.900"
  remove_default_node_pool        = true
  release_channel                 = null
  maintenance_start_time          = "01:00"
  initial_node_count              = 0
  enable_pod_security_policy      = false
  grant_registry_access           = true
  create_service_account          = true
  default_max_pods_per_node       = 100
  http_load_balancing             = true
  horizontal_pod_autoscaling      = true
  enable_vertical_pod_autoscaling = true
  network_policy                  = true
  enable_private_endpoint         = true
  enable_private_nodes            = true
  master_ipv4_cidr_block          = "10.101.255.240/28"
  istio                           = false
  cloudrun                        = false
  dns_cache                       = true
  add_cluster_firewall_rules      = true
  firewall_inbound_ports          = [
      "8443", # OPA
      "6443", # keda
      "8080", # vault
  ]
  master_authorized_networks = [
    {
      cidr_block   = "0.0.0.0/0",
      display_name = "Public-Creds-Restricted",
    }
  ]

  cluster_autoscaling = {
    enableNodeAutoprovisioning = false,
    autoscaling_profile        = "BALANCED",
    enabled                    = false,
    min_cpu_cores              = 0,
    max_cpu_cores              = 0,
    min_memory_gb              = 0,
    max_memory_gb              = 0,
    gpu_resources              = null,
  }

  node_pools = [
    {
      name         = "standard-1"
      machine_type = "n2d-highcpu-16"
      version            = "1.22.9-gke.2000"
      min_count          = 0
      max_count          = 170
      node_count         = 4
      initial_node_count = 1
      local_ssd_count    = 0
      disk_size_gb       = 200
      disk_type          = "pd-ssd"
      max_surge          = 3
      max_unavailable    = 2
      auto_repair        = true
      auto_upgrade       = false
      autoscaling        = true
      service_account    = google_service_account.gke.email
      preemptible        = false
      max_pods_per_node  = 64
      pod_range          = "gke-pods-1"
    },
    {
      name               = "maxscale"
      machine_type       = "n2d-standard-2"
      version            = "1.22.9-gke.2000"
      min_count          = 0
      max_count          = 20
      node_count         = 1
      initial_node_count = 1
      local_ssd_count    = 0
      disk_size_gb       = 200
      disk_type          = "pd-ssd"
      max_surge          = 3
      max_unavailable    = 2
      node_metadata      = "GCE_METADATA"
      auto_repair        = true
      auto_upgrade       = false
      autoscaling        = true
      service_account    = google_service_account.gke.email
      preemptible        = false
      max_pods_per_node  = 30
      pod_range          = "gke-pods"
    },
  ]

  node_pools_oauth_scopes = {
    all = [
      "https://www.googleapis.com/auth/cloud-platform",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/cloud-platform",
      "https://www.googleapis.com/auth/ndev.clouddns.readwrite",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/compute",
    ]
  }
  node_pools_labels = {
    all = {}

    maxscale = {
      role = "maxscale"
    }
  }
  node_pools_taints = {
    all = []

    maxscale = [
      {
        key    = "role"
        value  = "maxscale"
        effect = "NO_SCHEDULE"
      },
    ]
  }

  cluster_resource_labels = {
    "env"   = "stage",
    "owner" = "<redacted>",
    "role"  = "k8s"
    "name"  = "staging"
  }
}

Relevant terraform debug logs from a targeted apply:

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: 2022-12-19T11:17:56.652-0600 [DEBUG] command: asking for input: "\nDo you want to perform these actions?"
yes

2022-12-19T11:18:04.432-0600 [INFO]  backend/local: apply calling Apply
2022-12-19T11:18:04.433-0600 [INFO]  terraform: building graph: GraphTypeApply
2022-12-19T11:18:04.440-0600 [DEBUG] Resource instance state not found for node "module.gke.google_container_node_pool.pools[\"standard-2\"]", instance module.gke.google_container_node_pool.pools["standard-2"]
2022-12-19T11:18:04.443-0600 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/random"], implied first by module.gke.random_id.name (expand)
2022-12-19T11:18:04.444-0600 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/google-beta"], implied first by module.gke.data.google_compute_zones.available (expand)
2022-12-19T11:18:04.444-0600 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/kubernetes"], implied first by module.gke.kubernetes_config_map_v1_data.kube-dns-upstream-namservers (expand)
2022-12-19T11:18:04.445-0600 [DEBUG] ProviderTransformer: "module.gke.google_service_account.cluster_service_account (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_compute_firewall.master_webhooks (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.kubernetes_config_map_v1_data.kube-dns-upstream-namservers (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/kubernetes"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_container_node_pool.windows_pools (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google-beta"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_compute_firewall.shadow_allow_pods (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_project_iam_member.cluster_service_account-resourceMetadata-writer (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.random_shuffle.available_zones (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/random"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_compute_firewall.shadow_allow_nodes (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.data.google_compute_zones.available (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google-beta"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "google_storage_bucket_iam_member.gcr-access (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_container_node_pool.pools (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google-beta"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "google_storage_bucket_iam_member.kube_node_tracer (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.kubernetes_config_map.ip-masq-agent (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/kubernetes"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_project_iam_member.cluster_service_account-log_writer (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_project_iam_member.cluster_service_account-monitoring_viewer (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.data.google_container_engine_versions.zone (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "google_storage_bucket.kube_node_tracer (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "google_service_account.gke (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.random_id.name (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/random"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "google_compute_firewall.stage_gke_ingress (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.kubernetes_config_map_v1_data.kube-dns (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/kubernetes"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_project_iam_member.cluster_service_account-gcr (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "google_compute_subnetwork.gke_network (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.kubernetes_config_map_v1_data.kube-dns-upstream-nameservers-and-stub-domains (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/kubernetes"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_container_cluster.primary (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google-beta"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_project_iam_member.cluster_service_account-artifact-registry (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_compute_firewall.intra_egress (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.data.google_container_engine_versions.region (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.446-0600 [DEBUG] ProviderTransformer: "module.gke.google_compute_firewall.tpu_egress (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.447-0600 [DEBUG] ProviderTransformer: "module.gke.google_compute_firewall.shadow_allow_master (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.447-0600 [DEBUG] ProviderTransformer: "module.gke.google_project_iam_member.cluster_service_account-metric_writer (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.447-0600 [DEBUG] ProviderTransformer: "module.gke.google_container_node_pool.pools[\"standard-2\"]" (*terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/hashicorp/google-beta"]
2022-12-19T11:18:04.447-0600 [DEBUG] ProviderTransformer: "module.gke.random_string.cluster_service_account_suffix (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/random"]
2022-12-19T11:18:04.447-0600 [DEBUG] ProviderTransformer: "module.gke.data.google_compute_subnetwork.gke_subnetwork (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/google"]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.var.node_pools (expand)" references: [google_service_account.gke (expand) google_service_account.gke (expand) google_service_account.gke (expand) var.gke]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.var.dns_cache (expand)" references: [var.gke]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.local.workload_identity_enabled (expand)" references: [module.gke.var.identity_namespace (expand) module.gke.var.identity_namespace (expand)]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.output.location (expand)" references: [module.gke.local.cluster_location (expand)]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "google_storage_bucket_iam_member.gcr-access (expand)" references: [var.gke]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.data.google_compute_zones.available (expand)" references: []
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_resource_consumption_export (expand)" references: []
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.local.service_account_list (expand)" references: [module.gke.google_service_account.cluster_service_account (expand)]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_cloudrun_config_load_balancer_config (expand)" references: [module.gke.var.cloudrun (expand) module.gke.var.cloudrun_load_balancer_type (expand) module.gke.var.cloudrun_load_balancer_type (expand)]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.output.cloudrun_enabled (expand)" references: [module.gke.local.cluster_cloudrun_enabled (expand)]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.google_project_iam_member.cluster_service_account-monitoring_viewer (expand)" references: [module.gke.var.create_service_account (expand)]
2022-12-19T11:18:04.460-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_id (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_node_pools_versions (expand)" references: [module.gke.google_container_node_pool.pools (expand) module.gke.google_container_node_pool.pools["standard-2"] module.gke.google_container_node_pool.windows_pools (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_pools_tags (expand)" references: [module.gke.var.node_pools (expand) module.gke.var.node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.node_pools_tags (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.output.endpoint (expand)" references: [module.gke.local.cluster_endpoint (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_node_pool.pools (expand) module.gke.google_container_node_pool.pools["standard-2"]]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_region (expand)" references: [module.gke.var.regional (expand) module.gke.var.region (expand) module.gke.local.cluster_location (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.output.logging_service (expand)" references: [module.gke.local.cluster_logging_service (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/google\"]" references: [var.project_id var.region]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.windows_node_pool_names (expand)" references: [module.gke.var.windows_node_pools (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.var.configure_ip_masq (expand)" references: []
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.var.http_load_balancing (expand)" references: [var.gke]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_pools_taints (expand)" references: [module.gke.var.node_pools (expand) module.gke.var.node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.node_pools_taints (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.var.datapath_provider (expand)" references: []
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.var.authenticator_security_group (expand)" references: []
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_pools_oauth_scopes (expand)" references: [module.gke.var.node_pools (expand) module.gke.var.node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.node_pools_oauth_scopes (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/random\"]" references: []
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.google_compute_firewall.shadow_allow_master (expand)" references: [module.gke.var.add_shadow_firewall_rules (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.output.dns_cache_enabled (expand)" references: [module.gke.local.cluster_dns_cache_enabled (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.var.identity_namespace (expand)" references: []
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.var.sandbox_enabled (expand)" references: []
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_horizontal_pod_autoscaling_enabled (expand)" references: [module.gke.local.cluster_output_horizontal_pod_autoscaling_enabled (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_logging_service (expand)" references: [module.gke.local.cluster_output_logging_service (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_name_parts_from_nodepool (expand)" references: [module.gke.google_container_node_pool.pools (expand) module.gke.google_container_node_pool.pools["standard-2"]]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.output.vertical_pod_autoscaling_enabled (expand)" references: [module.gke.local.cluster_vertical_pod_autoscaling_enabled (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "google_compute_firewall.stage_gke_ingress (expand)" references: []
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.location (expand)" references: [module.gke.var.regional (expand) module.gke.var.region (expand) module.gke.var.zones (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_network_policy (expand)" references: [module.gke.var.network_policy (expand) module.gke.var.network_policy_provider (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_endpoint (expand)" references: [module.gke.var.enable_private_nodes (expand) module.gke.google_container_cluster.primary (expand) module.gke.var.deploy_using_private_endpoint (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "google_storage_bucket.kube_node_tracer (expand)" references: [var.tcpdump_bucket]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_master_auth_map (expand)" references: [module.gke.local.cluster_master_auth_list_layer2 (expand)]
2022-12-19T11:18:04.461-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_dns_cache_enabled (expand)" references: [module.gke.var.dns_cache (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.issue_client_certificate (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_http_load_balancing_enabled (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.tpu_ipv4_cidr_block (expand)" references: [module.gke.var.enable_tpu (expand) module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.http_load_balancing_enabled (expand)" references: [module.gke.local.cluster_http_load_balancing_enabled (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.maintenance_exclusions (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cluster_dns_domain (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.gce_pd_csi_driver (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.instance_group_urls (expand)" references: [module.gke.google_container_node_pool.pools (expand) module.gke.google_container_node_pool.pools["standard-2"]]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.release_channel (expand)" references: [module.gke.var.release_channel (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.timeouts (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_network_policy_enabled (expand)" references: [module.gke.local.cluster_output_network_policy_enabled (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_cloudrun_config (expand)" references: [module.gke.var.cloudrun (expand) module.gke.local.cluster_cloudrun_config_load_balancer_config (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_pools_labels (expand)" references: [module.gke.var.node_pools (expand) module.gke.var.node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.node_pools_labels (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.node_pools_versions (expand)" references: [module.gke.local.cluster_node_pools_versions (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.type (expand)" references: [module.gke.local.cluster_type (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_tpu (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.regional (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_l4_ilb_subsetting (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_confidential_nodes (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.network_policy_enabled (expand)" references: [module.gke.local.cluster_network_policy_enabled (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.cluster_id (expand)" references: [module.gke.local.cluster_id (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "var.gke" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_istio_disabled (expand)" references: [module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_zones (expand)" references: [module.gke.local.cluster_output_zones (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.service_account (expand)" references: [module.gke.var.service_account (expand) module.gke.var.service_account (expand) module.gke.var.create_service_account (expand) module.gke.local.service_account_list (expand) module.gke.var.service_account (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.zones (expand)" references: [module.gke.local.cluster_zones (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.peering_name (expand)" references: [module.gke.local.cluster_peering_name (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.random_string.cluster_service_account_suffix (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.google_project_iam_member.cluster_service_account-metric_writer (expand)" references: [module.gke.var.create_service_account (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_endpoint_for_nodes (expand)" references: [module.gke.var.master_ipv4_cidr_block (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.google_compute_firewall.master_webhooks (expand)" references: [module.gke.var.add_cluster_firewall_rules (expand) module.gke.var.add_master_webhook_firewall_rules (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_network_egress_export (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.monitoring_enable_managed_prometheus (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.intranode_visibility_enabled (expand)" references: [module.gke.local.cluster_intranode_visibility_enabled (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.output.monitoring_service (expand)" references: [module.gke.local.cluster_monitoring_service (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.data.google_container_engine_versions.zone (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.logging_enabled_components (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.var.gke_backup_agent_config (expand)" references: []
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.master_version (expand)" references: [module.gke.var.regional (expand) module.gke.local.master_version_regional (expand) module.gke.local.master_version_zonal (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_zones (expand)" references: [module.gke.local.cluster_output_regional_zones (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke.google_compute_firewall.intra_egress (expand)" references: [module.gke.var.add_cluster_firewall_rules (expand)]
2022-12-19T11:18:04.462-0600 [DEBUG] ReferenceTransformer: "module.gke (close)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_pool_names (expand)" references: [module.gke.var.node_pools (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_identity_service_enabled (expand)" references: [module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.kubernetes_config_map_v1_data.kube-dns-upstream-namservers (expand)" references: [module.gke.local.custom_kube_dns_config (expand) module.gke.local.upstream_nameservers_config (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.data.google_container_engine_versions.region (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.node_metadata (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.network_policy_provider (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.registry_project_ids (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.output.master_authorized_networks_config (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.output.min_master_version (expand)" references: [module.gke.local.cluster_min_master_version (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "google_storage_bucket_iam_member.kube_node_tracer (expand)" references: [var.tcpdump_bucket]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.node_pools_linux_node_configs_sysctls (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.network_policy (expand)" references: [var.gke]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.maintenance_start_time (expand)" references: [var.gke]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.node_pools_oauth_scopes (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.google_compute_firewall.tpu_egress (expand)" references: [module.gke.var.add_cluster_firewall_rules (expand) module.gke.var.enable_tpu (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.google_service_account.cluster_service_account (expand)" references: [module.gke.var.create_service_account (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.disable_legacy_metadata_endpoints (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.monitoring_service (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.kubernetes_config_map_v1_data.kube-dns (expand)" references: [module.gke.local.custom_kube_dns_config (expand) module.gke.local.upstream_nameservers_config (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.add_shadow_firewall_rules (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_kubernetes_alpha (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.add_master_webhook_firewall_rules (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.description (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_pod_security_policy_enabled (expand)" references: [module.gke.local.cluster_output_pod_security_policy_enabled (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_maintenance_window_is_daily (expand)" references: [module.gke.local.cluster_maintenance_window_is_recurring (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.registry_projects_list (expand)" references: [module.gke.var.registry_project_ids (expand) module.gke.var.project_id (expand) module.gke.var.registry_project_ids (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.google_container_node_pool.windows_pools (expand)" references: [module.gke.local.windows_node_pools (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.network_project_id (expand)" references: [module.gke.var.network_project_id (expand) module.gke.var.network_project_id (expand) module.gke.var.project_id (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.initial_node_count (expand)" references: [var.gke]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.horizontal_pod_autoscaling (expand)" references: [var.gke]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_name_computed (expand)" references: [module.gke.local.cluster_name_parts_from_nodepool (expand) module.gke.local.cluster_name_parts_from_nodepool (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_master_version (expand)" references: [module.gke.local.cluster_output_master_version (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_pod_security_policy_enabled (expand)" references: [module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_min_master_version (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.output.master_ipv4_cidr_block (expand)" references: [module.gke.var.master_ipv4_cidr_block (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cluster_telemetry_type (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.node_pools_metadata (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.create_service_account (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.name (expand)" references: [var.gke]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_gce_pd_csi_config (expand)" references: [module.gke.var.gce_pd_csi_driver (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/google-beta\"]" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "var.tcpdump_bucket" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_shielded_nodes (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.grant_registry_access (expand)" references: [var.gke]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_zonal_zones (expand)" references: [module.gke.local.zone_count (expand) module.gke.var.zones (expand) module.gke.local.zone_count (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_type (expand)" references: [module.gke.var.regional (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.istio (expand)" references: [var.gke]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cluster_autoscaling (expand)" references: [var.gke var.gke var.gke var.gke var.gke var.gke var.gke]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.non_masquerade_cidrs (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.service_account (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_monitoring_service (expand)" references: [module.gke.local.cluster_output_monitoring_service (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_ca_certificate (expand)" references: [module.gke.local.cluster_master_auth_map (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.var.skip_provisioners (expand)" references: []
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_workload_identity_config (expand)" references: [module.gke.local.workload_identity_enabled (expand) module.gke.var.identity_namespace (expand) module.gke.var.project_id (expand) module.gke.var.identity_namespace (expand)]
2022-12-19T11:18:04.463-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_vertical_pod_autoscaling_enabled (expand)" references: [module.gke.local.cluster_output_vertical_pod_autoscaling_enabled (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_intranode_visibility_enabled (expand)" references: [module.gke.local.cluster_output_intranode_visbility_enabled (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.force_node_pool_recreation_resources (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.filestore_csi_driver (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.kubernetes_config_map.ip-masq-agent (expand)" references: [module.gke.var.configure_ip_masq (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_binary_authorization (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_master_auth_list_layer1 (expand)" references: [module.gke.local.cluster_output_master_auth (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.custom_kube_dns_config (expand)" references: [module.gke.var.stub_domains (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_subnet_cidr (expand)" references: [module.gke.var.add_cluster_firewall_rules (expand) module.gke.data.google_compute_subnetwork.gke_subnetwork (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_master_version (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.google_container_node_pool.pools (expand)" references: [module.gke.local.node_pools (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cluster_dns_provider (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.project_id (expand)" references: [var.project_id]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.master_authorized_networks_config (expand)" references: [module.gke.var.master_authorized_networks (expand) module.gke.var.master_authorized_networks (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_istio_enabled (expand)" references: [module.gke.local.cluster_output_istio_disabled (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.google_project_iam_member.cluster_service_account-gcr (expand)" references: [module.gke.var.create_service_account (expand) module.gke.var.grant_registry_access (expand) module.gke.local.registry_projects_list (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.ip_range_services (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.node_pools_labels (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.output.horizontal_pod_autoscaling_enabled (expand)" references: [module.gke.local.cluster_horizontal_pod_autoscaling_enabled (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.istio_auth (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.data.google_compute_subnetwork.gke_subnetwork (expand)" references: [module.gke.var.add_cluster_firewall_rules (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.master_ipv4_cidr_block (expand)" references: [var.gke]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_http_load_balancing_enabled (expand)" references: [module.gke.local.cluster_output_http_load_balancing_enabled (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "google_service_account.gke (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_cloudrun_enabled (expand)" references: [module.gke.var.cloudrun (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_telemetry_type_is_set (expand)" references: [module.gke.var.cluster_telemetry_type (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.stub_domains (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.logging_service (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.notification_config_topic (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_intranode_visibility (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_node_pools_versions (expand)" references: [module.gke.local.cluster_output_node_pools_versions (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.output.identity_namespace (expand)" references: [module.gke.local.cluster_workload_identity_config (expand) module.gke.local.cluster_workload_identity_config (expand) module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_private_nodes (expand)" references: [var.gke]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.google_project_iam_member.cluster_service_account-resourceMetadata-writer (expand)" references: [module.gke.var.create_service_account (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.config_connector (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_locations (expand)" references: [module.gke.var.regional (expand) module.gke.var.zones (expand) module.gke.random_shuffle.available_zones (expand) module.gke.var.zones (expand) module.gke.var.zones (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_horizontal_pod_autoscaling_enabled (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.google_compute_firewall.shadow_allow_pods (expand)" references: [module.gke.var.add_shadow_firewall_rules (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.node_pools_taints (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.master_authorized_networks (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.firewall_inbound_ports (expand)" references: [var.gke]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.node_pools_tags (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_pools_metadata (expand)" references: [module.gke.var.node_pools (expand) module.gke.var.node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.windows_node_pools (expand) module.gke.var.node_pools_metadata (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_network_policy_enabled (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "var.environment" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.kalm_config (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.zone_count (expand)" references: [module.gke.var.zones (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.default_auto_upgrade (expand)" references: [module.gke.var.regional (expand) module.gke.var.release_channel (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.output.pod_security_policy_enabled (expand)" references: [module.gke.local.cluster_pod_security_policy_enabled (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.google_project_iam_member.cluster_service_account-log_writer (expand)" references: [module.gke.var.create_service_account (expand)]
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.zones (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.var.maintenance_end_time (expand)" references: []
2022-12-19T11:18:04.464-0600 [DEBUG] ReferenceTransformer: "module.gke.local.gke_backup_agent_config (expand)" references: [module.gke.var.gke_backup_agent_config (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.old_node_metadata_config_mapping (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.release_channel (expand)" references: [var.gke]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_intranode_visbility_enabled (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "google_compute_subnetwork.gke_network (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.upstream_nameservers (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.region (expand)" references: [module.gke.var.regional (expand) module.gke.var.region (expand) module.gke.var.zones (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.release_channel (expand)" references: [module.gke.var.release_channel (expand) module.gke.var.release_channel (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.output.node_pools_names (expand)" references: [module.gke.local.cluster_node_pools_names (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.resource_usage_export_dataset_id (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.kubernetes_config_map_v1_data.kube-dns-upstream-nameservers-and-stub-domains (expand)" references: [module.gke.local.custom_kube_dns_config (expand) module.gke.local.upstream_nameservers_config (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.firewall_priority (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.ip_masq_link_local (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_min_master_version (expand)" references: [module.gke.local.cluster_output_min_master_version (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/hashicorp/kubernetes\"]" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.random_shuffle.available_zones (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.shadow_firewall_rules_priority (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_pod_security_policy (expand)" references: [var.gke]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_location (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_authenticator_security_group (expand)" references: [module.gke.var.authenticator_security_group (expand) module.gke.var.authenticator_security_group (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.default_max_pods_per_node (expand)" references: [var.gke]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.kubernetes_version (expand)" references: [var.gke]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.windows_node_pools (expand)" references: [module.gke.local.windows_node_pool_names (expand) module.gke.var.windows_node_pools (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_maintenance_window_is_recurring (expand)" references: [module.gke.var.maintenance_recurrence (expand) module.gke.var.maintenance_end_time (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_identity_service (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.output.master_version (expand)" references: [module.gke.local.cluster_master_version (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.output.service_account (expand)" references: [module.gke.local.service_account (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.maintenance_recurrence (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.ip_masq_resync_interval (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.google_container_cluster.primary (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cloudrun_load_balancer_type (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cluster_dns_scope (expand)" references: []
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cloudrun (expand)" references: [var.gke]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.master_version_zonal (expand)" references: [module.gke.var.kubernetes_version (expand) module.gke.var.kubernetes_version (expand) module.gke.data.google_container_engine_versions.zone (expand)]
2022-12-19T11:18:04.465-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_monitoring_service (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.465-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.465-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.465-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.465-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.465-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.465-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.465-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.key"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.google_container_node_pool.pools[\"standard-2\"]" references: [module.gke.local.node_pools (expand) module.gke.local.default_auto_upgrade (expand) module.gke.google_container_cluster.primary (expand) module.gke.var.project_id (expand) module.gke.random_id.name (expand) module.gke.local.location (expand) module.gke.google_container_cluster.primary (expand) module.gke.local.default_auto_upgrade (expand) module.gke.local.node_pools_tags (expand) module.gke.local.cluster_network_tag (expand) module.gke.local.node_pools_tags (expand) module.gke.local.cluster_network_tag (expand) module.gke.local.node_pools_tags (expand) module.gke.local.node_pools_tags (expand) module.gke.local.node_pools_oauth_scopes (expand) module.gke.local.node_pools_oauth_scopes (expand) module.gke.local.node_pools_labels (expand) module.gke.var.name (expand) module.gke.local.node_pools_labels (expand) module.gke.local.node_pools_labels (expand) module.gke.local.node_pools_labels (expand) module.gke.local.node_pools_metadata (expand) module.gke.var.name (expand) module.gke.local.node_pools_metadata (expand) module.gke.local.node_pools_metadata (expand) module.gke.local.node_pools_metadata (expand) module.gke.var.disable_legacy_metadata_endpoints (expand) module.gke.local.service_account (expand) module.gke.local.node_pools_taints (expand) module.gke.local.node_pools_taints (expand) module.gke.local.cluster_node_metadata_config (expand) module.gke.var.sandbox_enabled (expand) module.gke.local.node_pools_linux_node_configs_sysctls (expand) module.gke.local.node_pools_linux_node_configs_sysctls (expand) module.gke.local.node_pools_linux_node_configs_sysctls (expand) module.gke.local.node_pools_linux_node_configs_sysctls (expand) module.gke.var.timeouts (expand) module.gke.var.timeouts (expand) module.gke.var.timeouts (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_vertical_pod_autoscaling (expand)" references: [var.gke]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.var.add_cluster_firewall_rules (expand)" references: [var.gke]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_name (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.google_project_iam_member.cluster_service_account-artifact-registry (expand)" references: [module.gke.var.create_service_account (expand) module.gke.var.grant_registry_access (expand) module.gke.local.registry_projects_list (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.output.istio_enabled (expand)" references: [module.gke.local.cluster_istio_enabled (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.var.network_project_id (expand)" references: []
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_node_pools_names (expand)" references: [module.gke.google_container_node_pool.pools (expand) module.gke.google_container_node_pool.pools["standard-2"] module.gke.google_container_node_pool.windows_pools (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.var.deploy_using_private_endpoint (expand)" references: []
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.var.master_global_access_enabled (expand)" references: []
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.var.windows_node_pools (expand)" references: []
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_node_metadata_config (expand)" references: [module.gke.var.node_metadata (expand) module.gke.local.old_node_metadata_config_mapping (expand) module.gke.var.node_metadata (expand) module.gke.var.node_metadata (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_network_tag (expand)" references: [module.gke.var.name (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.local.autoscaling_resource_limits (expand)" references: [module.gke.var.cluster_autoscaling (expand) module.gke.var.cluster_autoscaling (expand) module.gke.var.cluster_autoscaling (expand) module.gke.var.cluster_autoscaling (expand) module.gke.var.cluster_autoscaling (expand) module.gke.var.cluster_autoscaling (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "module.gke.output.region (expand)" references: [module.gke.local.cluster_region (expand)]
2022-12-19T11:18:04.466-0600 [DEBUG] ReferenceTransformer: "var.region" references: []
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_logging_service (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_node_pools_names (expand)" references: [module.gke.local.cluster_output_node_pools_names (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.logmon_config_is_set (expand)" references: [module.gke.var.logging_enabled_components (expand) module.gke.var.monitoring_enabled_components (expand) module.gke.var.monitoring_enable_managed_prometheus (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.output.ca_certificate (expand)" references: [module.gke.local.cluster_ca_certificate (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.region (expand)" references: [var.region]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.monitoring_enabled_components (expand)" references: []
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.enable_private_endpoint (expand)" references: [var.gke]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.subnetwork (expand)" references: [google_compute_subnetwork.gke_network (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.remove_default_node_pool (expand)" references: []
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_master_auth_list_layer2 (expand)" references: [module.gke.local.cluster_master_auth_list_layer1 (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_regional_zones (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.random_id.name (expand)" references: [module.gke.local.node_pools (expand) module.gke.local.windows_node_pools (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.database_encryption (expand)" references: []
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.master_version_regional (expand)" references: [module.gke.var.kubernetes_version (expand) module.gke.var.kubernetes_version (expand) module.gke.data.google_container_engine_versions.region (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.confidential_node_config (expand)" references: [module.gke.var.enable_confidential_nodes (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.output.name (expand)" references: [module.gke.local.cluster_name_computed (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_node_pool.pools (expand) module.gke.google_container_node_pool.pools["standard-2"]]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.google_compute_firewall.shadow_allow_nodes (expand)" references: [module.gke.var.add_shadow_firewall_rules (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.disable_default_snat (expand)" references: []
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.network (expand)" references: [var.gke]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_pools (expand)" references: [module.gke.local.node_pool_names (expand) module.gke.var.node_pools (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.output.identity_service_enabled (expand)" references: [module.gke.local.cluster_pod_security_policy_enabled (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cluster_resource_labels (expand)" references: [var.environment var.gke]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_master_auth (expand)" references: [module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_peering_name (expand)" references: [module.gke.var.enable_private_nodes (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.ip_range_pods (expand)" references: []
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "var.project_id" references: []
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.upstream_nameservers_config (expand)" references: [module.gke.var.upstream_nameservers (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_output_vertical_pod_autoscaling_enabled (expand)" references: [module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand) module.gke.google_container_cluster.primary (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.node_pools_linux_node_configs_sysctls (expand)" references: [module.gke.var.node_pools (expand) module.gke.var.node_pools (expand) module.gke.var.node_pools_linux_node_configs_sysctls (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.local.cluster_alias_ranges_cidr (expand)" references: [module.gke.var.add_cluster_firewall_rules (expand) module.gke.data.google_compute_subnetwork.gke_subnetwork (expand)]
2022-12-19T11:18:04.467-0600 [DEBUG] ReferenceTransformer: "module.gke.var.cluster_ipv4_cidr (expand)" references: []
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_kubernetes_alpha (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.description (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_maintenance_window_is_daily (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.initial_node_count (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.local.network_project_id (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.output.master_ipv4_cidr_block (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.horizontal_pod_autoscaling (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_gce_pd_csi_config (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_zonal_zones (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.istio (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_shielded_nodes (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.non_masquerade_cidrs (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.skip_provisioners (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.local.force_node_pool_recreation_resources (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.filestore_csi_driver (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_binary_authorization (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_subnet_cidr (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.kubernetes_config_map.ip-masq-agent (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.local.master_authorized_networks_config (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.google_project_iam_member.cluster_service_account-gcr (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cluster_dns_provider (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.output.horizontal_pod_autoscaling_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.istio_auth (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.var.ip_range_services (expand) is no longer needed, removing
2022-12-19T11:18:04.475-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_telemetry_type_is_set (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.notification_config_topic (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_intranode_visibility (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.identity_namespace (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.logging_service (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.config_connector (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.node_locations (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.google_compute_firewall.shadow_allow_pods (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.google_project_iam_member.cluster_service_account-resourceMetadata-writer (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.master_authorized_networks (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.firewall_inbound_ports (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.zone_count (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.pod_security_policy_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.google_project_iam_member.cluster_service_account-log_writer (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.kalm_config (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.gke_backup_agent_config (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.region (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.release_channel (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.node_pools_names (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.resource_usage_export_dataset_id (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.firewall_priority (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.ip_masq_link_local (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.random_shuffle.available_zones (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.kubernetes_config_map_v1_data.kube-dns-upstream-nameservers-and-stub-domains (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_pod_security_policy (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_authenticator_security_group (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.default_max_pods_per_node (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.shadow_firewall_rules_priority (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_maintenance_window_is_recurring (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_identity_service (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.service_account (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.maintenance_recurrence (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.master_version (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.ip_masq_resync_interval (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cluster_dns_scope (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_vertical_pod_autoscaling (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_name (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.google_project_iam_member.cluster_service_account-artifact-registry (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.network_project_id (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.istio_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.master_global_access_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.autoscaling_resource_limits (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.region (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_node_pools_names (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.logmon_config_is_set (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.ca_certificate (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_private_endpoint (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.subnetwork (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.remove_default_node_pool (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.var.monitoring_enabled_components (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.local.confidential_node_config (expand) is no longer needed, removing
2022-12-19T11:18:04.476-0600 [DEBUG] pruneUnusedNodes: module.gke.output.name (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.google_compute_firewall.shadow_allow_nodes (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.database_encryption (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.network (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.identity_service_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cluster_resource_labels (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.disable_default_snat (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.ip_range_pods (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_alias_ranges_cidr (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cluster_ipv4_cidr (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.location (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: google_storage_bucket_iam_member.gcr-access (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_resource_consumption_export (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.cloudrun_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.google_project_iam_member.cluster_service_account-monitoring_viewer (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.data.google_compute_zones.available (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.google_compute_firewall.tpu_egress (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.kubernetes_config_map_v1_data.kube-dns (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_pod_security_policy_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.monitoring_service (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.registry_projects_list (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_name_computed (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_master_version (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_pod_security_policy_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cluster_telemetry_type (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.maintenance_start_time (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.grant_registry_access (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: google_storage_bucket_iam_member.kube_node_tracer (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.min_master_version (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_ca_certificate (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.master_authorized_networks_config (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cluster_autoscaling (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.registry_project_ids (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_workload_identity_config (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_master_version (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.kubernetes_config_map_v1_data.kube-dns-upstream-namservers (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_identity_service_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_istio_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.logging_enabled_components (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.google_compute_firewall.intra_egress (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.master_version (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.data.google_compute_subnetwork.gke_subnetwork (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.gke_backup_agent_config (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_cloudrun_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_network_egress_export (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.monitoring_service (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.intranode_visibility_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.monitoring_enable_managed_prometheus (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.google_project_iam_member.cluster_service_account-metric_writer (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.google_compute_firewall.master_webhooks (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_endpoint_for_nodes (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_istio_disabled (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.random_string.cluster_service_account_suffix (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.peering_name (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.zones (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.var.maintenance_end_time (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_zones (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: google_compute_subnetwork.gke_network (expand) is no longer needed, removing
2022-12-19T11:18:04.477-0600 [DEBUG] pruneUnusedNodes: module.gke.output.cluster_id (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.network_policy_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_confidential_nodes (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_l4_ilb_subsetting (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_min_master_version (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: provider["registry.terraform.io/hashicorp/kubernetes"] is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_network_policy_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.type (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.node_pools_versions (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_cloudrun_config (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cluster_dns_domain (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.release_channel (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.instance_group_urls (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.gce_pd_csi_driver (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.issue_client_certificate (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.maintenance_exclusions (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.http_load_balancing_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.add_master_webhook_firewall_rules (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_id (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_region (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_cloudrun_config_load_balancer_config (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.endpoint (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_min_master_version (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.workload_identity_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_type (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_monitoring_service (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.logging_service (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.http_load_balancing (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.upstream_nameservers_config (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_intranode_visibility_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.custom_kube_dns_config (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_peering_name (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.datapath_provider (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.configure_ip_masq (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_zones (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.master_ipv4_cidr_block (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_http_load_balancing_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.google_compute_firewall.shadow_allow_master (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.stub_domains (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.authenticator_security_group (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.var.identity_namespace (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_node_pools_versions (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.dns_cache_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.master_version_regional (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_horizontal_pod_autoscaling_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_horizontal_pod_autoscaling_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_regional_zones (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_logging_service (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_name_parts_from_nodepool (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_network_policy_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.output.vertical_pod_autoscaling_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_logging_service (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: google_compute_firewall.stage_gke_ingress (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_node_pools_versions (expand) is no longer needed, removing
2022-12-19T11:18:04.478-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_monitoring_service (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cloudrun_load_balancer_type (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.cloudrun (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.add_shadow_firewall_rules (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.master_version_zonal (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.output.tpu_ipv4_cidr_block (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_http_load_balancing_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.data.google_container_engine_versions.region (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.add_cluster_firewall_rules (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_master_auth_map (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.kubernetes_version (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_vertical_pod_autoscaling_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_dns_cache_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_vertical_pod_autoscaling_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_node_pools_names (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_location (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_tpu (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: google_storage_bucket.kube_node_tracer (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_endpoint (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.data.google_container_engine_versions.zone (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.google_container_node_pool.windows_pools (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.dns_cache (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_intranode_visbility_enabled (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_master_auth_list_layer2 (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_master_auth_list_layer1 (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_output_master_auth (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.local.cluster_network_policy (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.network_policy (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.upstream_nameservers (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.enable_private_nodes (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.deploy_using_private_endpoint (expand) is no longer needed, removing
2022-12-19T11:18:04.479-0600 [DEBUG] pruneUnusedNodes: module.gke.var.network_policy_provider (expand) is no longer needed, removing
2022-12-19T11:18:04.480-0600 [DEBUG] Removing "var.environment", filtered by targeting.
2022-12-19T11:18:04.480-0600 [DEBUG] Removing "module.gke (close)", filtered by targeting.
2022-12-19T11:18:04.480-0600 [DEBUG] Removing "var.tcpdump_bucket", filtered by targeting.
2022-12-19T11:18:04.482-0600 [DEBUG] Starting graph walk: walkApply
2022-12-19T11:18:04.483-0600 [DEBUG] created provider logger: level=debug
2022-12-19T11:18:04.483-0600 [INFO]  provider: configuring client automatic mTLS
2022-12-19T11:18:04.520-0600 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.4.3/darwin_amd64/terraform-provider-random_v3.4.3_x5 args=[.terraform/providers/registry.terraform.io/hashicorp/random/3.4.3/darwin_amd64/terraform-provider-random_v3.4.3_x5]
2022-12-19T11:18:04.529-0600 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.4.3/darwin_amd64/terraform-provider-random_v3.4.3_x5 pid=77875
2022-12-19T11:18:04.529-0600 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.4.3/darwin_amd64/terraform-provider-random_v3.4.3_x5
2022-12-19T11:18:04.539-0600 [INFO]  provider.terraform-provider-random_v3.4.3_x5: configuring server automatic mTLS: timestamp=2022-12-19T11:18:04.539-0600
2022-12-19T11:18:04.560-0600 [DEBUG] provider: using plugin: version=5
2022-12-19T11:18:04.560-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: plugin address: address=/var/folders/17/b_cdl_0s3j324mq5qg4yh4340000gp/T/plugin1424245481 network=unix timestamp=2022-12-19T11:18:04.559-0600
2022-12-19T11:18:04.598-0600 [DEBUG] created provider logger: level=debug
2022-12-19T11:18:04.599-0600 [INFO]  provider: configuring client automatic mTLS
2022-12-19T11:18:04.601-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined Provider GetSchema: tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:211 @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e timestamp=2022-12-19T11:18:04.601-0600
2022-12-19T11:18:04.601-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined Provider GetSchema: @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:213 timestamp=2022-12-19T11:18:04.601-0600
2022-12-19T11:18:04.601-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined Provider GetResources: tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:338 @module=sdk.framework timestamp=2022-12-19T11:18:04.601-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined Provider GetResources: @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:340 timestamp=2022-12-19T11:18:04.601-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined ResourceType GetSchema: @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:293 @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_rpc=GetProviderSchema tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_string timestamp=2022-12-19T11:18:04.601-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined ResourceType GetSchema: @module=sdk.framework tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_string @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:295 tf_provider_addr=registry.terraform.io/hashicorp/random tf_rpc=GetProviderSchema timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined ResourceType GetSchema: tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:293 @module=sdk.framework tf_resource_type=random_uuid tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined ResourceType GetSchema: tf_provider_addr=registry.terraform.io/hashicorp/random tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:295 tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_uuid @module=sdk.framework timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined ResourceType GetSchema: @module=sdk.framework tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_id @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:293 tf_provider_addr=registry.terraform.io/hashicorp/random tf_rpc=GetProviderSchema timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined ResourceType GetSchema: @module=sdk.framework @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:295 tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_id tf_rpc=GetProviderSchema timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined ResourceType GetSchema: tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:293 @module=sdk.framework tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_integer tf_provider_addr=registry.terraform.io/hashicorp/random timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined ResourceType GetSchema: tf_resource_type=random_integer tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:295 tf_provider_addr=registry.terraform.io/hashicorp/random @module=sdk.framework tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined ResourceType GetSchema: tf_provider_addr=registry.terraform.io/hashicorp/random tf_resource_type=random_password tf_rpc=GetProviderSchema tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:293 @module=sdk.framework timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined ResourceType GetSchema: @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:295 @module=sdk.framework tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_password tf_provider_addr=registry.terraform.io/hashicorp/random tf_rpc=GetProviderSchema timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined ResourceType GetSchema: @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:293 tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_resource_type=random_pet tf_rpc=GetProviderSchema timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined ResourceType GetSchema: @module=sdk.framework tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_pet tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:295 tf_provider_addr=registry.terraform.io/hashicorp/random timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined ResourceType GetSchema: tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:293 @module=sdk.framework tf_resource_type=random_shuffle timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined ResourceType GetSchema: @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:295 @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e tf_resource_type=random_shuffle tf_rpc=GetProviderSchema timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined Provider GetDataSources: tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:193 @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.602-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined Provider GetDataSources: tf_rpc=GetProviderSchema @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server.go:195 @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=004a201d-7e06-61a1-1dc1-e38edbce0c5e timestamp=2022-12-19T11:18:04.602-0600
2022-12-19T11:18:04.603-0600 [DEBUG] No provider meta schema returned
2022-12-19T11:18:04.639-0600 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/google-beta/4.46.0/darwin_amd64/terraform-provider-google-beta_v4.46.0_x5 args=[.terraform/providers/registry.terraform.io/hashicorp/google-beta/4.46.0/darwin_amd64/terraform-provider-google-beta_v4.46.0_x5]
2022-12-19T11:18:04.648-0600 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/google-beta/4.46.0/darwin_amd64/terraform-provider-google-beta_v4.46.0_x5 pid=77879
2022-12-19T11:18:04.649-0600 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/google-beta/4.46.0/darwin_amd64/terraform-provider-google-beta_v4.46.0_x5
2022-12-19T11:18:04.668-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: configuring server automatic mTLS: timestamp=2022-12-19T11:18:04.667-0600
2022-12-19T11:18:04.704-0600 [DEBUG] provider: using plugin: version=5
2022-12-19T11:18:04.704-0600 [DEBUG] provider.terraform-provider-google-beta_v4.46.0_x5: plugin address: address=/var/folders/17/b_cdl_0s3j324mq5qg4yh4340000gp/T/plugin3691803732 network=unix timestamp=2022-12-19T11:18:04.704-0600
2022-12-19T11:18:04.742-0600 [DEBUG] created provider logger: level=debug
2022-12-19T11:18:04.742-0600 [INFO]  provider: configuring client automatic mTLS
2022-12-19T11:18:04.782-0600 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/google/3.90.1/darwin_amd64/terraform-provider-google_v3.90.1_x5 args=[.terraform/providers/registry.terraform.io/hashicorp/google/3.90.1/darwin_amd64/terraform-provider-google_v3.90.1_x5]
2022-12-19T11:18:04.791-0600 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/google/3.90.1/darwin_amd64/terraform-provider-google_v3.90.1_x5 pid=77883
2022-12-19T11:18:04.792-0600 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/hashicorp/google/3.90.1/darwin_amd64/terraform-provider-google_v3.90.1_x5
2022-12-19T11:18:04.848-0600 [WARN]  ValidateProviderConfig from "provider[\"registry.terraform.io/hashicorp/google-beta\"]" changed the config value, but that value is unused
2022-12-19T11:18:04.861-0600 [INFO]  provider.terraform-provider-google_v3.90.1_x5: configuring server automatic mTLS: timestamp=2022-12-19T11:18:04.861-0600
2022-12-19T11:18:04.902-0600 [DEBUG] provider.terraform-provider-google_v3.90.1_x5: plugin address: address=/var/folders/17/b_cdl_0s3j324mq5qg4yh4340000gp/T/plugin824729198 network=unix timestamp=2022-12-19T11:18:04.902-0600
2022-12-19T11:18:04.902-0600 [DEBUG] provider: using plugin: version=5
2022-12-19T11:18:04.949-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Calling provider defined Provider Configure: tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=32c620ac-9e49-e10d-5de9-ec1fcd4e4f7d tf_rpc=Configure @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server_configureprovider.go:12 @module=sdk.framework timestamp=2022-12-19T11:18:04.949-0600
2022-12-19T11:18:04.949-0600 [DEBUG] provider.terraform-provider-random_v3.4.3_x5: Called provider defined Provider Configure: @caller=github.com/hashicorp/terraform-plugin-framework@v0.11.1/internal/fwserver/server_configureprovider.go:20 @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/random tf_req_id=32c620ac-9e49-e10d-5de9-ec1fcd4e4f7d tf_rpc=Configure timestamp=2022-12-19T11:18:04.949-0600
2022-12-19T11:18:04.951-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:04 [INFO] Authenticating using DefaultClient...: timestamp=2022-12-19T11:18:04.951-0600
2022-12-19T11:18:04.951-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:04 [INFO]   -- Scopes: [https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/userinfo.email]: timestamp=2022-12-19T11:18:04.951-0600
2022-12-19T11:18:04.951-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:04 [INFO] Authenticating using DefaultClient...: timestamp=2022-12-19T11:18:04.951-0600
2022-12-19T11:18:04.951-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:04 [INFO]   -- Scopes: [https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/userinfo.email]: timestamp=2022-12-19T11:18:04.951-0600
2022-12-19T11:18:04.951-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:04 [DEBUG] Waiting for state to become: [success]: timestamp=2022-12-19T11:18:04.951-0600
2022-12-19T11:18:05.025-0600 [WARN]  ValidateProviderConfig from "provider[\"registry.terraform.io/hashicorp/google\"]" changed the config value, but that value is unused
2022-12-19T11:18:05.027-0600 [INFO]  provider.terraform-provider-google_v3.90.1_x5: 2022/12/19 11:18:05 [INFO] Authenticating using DefaultClient...: timestamp=2022-12-19T11:18:05.027-0600
2022-12-19T11:18:05.027-0600 [INFO]  provider.terraform-provider-google_v3.90.1_x5: 2022/12/19 11:18:05 [INFO]   -- Scopes: [https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/cloud-identity https://www.googleapis.com/auth/ndev.clouddns.readwrite https://www.googleapis.com/auth/devstorage.full_control https://www.googleapis.com/auth/userinfo.email]: timestamp=2022-12-19T11:18:05.027-0600
2022-12-19T11:18:05.028-0600 [INFO]  provider.terraform-provider-google_v3.90.1_x5: 2022/12/19 11:18:05 [INFO] Authenticating using DefaultClient...: timestamp=2022-12-19T11:18:05.028-0600
2022-12-19T11:18:05.028-0600 [INFO]  provider.terraform-provider-google_v3.90.1_x5: 2022/12/19 11:18:05 [INFO]   -- Scopes: [https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/cloud-identity https://www.googleapis.com/auth/ndev.clouddns.readwrite https://www.googleapis.com/auth/devstorage.full_control https://www.googleapis.com/auth/userinfo.email]: timestamp=2022-12-19T11:18:05.028-0600
2022-12-19T11:18:05.028-0600 [INFO]  provider.terraform-provider-google_v3.90.1_x5: 2022/12/19 11:18:05 [DEBUG] Waiting for state to become: [success]: timestamp=2022-12-19T11:18:05.028-0600
2022-12-19T11:18:05.145-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [INFO] Terraform is using this identity: tf-mhuber@<redacted>.iam.gserviceaccount.com: timestamp=2022-12-19T11:18:05.145-0600
2022-12-19T11:18:05.295-0600 [INFO]  provider.terraform-provider-google_v3.90.1_x5: 2022/12/19 11:18:05 [INFO] Terraform is using this identity: tf-mhuber@<redacted>.iam.gserviceaccount.com: timestamp=2022-12-19T11:18:05.295-0600
2022-12-19T11:18:05.299-0600 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2022-12-19T11:18:05.304-0600 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/google/3.90.1/darwin_amd64/terraform-provider-google_v3.90.1_x5 pid=77883
2022-12-19T11:18:05.304-0600 [DEBUG] provider: plugin exited
2022-12-19T11:18:05.305-0600 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2022-12-19T11:18:05.308-0600 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/random/3.4.3/darwin_amd64/terraform-provider-random_v3.4.3_x5 pid=77875
2022-12-19T11:18:05.308-0600 [DEBUG] provider: plugin exited
2022-12-19T11:18:05.335-0600 [WARN]  Provider "registry.terraform.io/hashicorp/google-beta" produced an invalid plan for module.gke.google_container_node_pool.pools["standard-2"], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .upgrade_settings[0].strategy: planned value cty.StringVal("SURGE") for a non-computed attribute
      - .upgrade_settings[0].blue_green_settings: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .node_config[0].min_cpu_platform: planned value cty.UnknownVal(cty.String) does not match config value cty.StringVal("")
      - .node_config[0].logging_variant: planned value cty.StringVal("DEFAULT") for a non-computed attribute
module.gke.google_container_node_pool.pools["standard-2"]: Creating...
2022-12-19T11:18:05.338-0600 [INFO]  Starting apply for module.gke.google_container_node_pool.pools["standard-2"]
2022-12-19T11:18:05.346-0600 [DEBUG] module.gke.google_container_node_pool.pools["standard-2"]: applying the planned Create change
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "node_config.0.guest_accelerator" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "node_config.0.taint" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "node_config.0.guest_accelerator" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "node_config.0.taint" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "node_config.0.taint" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "managed_instance_group_urls" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "upgrade_settings.0.blue_green_settings" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "upgrade_settings.0.blue_green_settings" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "node_locations" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] setting computed for "instance_group_urls" from ComputedKeys: timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.348-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] parent cluster staging does not match regex projects/(?P<project>[^/]+)/locations/(?P<location>[^/]+)/clusters/(?P<name>[^/]+): timestamp=2022-12-19T11:18:05.348-0600
2022-12-19T11:18:05.349-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] RLocking "google-container-cluster/<redacted>/us-east4/staging": timestamp=2022-12-19T11:18:05.349-0600
2022-12-19T11:18:05.349-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] RLocked "google-container-cluster/<redacted>/us-east4/staging": timestamp=2022-12-19T11:18:05.349-0600
2022-12-19T11:18:05.349-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Locking "projects/<redacted>/locations/us-east4/clusters/staging/nodePools/standard-2-17f9": timestamp=2022-12-19T11:18:05.349-0600
2022-12-19T11:18:05.349-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Locked "projects/<redacted>/locations/us-east4/clusters/staging/nodePools/standard-2-17f9": timestamp=2022-12-19T11:18:05.349-0600
2022-12-19T11:18:05.349-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [INFO] Instantiating GKE client for path https://container.googleapis.com/: timestamp=2022-12-19T11:18:05.349-0600
2022-12-19T11:18:05.351-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Retry Transport: starting RoundTrip retry loop: timestamp=2022-12-19T11:18:05.351-0600
2022-12-19T11:18:05.351-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Retry Transport: request attempt 0: timestamp=2022-12-19T11:18:05.351-0600
2022-12-19T11:18:05.351-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
GET /v1beta1/projects/<redacted>/locations/us-east4/clusters/staging/nodePools/standard-2-17f9?alt=json&prettyPrint=false HTTP/1.1
Host: container.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/1.0.0 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google-beta/dev blueprints/terraform/terraform-google-kubernetes-engine:beta-private-cluster-update-variant/v23.3.0
X-Goog-Api-Client: gl-go/1.18.1 gdcl/0.102.0
Accept-Encoding: gzip


-----------------------------------------------------: timestamp=2022-12-19T11:18:05.351-0600
2022-12-19T11:18:05.506-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 404 Not Found
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 19 Dec 2022 17:18:05 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 404,
    "message": "Not found: node pool \"standard-2-17f9\" not found.",
    "errors": [
      {
        "message": "Not found: node pool \"standard-2-17f9\" not found.",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}

-----------------------------------------------------: timestamp=2022-12-19T11:18:05.505-0600
2022-12-19T11:18:05.506-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 404 with body: HTTP/2.0 404 Not Found
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 19 Dec 2022 17:18:05 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 404,
    "message": "Not found: node pool \"standard-2-17f9\" not found.",
    "errors": [
      {
        "message": "Not found: node pool \"standard-2-17f9\" not found.",
        "domain": "global",
        "reason": "notFound"
      }
    ],
    "status": "NOT_FOUND"
  }
}: timestamp=2022-12-19T11:18:05.506-0600
2022-12-19T11:18:05.506-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Retry Transport: Returning after 1 attempts: timestamp=2022-12-19T11:18:05.506-0600
2022-12-19T11:18:05.507-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Waiting for state to become: [success]: timestamp=2022-12-19T11:18:05.507-0600
2022-12-19T11:18:05.509-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [INFO] Instantiating GKE client for path https://container.googleapis.com/: timestamp=2022-12-19T11:18:05.509-0600
2022-12-19T11:18:05.510-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Retry Transport: starting RoundTrip retry loop: timestamp=2022-12-19T11:18:05.510-0600
2022-12-19T11:18:05.510-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Retry Transport: request attempt 0: timestamp=2022-12-19T11:18:05.510-0600
2022-12-19T11:18:05.510-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:05 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
POST /v1beta1/projects/<redacted>/locations/us-east4/clusters/staging/nodePools?alt=json&prettyPrint=false HTTP/1.1
Host: container.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/1.0.0 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google-beta/dev blueprints/terraform/terraform-google-kubernetes-engine:beta-private-cluster-update-variant/v23.3.0
Content-Length: 1292
Content-Type: application/json
X-Goog-Api-Client: gl-go/1.18.1 gdcl/0.102.0
Accept-Encoding: gzip

{
 "nodePool": {
  "autoscaling": {
   "enabled": true,
   "maxNodeCount": 170,
   "minNodeCount": 35,
   "totalMaxNodeCount": 0,
   "totalMinNodeCount": 0
  },
  "config": {
   "diskSizeGb": 200,
   "diskType": "pd-ssd",
   "imageType": "COS_CONTAINERD",
   "labels": {
    "cluster_name": "staging",
    "node_pool": "standard-2"
   },
   "loggingConfig": {
    "variantConfig": {
     "variant": "DEFAULT"
    }
   },
   "machineType": "n2d-highcpu-16",
   "metadata": {
    "cluster_name": "staging",
    "disable-legacy-endpoints": "true",
    "node_pool": "standard-2"
   },
   "oauthScopes": [
    "https://www.googleapis.com/auth/ndev.clouddns.readwrite",
    "https://www.googleapis.com/auth/monitoring",
    "https://www.googleapis.com/auth/devstorage.read_only",
    "https://www.googleapis.com/auth/logging.write",
    "https://www.googleapis.com/auth/cloud-platform",
    "https://www.googleapis.com/auth/compute"
   ],
   "serviceAccount": "<redacted>",
   "shieldedInstanceConfig": {
    "enableIntegrityMonitoring": true
   },
   "tags": [
    "gke-staging",
    "gke-staging-standard-2"
   ],
   "workloadMetadataConfig": {
    "mode": "GKE_METADATA"
   }
  },
  "initialNodeCount": 1,
  "management": {
   "autoRepair": true
  },
  "maxPodsConstraint": {
   "maxPodsPerNode": "64"
  },
  "name": "standard-2-17f9",
  "networkConfig": {
   "enablePrivateNodes": false,
   "podRange": "gke-pods-1"
  },
  "upgradeSettings": {
   "maxSurge": 3,
   "maxUnavailable": 2,
   "strategy": "SURGE"
  },
  "version": "1.23.13-gke.900"
 }
}

-----------------------------------------------------: timestamp=2022-12-19T11:18:05.510-0600
2022-12-19T11:18:06.055-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:06 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 400 Bad Request
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 19 Dec 2022 17:18:06 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 400,
    "message": "EnablePrivateNodes must be enabled for private clusters with valid masterIpv4Cidr.",
    "errors": [
      {
        "message": "EnablePrivateNodes must be enabled for private clusters with valid masterIpv4Cidr.",
        "domain": "global",
        "reason": "badRequest"
      }
    ],
    "status": "INVALID_ARGUMENT"
  }
}

-----------------------------------------------------: timestamp=2022-12-19T11:18:06.051-0600
2022-12-19T11:18:06.055-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:06 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 400 with body: HTTP/2.0 400 Bad Request
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Mon, 19 Dec 2022 17:18:06 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 400,
    "message": "EnablePrivateNodes must be enabled for private clusters with valid masterIpv4Cidr.",
    "errors": [
      {
        "message": "EnablePrivateNodes must be enabled for private clusters with valid masterIpv4Cidr.",
        "domain": "global",
        "reason": "badRequest"
      }
    ],
    "status": "INVALID_ARGUMENT"
  }
}: timestamp=2022-12-19T11:18:06.051-0600
2022-12-19T11:18:06.055-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:06 [DEBUG] Retry Transport: Returning after 1 attempts: timestamp=2022-12-19T11:18:06.052-0600
2022-12-19T11:18:06.055-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:06 [DEBUG] Unlocking "projects/<redacted>/locations/us-east4/clusters/staging/nodePools/standard-2-17f9": timestamp=2022-12-19T11:18:06.052-0600
2022-12-19T11:18:06.055-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:06 [DEBUG] Unlocked "projects/<redacted>/locations/us-east4/clusters/staging/nodePools/standard-2-17f9": timestamp=2022-12-19T11:18:06.053-0600
2022-12-19T11:18:06.055-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:06 [DEBUG] RUnlocking "google-container-cluster/<redacted>/us-east4/staging": timestamp=2022-12-19T11:18:06.053-0600
2022-12-19T11:18:06.055-0600 [INFO]  provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:06 [DEBUG] RUnlocked "google-container-cluster/<redacted>/us-east4/staging": timestamp=2022-12-19T11:18:06.053-0600
2022-12-19T11:18:06.056-0600 [ERROR] provider.terraform-provider-google-beta_v4.46.0_x5: Response contains error diagnostic: diagnostic_detail= diagnostic_severity=ERROR diagnostic_summary="error creating NodePool: googleapi: Error 400: EnablePrivateNodes must be enabled for private clusters with valid masterIpv4Cidr., badRequest" tf_proto_version=5.3 tf_provider_addr=provider tf_resource_type=google_container_node_pool tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto tf_req_id=5297aabb-9c27-e4ce-d836-816775a2c959 timestamp=2022-12-19T11:18:06.056-0600
╷
│ Warning: Applied changes may be incomplete
│
│ The plan was created with the -target option in effect, so some changes requested in the configuration may have been ignored and the output values may not be fully updated. Run the following command to verify that no other changes are pending:
│     terraform plan
│
│ Note that the -target option is not suitable for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use it as part of an error message.
╵
╷
│ Error: error creating NodePool: googleapi: Error 400: EnablePrivateNodes must be enabled for private clusters with valid masterIpv4Cidr., badRequest
│
│   with module.gke.google_container_node_pool.pools["standard-2"],
│   on .terraform/modules/gke/modules/beta-private-cluster-update-variant/cluster.tf line 508, in resource "google_container_node_pool" "pools":
│  508: resource "google_container_node_pool" "pools" {
│
╵
2022-12-19T11:18:06.689-0600 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2022-12-19T11:18:06.689-0600 [DEBUG] provider.terraform-provider-google-beta_v4.46.0_x5: 2022/12/19 11:18:06 [DEBUG] [transport] transport: http2Server.HandleStreams failed to read frame: read unix /var/folders/17/b_cdl_0s3j324mq5qg4yh4340000gp/T/plugin3691803732->: use of closed network connection
2022-12-19T11:18:06.694-0600 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/google-beta/4.46.0/darwin_amd64/terraform-provider-google-beta_v4.46.0_x5 pid=77879
2022-12-19T11:18:06.694-0600 [DEBUG] provider: plugin exited

@splichy
Copy link
Contributor

splichy commented Dec 22, 2022

Hi, in TF google provider v4.45.0 release there were introduced new node_pool network_config attributes, one of them is enable_private_nodes. I have faced the same issue, the solution was to define enable_private_nodes = true on node_pool. A little bit strange is that it happened only on some clusters, I did the same change on multiple almost identical clusters - on some without a problem and some with this error - anyway I see one connection here, I'm using discontiguous pod ranges for the affected node_pool, same as you. (just verified on the second cluster with disco pod ranges)

I can prepare PR to fix this in this module when I have time.

@Shelby-Jiao
Copy link
Author

Hi splichy @splichy , as you can see in pasted template, we've already enabled enable_private_nodes but still the problem occurs...

@splichy
Copy link
Contributor

splichy commented Dec 27, 2022

@Shelby-Jiao it must be enabled also on node pool level, but that's currently missing in module.

@splichy splichy mentioned this issue Dec 30, 2022
Merged
splichy added a commit to splichy/terraform-google-kubernetes-engine that referenced this issue Dec 30, 2022
@splichy
fix: enable private nodes with pod ip range
eadd5b9 
fixes terraform-google-modules#1493
splichy added a commit to splichy/terraform-google-kubernetes-engine that referenced this issue Jan 3, 2023
@splichy
fix: enable private nodes with pod_ip_range
cfc6ea3 
fixes terraform-google-modules#1493
splichy added a commit to splichy/terraform-google-kubernetes-engine that referenced this issue Jan 3, 2023
@splichy
fix: enable private nodes with pod_ip_range
2353322 
fixes terraform-google-modules#1493
@splichy splichy mentioned this issue Jan 3, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
3 participants
@splichy @mjhuber @Shelby-Jiao