min_master_version and release_channel conflicting

[glennmcallister]

I want to use the release_channel feature in 6.1.1, specifically in the beta-private-cluster module, however, skimming the code I'm pretty sure this applies to any of the beta_* modules. In this specific case, I'm using the release_channel and specifying a region with 3 zones, resulting in a master_version (as of last Thursday) of

master_version              = "1.14.8-gke.12"

The cluster deployed just fine. However now when I trying to apply an unrelated change, I'm seeing

        master_version              = "1.14.8-gke.12"
      ~ min_master_version          = "1.14.8-gke.17" -> "1.15.4-gke.22"

Trying to apply the change to the cluster results in the following error:

Error: googleapi: Error 400: Current master version cannot be updated when releaseChannel REGULAR is set., badRequest

I initially tried to use ignore_changes in my module invocation to ignore min_master_version, and immediately ran into the fact this can't currently be done.

[morgante]

Can you share the Terraform config you used?

[glennmcallister]

The relevant bits, slightly edited:

variable "region" {
  description = "The region to host the cluster in"
  default     = "northamerica-northeast1"
}

variable "zones" {
  type        = list(string)
  description = "The zones to host the cluster in (optional if regional cluster / required if zonal)"
  default     = ["northamerica-northeast1-a", "northamerica-northeast1-b", "northamerica-northeast1-c"]
}

variable "release_channel" {
  type        = string
  description = "(Beta) The release channel of this cluster. Accepted values are `UNSPECIFIED`, `RAPID`, `REGULAR` and `STABLE`. Defaults to `UNSPECIFIED`."
  default     = "REGULAR"
}

module "gke" {
  source                           = "terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster"
  version                          = "6.1.1"
  project_id                       = var.project_id
  name                             = var.cluster_name
  region                           = var.region
  zones                            = var.zones
  network                          = data.google_compute_network.default-network.name
  subnetwork                       = google_compute_subnetwork.gke-subnet.name
  ip_range_pods                    = google_compute_subnetwork.gke-subnet.secondary_ip_range[0].range_name
  ip_range_services                = google_compute_subnetwork.gke-subnet.secondary_ip_range[1].range_name
  http_load_balancing              = true
  horizontal_pod_autoscaling       = true
  network_policy                   = true
  enable_private_endpoint          = false
  enable_private_nodes             = true
  enable_shielded_nodes            = true
  master_ipv4_cidr_block           = var.master_ipv4_cidr_block
  grant_registry_access            = var.registry_project_id != "" ? true : false
  logging_service                  = "logging.googleapis.com/kubernetes"
  monitoring_service               = "monitoring.googleapis.com/kubernetes"
  registry_project_id              = var.registry_project_id
  remove_default_node_pool         = true
  release_channel                  = var.release_channel
  istio                            = false
  resource_usage_export_dataset_id = var.resource_usage_export_dataset_id


  # See https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control
  authenticator_security_group = "gke-security-groups@example.com"

  database_encryption = [
    {
      state    = "ENCRYPTED"
      key_name = data.google_kms_crypto_key.gke-crypto-key.self_link
    }
  ]

  master_authorized_networks = concat(
    [
      # We always want to include the region subnet where the cluster is deployed
      {
        cidr_block   = data.google_compute_subnetwork.region-subnet.ip_cidr_range
        display_name = "Region subnet"
      }
    ],
    var.master_authorized_networks
  )

  node_pools = [for np in var.node_pools : {
    name               = np.name
    machine_type       = np.machine_type
    min_count          = np.min_count
    max_count          = np.max_count
    disk_size_gb       = np.disk_size_gb
    disk_type          = np.disk_type
    image_type         = "COS"
    auto_repair        = true
    auto_upgrade       = true
    preemptible        = false
    initial_node_count = 1
  }]


  node_pools_oauth_scopes = merge(
    { all = [] },

    zipmap(
      [for np in var.node_pools : np.name],
      [for np in var.node_pools : ["https://www.googleapis.com/auth/cloud-platform"]]
    )
  )

  node_pools_labels = merge(
    { all = {} },

    zipmap(
      [for np in var.node_pools : np.name],
      [for np in var.node_pools : np.labels]
    )
  )

  node_pools_metadata = merge(
    { all = {} },

    zipmap(
      [for np in var.node_pools : np.name],
      [for np in var.node_pools : np.metadata]
    )
  )

  node_pools_tags = merge(
    { all = [] },

    zipmap(
      [for np in var.node_pools : np.name],
      [for np in var.node_pools : np.tags]
    )
  )
}

If you need everything, I can pull it together and scrub it.

[morgante]

Thanks, I think I see the issue.

If release_channel is active on the module, we should set min_master_version to null here.

[glennmcallister]

As another data point, since this is a development cluster that I'm experimenting with I'm constantly tearing it down, and building it back again and I hit the same problem:

Error 400: Master version 1.15.4-gke.22 must be set to the default REGULAR releaseChannel version 1.14.8-gke.12., badRequest

[naseemkullah]

Thanks, I think I see the issue.

If release_channel is active on the module, we should set min_master_version to null here.

Also experiencing this issue... How do we set it to null? I've tried setting kubenetes_version to null but get Error: Malformed version:

[benoitm76]

Same for me.

I think that Google made a change recently on that part because it was working few days ago.

[marshallford]

I'm facing the same issue. Version 6.2.0 of the module.

Error: googleapi: Error 400: Master version 1.15.4-gke.22 must be set to the default RAPID releaseChannel version 1.16.0-gke.20., badRequest