hd_insight_kafka cluster always gets recreated #7067

yhekma · 2020-05-25T11:41:19Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform version: v0.12.24 (via TFE)
Azurerm version: 2.11.0

Affected Resource(s)

azurerm_hdinsight_kafka_cluster

Terraform Configuration Files

resource "azurerm_storage_account" "tnt_kafka" {
  name                     = module.labels.id_for_az_storage_account
  resource_group_name      = module.aks.resource_group.name
  location                 = var.location
  account_tier             = "Standard"
  account_replication_type = "GRS"

  tags = module.labels.tags
}

resource "azurerm_storage_container" "this" {
  name                  = "test-kafka-storage"
  storage_account_name  = azurerm_storage_account.tnt_kafka.name
  container_access_type = "private"
}


resource "azurerm_hdinsight_kafka_cluster" "this" {
  name                = "my-test-kafka-cluster"
  resource_group_name = module.aks.resource_group.name
  location            = "northeurope"
  cluster_version     = "4.0.1000.0"
  tier                = "Standard"
  tags                = module.labels.tags

  component_version {
    kafka = "1.0"
  }

  gateway {
    enabled  = false
    username = random_string.gateway_username.result
    password = random_password.gateway_password.result
  }

  storage_account {
    storage_container_id = azurerm_storage_container.this.id
    storage_account_key  = azurerm_storage_account.tnt_kafka.primary_access_key
    is_default           = true
  }

  roles {
    head_node {
      vm_size            = "Standard_D3_V2"
      username           = random_string.kafka_username.result
      password           = random_password.kafka_password.result
      virtual_network_id = local.vnet.id
      subnet_id          = local.vnet_subnet_id
    }

    worker_node {
      vm_size                  = "Standard_D3_V2"
      username                 = random_string.kafka_username.result
      password                 = random_password.kafka_password.result
      number_of_disks_per_node = 1
      min_instance_count       = 3
      target_instance_count    = 3
      virtual_network_id       = local.vnet.id
      subnet_id                = local.vnet_subnet_id
    }

    zookeeper_node {
      vm_size            = "Standard_D3_V2"
      username           = random_string.zookeeper_username.result
      password           = random_password.zookeeper_password.result
      virtual_network_id = local.vnet.id
      subnet_id          = local.vnet_subnet_id
    }
  }
}

Initial output during creation (only relevant bits)

# azurerm_hdinsight_kafka_cluster.this will be created
  + resource "azurerm_hdinsight_kafka_cluster" "this" {
      + cluster_version     = "4.0.1000.0"
      + https_endpoint      = (known after apply)
      + id                  = (known after apply)
      + location            = "northeurope"
      + name                = "my-test-kafka-cluster"
      + resource_group_name = "<snip>"
      + ssh_endpoint        = (known after apply)
      + tags                = {
          + "ClusterName" = "neur-generic"
          + "Environment" = "nonprod"
          + "created-by"  = "terraform"
          + "owner"       = "<snip>"
        }
      + tier                = "Standard"

      + component_version {
          + kafka = "1.0"
        }

      + gateway {
          + enabled  = false
          + password = (sensitive value)
          + username = (known after apply)
        }

      + roles {
          + head_node {
              + password           = (sensitive value)
              + subnet_id          = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/<snip>"
              + username           = (known after apply)
              + virtual_network_id = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
              + vm_size            = "Standard_D3_V2"
            }

          + worker_node {
              + min_instance_count       = 3
              + number_of_disks_per_node = 1
              + password                 = (sensitive value)
              + subnet_id                = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/<snip>"
              + target_instance_count    = 3
              + username                 = (known after apply)
              + virtual_network_id       = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
              + vm_size                  = "Standard_D3_V2"
            }

          + zookeeper_node {
              + password           = (sensitive value)
              + subnet_id          = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/<snip>"
              + username           = (known after apply)
              + virtual_network_id = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
              + vm_size            = "Standard_D3_V2"
            }
        }

      + storage_account {
          + is_default           = true
          + storage_account_key  = (sensitive value)
          + storage_container_id = (known after apply)
        }
    }

  # azurerm_storage_account.tnt_kafka will be created
  + resource "azurerm_storage_account" "tnt_kafka" {
      + access_tier                      = (known after apply)
      + account_kind                     = "StorageV2"
      + account_replication_type         = "GRS"
      + account_tier                     = "Standard"
      + enable_https_traffic_only        = true
      + id                               = (known after apply)
      + is_hns_enabled                   = false
      + location                         = "northeurope"
      + name                             = "st371eb4d3af4c632837c0c8"
      + primary_access_key               = (sensitive value)
      + primary_blob_connection_string   = (sensitive value)
      + primary_blob_endpoint            = (known after apply)
      + primary_blob_host                = (known after apply)
      + primary_connection_string        = (sensitive value)
      + primary_dfs_endpoint             = (known after apply)
      + primary_dfs_host                 = (known after apply)
      + primary_file_endpoint            = (known after apply)
      + primary_file_host                = (known after apply)
      + primary_location                 = (known after apply)
      + primary_queue_endpoint           = (known after apply)
      + primary_queue_host               = (known after apply)
      + primary_table_endpoint           = (known after apply)
      + primary_table_host               = (known after apply)
      + primary_web_endpoint             = (known after apply)
      + primary_web_host                 = (known after apply)
      + resource_group_name              = "<snip>"
      + secondary_access_key             = (sensitive value)
      + secondary_blob_connection_string = (sensitive value)
      + secondary_blob_endpoint          = (known after apply)
      + secondary_blob_host              = (known after apply)
      + secondary_connection_string      = (sensitive value)
      + secondary_dfs_endpoint           = (known after apply)
      + secondary_dfs_host               = (known after apply)
      + secondary_file_endpoint          = (known after apply)
      + secondary_file_host              = (known after apply)
      + secondary_location               = (known after apply)
      + secondary_queue_endpoint         = (known after apply)
      + secondary_queue_host             = (known after apply)
      + secondary_table_endpoint         = (known after apply)
      + secondary_table_host             = (known after apply)
      + secondary_web_endpoint           = (known after apply)
      + secondary_web_host               = (known after apply)
      + tags                             = {
          + "ClusterName" = "neur-generic"
          + "Environment" = "nonprod"
          + "created-by"  = "terraform"
          + "owner"       = "<snip>"
        }

      + blob_properties {
          + cors_rule {
              + allowed_headers    = (known after apply)
              + allowed_methods    = (known after apply)
              + allowed_origins    = (known after apply)
              + exposed_headers    = (known after apply)
              + max_age_in_seconds = (known after apply)
            }

          + delete_retention_policy {
              + days = (known after apply)
            }
        }

      + identity {
          + principal_id = (known after apply)
          + tenant_id    = (known after apply)
          + type         = (known after apply)
        }

      + network_rules {
          + bypass                     = (known after apply)
          + default_action             = (known after apply)
          + ip_rules                   = (known after apply)
          + virtual_network_subnet_ids = (known after apply)
        }

      + queue_properties {
          + cors_rule {
              + allowed_headers    = (known after apply)
              + allowed_methods    = (known after apply)
              + allowed_origins    = (known after apply)
              + exposed_headers    = (known after apply)
              + max_age_in_seconds = (known after apply)
            }

          + hour_metrics {
              + enabled               = (known after apply)
              + include_apis          = (known after apply)
              + retention_policy_days = (known after apply)
              + version               = (known after apply)
            }

          + logging {
              + delete                = (known after apply)
              + read                  = (known after apply)
              + retention_policy_days = (known after apply)
              + version               = (known after apply)
              + write                 = (known after apply)
            }

          + minute_metrics {
              + enabled               = (known after apply)
              + include_apis          = (known after apply)
              + retention_policy_days = (known after apply)
              + version               = (known after apply)
            }
        }
    }

  # azurerm_storage_container.this will be created
  + resource "azurerm_storage_container" "this" {
      + container_access_type   = "private"
      + has_immutability_policy = (known after apply)
      + has_legal_hold          = (known after apply)
      + id                      = (known after apply)
      + metadata                = (known after apply)
      + name                    = "test-kafka-storage"
      + resource_manager_id     = (known after apply)
      + storage_account_name    = "st371eb4d3af4c632837c0c8"
    }

Output of replan (no code changed)

  # azurerm_hdinsight_kafka_cluster.this must be replaced
-/+ resource "azurerm_hdinsight_kafka_cluster" "this" {
        cluster_version     = "4.0.1000.0"
      ~ https_endpoint      = "my-test-kafka-cluster.azurehdinsight.net" -> (known after apply)
      ~ id                  = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.HDInsight/clusters/my-test-kafka-cluster" -> (known after apply)
        location            = "northeurope"
        name                = "my-test-kafka-cluster"
        resource_group_name = "<snip>"
      ~ ssh_endpoint        = "my-test-kafka-cluster-ssh.azurehdinsight.net" -> (known after apply)
        tags                = {
            "ClusterName" = "neur-generic"
            "Environment" = "nonprod"
            "created-by"  = "terraform"
            "owner"       = "<snip>"
        }
      ~ tier                = "standard" -> "Standard"

        component_version {
            kafka = "1.0"
        }

      ~ gateway {
          ~ enabled  = true -> false # forces replacement
          ~ password = (sensitive value)
            username = "raet3vn3irt5"
        }

      ~ roles {
          ~ head_node {
                password           = (sensitive value)
              - ssh_keys           = [] -> null
                subnet_id          = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/<snip>"
                username           = "9rwac3708dvg"
                virtual_network_id = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
                vm_size            = "Standard_D3_V2"
            }

          ~ worker_node {
              ~ min_instance_count       = 0 -> 3 # forces replacement
                number_of_disks_per_node = 1
                password                 = (sensitive value)
              - ssh_keys                 = [] -> null
                subnet_id                = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/<snip>"
                target_instance_count    = 3
                username                 = "9rwac3708dvg"
                virtual_network_id       = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
                vm_size                  = "Standard_D3_V2"
            }

          ~ zookeeper_node {
                password           = (sensitive value)
              - ssh_keys           = [] -> null
                subnet_id          = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/<snip>"
                username           = "9sa13sol0ldj"
                virtual_network_id = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
                vm_size            = "Standard_D3_V2"
            }
        }

        storage_account {
            is_default           = true
            storage_account_key  = (sensitive value)
            storage_container_id = "https://st371eb4d3af4c632837c0c8.blob.core.windows.net/test-kafka-storage"
        }
    }

Note that setting ssh_keys = null has no effect

Expected Behavior

Nothing would change

Actual Behavior

CLuster will be rerolled

Steps to Reproduce

terraform apply
terraform plan

Important Factoids

This seems to be the same behaviour witnessed in #4485
However, that incident is closed and also I am not convinced this is due to the password field as suggested in that issue. Furthermore it is suggested the issue would be resolved as of provider version 1.35 while I am running newer than that

The text was updated successfully, but these errors were encountered:

kosinsky · 2020-05-26T00:49:12Z

I think it's caused by gateway settings:

  gateway {
    enabled  = false
    username = random_string.gateway_username.result
    password = random_password.gateway_password.result
  }

In the plan I can see:

      ~ gateway {
          ~ enabled  = true -> false # forces replacement
          ~ password = (sensitive value)
            username = "raet3vn3irt5"
        }

It means that gateway wasn't really disabled by default and apply tries to recreated cluster. Current azurerm implementation doesn't support changing gateway settings. However, it should be possible with the most recent GoSDK: https://godoc.org/github.com/Azure/azure-sdk-for-go/services/preview/hdinsight/mgmt/2018-06-01-preview/hdinsight#ClustersClient.UpdateGatewaySettings

I was able to reproduce that by replacing true to false in following test: https://github.com/terraform-providers/terraform-provider-azurerm/blob/194c86e6ecf6eb0a5aeede9ca7cf4be71c1cbb8b/azurerm/internal/services/hdinsight/tests/hdinsight_hadoop_cluster_resource_test.go#L454

I'll try to figure out why gateway isn't configured properly.

kosinsky · 2020-05-26T03:11:05Z

Attempt to disable Gateway using API fails with "Linux clusters do not support revoking HTTP credentials." error. It's possible that gateway must be always enabled and enabled should be always be equal to true

kosinsky · 2020-05-27T23:55:37Z

Update to avoid issues like this: #7111

yhekma · 2020-05-29T11:08:03Z

Even when I set it to create the gateway, the result is the same:

initial plan that I then apply:

  # azurerm_hdinsight_kafka_cluster.this will be created
  + resource "azurerm_hdinsight_kafka_cluster" "this" {
      + cluster_version     = "4.0.1000.0"
      + https_endpoint      = (known after apply)
      + id                  = (known after apply)
      + location            = "northeurope"
      + name                = "my-test-kafka-cluster"
      + resource_group_name = "di-nonprod"
      + ssh_endpoint        = (known after apply)
      + tags                = {
          + "ClusterName" = "di"
          + "Environment" = "nonprod"
          + "created-by"  = "terraform"
          + "owner"       = "<snip>"
        }
      + tier                = "Standard"

      + component_version {
          + kafka = "1.0"
        }

      + gateway {
          + enabled  = true
          + password = (sensitive value)
          + username = (known after apply)
        }

      + roles {
          + head_node {
              + password           = (sensitive value)
              + subnet_id          = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/aks-di"
              + username           = (known after apply)
              + virtual_network_id = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
              + vm_size            = "Standard_D3_V2"
            }

          + worker_node {
              + min_instance_count       = 3
              + number_of_disks_per_node = 1
              + password                 = (sensitive value)
              + subnet_id                = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/aks-di"
              + target_instance_count    = 3
              + username                 = (known after apply)
              + virtual_network_id       = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
              + vm_size                  = "Standard_D3_V2"
            }

          + zookeeper_node {
              + password           = (sensitive value)
              + subnet_id          = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/aks-di"
              + username           = (known after apply)
              + virtual_network_id = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
              + vm_size            = "Standard_D3_V2"
            }
        }

      + storage_account {
          + is_default           = true
          + storage_account_key  = (sensitive value)
          + storage_container_id = (known after apply)
        }
    }

  # azurerm_storage_account.tnt_kafka will be created
  + resource "azurerm_storage_account" "tnt_kafka" {
      + access_tier                      = (known after apply)
      + account_kind                     = "StorageV2"
      + account_replication_type         = "GRS"
      + account_tier                     = "Standard"
      + enable_https_traffic_only        = true
      + id                               = (known after apply)
      + is_hns_enabled                   = false
      + location                         = "northeurope"
      + name                             = "std9e625770f860d2663b1b8"
      + primary_access_key               = (sensitive value)
      + primary_blob_connection_string   = (sensitive value)
      + primary_blob_endpoint            = (known after apply)
      + primary_blob_host                = (known after apply)
      + primary_connection_string        = (sensitive value)
      + primary_dfs_endpoint             = (known after apply)
      + primary_dfs_host                 = (known after apply)
      + primary_file_endpoint            = (known after apply)
      + primary_file_host                = (known after apply)
      + primary_location                 = (known after apply)
      + primary_queue_endpoint           = (known after apply)
      + primary_queue_host               = (known after apply)
      + primary_table_endpoint           = (known after apply)
      + primary_table_host               = (known after apply)
      + primary_web_endpoint             = (known after apply)
      + primary_web_host                 = (known after apply)
      + resource_group_name              = "di-nonprod"
      + secondary_access_key             = (sensitive value)
      + secondary_blob_connection_string = (sensitive value)
      + secondary_blob_endpoint          = (known after apply)
      + secondary_blob_host              = (known after apply)
      + secondary_connection_string      = (sensitive value)
      + secondary_dfs_endpoint           = (known after apply)
      + secondary_dfs_host               = (known after apply)
      + secondary_file_endpoint          = (known after apply)
      + secondary_file_host              = (known after apply)
      + secondary_location               = (known after apply)
      + secondary_queue_endpoint         = (known after apply)
      + secondary_queue_host             = (known after apply)
      + secondary_table_endpoint         = (known after apply)
      + secondary_table_host             = (known after apply)
      + secondary_web_endpoint           = (known after apply)
      + secondary_web_host               = (known after apply)
      + tags                             = {
          + "ClusterName" = "di"
          + "Environment" = "nonprod"
          + "created-by"  = "terraform"
          + "owner"       = "<snip>"
        }

      + blob_properties {
          + cors_rule {
              + allowed_headers    = (known after apply)
              + allowed_methods    = (known after apply)
              + allowed_origins    = (known after apply)
              + exposed_headers    = (known after apply)
              + max_age_in_seconds = (known after apply)
            }

          + delete_retention_policy {
              + days = (known after apply)
            }
        }

      + identity {
          + principal_id = (known after apply)
          + tenant_id    = (known after apply)
          + type         = (known after apply)
        }

      + network_rules {
          + bypass                     = (known after apply)
          + default_action             = (known after apply)
          + ip_rules                   = (known after apply)
          + virtual_network_subnet_ids = (known after apply)
        }

      + queue_properties {
          + cors_rule {
              + allowed_headers    = (known after apply)
              + allowed_methods    = (known after apply)
              + allowed_origins    = (known after apply)
              + exposed_headers    = (known after apply)
              + max_age_in_seconds = (known after apply)
            }

          + hour_metrics {
              + enabled               = (known after apply)
              + include_apis          = (known after apply)
              + retention_policy_days = (known after apply)
              + version               = (known after apply)
            }

          + logging {
              + delete                = (known after apply)
              + read                  = (known after apply)
              + retention_policy_days = (known after apply)
              + version               = (known after apply)
              + write                 = (known after apply)
            }

          + minute_metrics {
              + enabled               = (known after apply)
              + include_apis          = (known after apply)
              + retention_policy_days = (known after apply)
              + version               = (known after apply)
            }
        }
    }

  # azurerm_storage_container.this will be created
  + resource "azurerm_storage_container" "this" {
      + container_access_type   = "private"
      + has_immutability_policy = (known after apply)
      + has_legal_hold          = (known after apply)
      + id                      = (known after apply)
      + metadata                = (known after apply)
      + name                    = "test-kafka-storage"
      + resource_manager_id     = (known after apply)
      + storage_account_name    = "std9e625770f860d2663b1b8"
    }

  # random_password.gateway_password will be created
  + resource "random_password" "gateway_password" {
      + id          = (known after apply)
      + length      = 18
      + lower       = true
      + min_lower   = 1
      + min_numeric = 1
      + min_special = 1
      + min_upper   = 1
      + number      = true
      + result      = (sensitive value)
      + special     = false
      + upper       = true
    }

Then, without changing anything the plan output is:

Terraform will perform the following actions:

  # azurerm_hdinsight_kafka_cluster.this must be replaced
-/+ resource "azurerm_hdinsight_kafka_cluster" "this" {
        cluster_version     = "4.0.1000.0"
      ~ https_endpoint      = "my-test-kafka-cluster.azurehdinsight.net" -> (known after apply)
      ~ id                  = "/subscriptions/<snip>/resourceGroups/di-nonprod/providers/Microsoft.HDInsight/clusters/my-test-kafka-cluster" -> (known after apply)
        location            = "northeurope"
        name                = "my-test-kafka-cluster"
        resource_group_name = "di-nonprod"
      ~ ssh_endpoint        = "my-test-kafka-cluster-ssh.azurehdinsight.net" -> (known after apply)
        tags                = {
            "ClusterName" = "di"
            "Environment" = "nonprod"
            "created-by"  = "terraform"
            "owner"       = "<snip>"
        }
      ~ tier                = "standard" -> "Standard"

        component_version {
            kafka = "1.0"
        }

      ~ gateway {
            enabled  = true
          ~ password = (sensitive value)
            username = "l0drn4oqdvp7"
        }

      ~ roles {
          ~ head_node {
                password           = (sensitive value)
              - ssh_keys           = [] -> null
                subnet_id          = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/aks-di"
                username           = "95q2cbpruto3"
                virtual_network_id = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
                vm_size            = "Standard_D3_V2"
            }

          ~ worker_node {
              ~ min_instance_count       = 0 -> 3 # forces replacement
                number_of_disks_per_node = 1
                password                 = (sensitive value)
              - ssh_keys                 = [] -> null
                subnet_id                = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/aks-di"
                target_instance_count    = 3
                username                 = "95q2cbpruto3"
                virtual_network_id       = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
                vm_size                  = "Standard_D3_V2"
            }

          ~ zookeeper_node {
                password           = (sensitive value)
              - ssh_keys           = [] -> null
                subnet_id          = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>/subnets/aks-di"
                username           = "lziisv28avek"
                virtual_network_id = "/subscriptions/<snip>/resourceGroups/<snip>/providers/Microsoft.Network/virtualNetworks/<snip>"
                vm_size            = "Standard_D3_V2"
            }
        }

        storage_account {
            is_default           = true
            storage_account_key  = (sensitive value)
            storage_container_id = "https://std9e625770f860d2663b1b8.blob.core.windows.net/test-kafka-storage"
        }
    }

Note that the tier, ssh_keys and password are flagged for update as well as min_instance_count for some reason

yhekma · 2020-06-02T11:49:01Z

Ok, so it seems that when I ignore changes to min_instance_count it doesn't reroll it. I was fairly certain I already tried that (since it's right there in the output) but apparantly missed that.
That still leaves this as a bug though.

magodo · 2020-06-09T06:14:04Z

@yhekma You're right, the min_instance_count seems has no direct effect to the HDInsight cluster if you are choosing to manual scale. I will further confirm and hopefully submit a PR to address that.
For now, I think you can just leaving that min_instance_count absent and setting the enable of gateway to true, then you should have no diff issue.

yhekma · 2020-06-09T07:28:42Z

@magodo This is indeed a valid workaround, but it makes it so that I cannot extend clusters which is not ideal of course.

magodo · 2020-06-10T02:41:38Z

@yhekma Can you just modify the target_instance_count to extend the cluster?

yhekma · 2020-06-15T06:43:00Z

@magodo sorry, you are correct, I can.
That makes the impact of this bug indeed a lot lower, but still something that should be fixed of course

This address part of #7067. min_instance_count has no effect during resource creation for hdinsight cluster resource set. Besides, it actually cause plan skew if user specified it to a non-zero value.

…ing cluster to be disabled (#7111) Fixes issues like #7067 HDInisght doesn't support disabling Gateway/Ambari for Linux clusters and only Linux clusters supported. Making gateway.enabled property always equal true. Also using API call to update Gateway settings (changing password)

magodo · 2020-06-29T23:47:04Z

@yhekma The two PRs are merged now and should be available in the next release. Hence, I'm going to close this issue for now.

ghost · 2020-07-03T01:59:05Z

This has been released in version 2.17.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.17.0"
}
# ... other configuration ...

ghost · 2020-07-30T14:50:00Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

tombuildsstuff added question service/hdinsight labels May 25, 2020

kosinsky mentioned this issue May 27, 2020

azurerm_hdinsight_*_cluster - fix broken updaten due to api not allowing cluster to be disabled #7111

Merged

magodo self-assigned this Jun 9, 2020

magodo mentioned this issue Jun 10, 2020

azurerm_hdinsight_cluster deprecate min_instance_count #7272

Merged

magodo added the waiting-response label Jun 10, 2020

ghost removed the waiting-response label Jun 15, 2020

katbyte added this to the v2.17.0 milestone Jun 29, 2020

magodo closed this as completed Jun 29, 2020

ghost locked and limited conversation to collaborators Jul 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hd_insight_kafka cluster always gets recreated #7067

hd_insight_kafka cluster always gets recreated #7067

yhekma commented May 25, 2020 •

edited

Loading

kosinsky commented May 26, 2020

kosinsky commented May 26, 2020

kosinsky commented May 27, 2020

yhekma commented May 29, 2020 •

edited

Loading

yhekma commented Jun 2, 2020

magodo commented Jun 9, 2020

yhekma commented Jun 9, 2020

magodo commented Jun 10, 2020

yhekma commented Jun 15, 2020

magodo commented Jun 29, 2020

ghost commented Jul 3, 2020

ghost commented Jul 30, 2020

hd_insight_kafka cluster always gets recreated #7067

hd_insight_kafka cluster always gets recreated #7067

Comments

yhekma commented May 25, 2020 • edited Loading

Community Note

Terraform (and AzureRM Provider) Version

Affected Resource(s)

Terraform Configuration Files

Initial output during creation (only relevant bits)

Output of replan (no code changed)

Expected Behavior

Actual Behavior

Steps to Reproduce

Important Factoids

kosinsky commented May 26, 2020

kosinsky commented May 26, 2020

kosinsky commented May 27, 2020

yhekma commented May 29, 2020 • edited Loading

yhekma commented Jun 2, 2020

magodo commented Jun 9, 2020

yhekma commented Jun 9, 2020

magodo commented Jun 10, 2020

yhekma commented Jun 15, 2020

magodo commented Jun 29, 2020

ghost commented Jul 3, 2020

ghost commented Jul 30, 2020

yhekma commented May 25, 2020 •

edited

Loading

yhekma commented May 29, 2020 •

edited

Loading