SKE Cluster: Provider produced inconsistent result after apply

[robertadesso1]

Provider Version

v0.35.0

Issue

The apply of ske cluster resource fails with the following error:

╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.ske_cluster.stackit_ske_cluster.cluster, provider "provider[\"registry.terraform.io/stackitcloud/stackit\"]" produced an unexpected new
│ value: .node_pools[1].taints[0].value: was null, but now cty.StringVal("").

Description

We have two node pools, one of which uses taints. I increased the maximum node count in one node pool. Even though I only changed the node count, the provider seemed to confuse the node pools. The first node pool values suddenly got replaced with the second node pool's values, and the second node pool values suddenly got replaced with the first node pool's values. This results in the same valid, desired state but shows up as "replacing" node pool names, OS versions, etc., in the plan.

Since the planned outcome matched the desired state, I applied the plan. However, the apply fails with the given error.

Im not sure if the replacing of valid node pools is connected to this error. There were no changes to the taint itself. (except that the plan unecessarily replaced taints from nodepool A with B, and B with A as described). The hcl code of the taint did not change.

Node pool configuration:

  node_pools = [
    {
      name           = "reg-amd64"
      machine_type = "g1a.4d"     # 4 CPU, 16 GB RAM, amd64 (x86)
      os_version_min = "3975.2.1"
      minimum        = "1"
      maximum        = "2"
      max_surge      = "1"
      availability_zones = ["eu01-3"]
      volume_size    = 30
      volume_type    = "storage_premium_perf0"
      labels = {}
      taints = []
    },
    {
      name           = "licensed-amd64"
      machine_type = "g1a.8d"     # 8 CPU, 16 GB RAM, amd64 (x86)
      os_version_min = "3975.2.1"
      minimum        = "1"
      maximum        = "1"
      max_surge      = "1"
      availability_zones = ["eu01-3"]
      volume_size    = 30
      volume_type    = "storage_premium_perf0"
      labels = {
        "licensed-node" = true
      }
      taints = [{
        key = "licensed-node"
        effect = "NoSchedule"
      }]
    }
  ]

[robertadesso1]

The bug seems to be less complicated than what I described in my previous comment.

We created a test cluster, first without any taints, this works fine.

  node_pools = [
    {
      name        = "reg-amd64"
      machine_type = "g1a.2d"     # 2 CPU, 8 GB RAM, amd64 (x86)
      os_version_min = "3975.2.1"
      minimum     = "1"
      maximum     = "3"
      max_surge   = "1"
      availability_zones = ["eu01-3"]
      volume_size = 20
      volume_type = "storage_premium_perf0"
      labels = {}
      taints = []
    },
    {
      name        = "reg-arm64"
      machine_type = "g1r.2d"   # 2 CPU, 8 GB RAM arm64
      os_version_min = "3975.2.1"
      minimum     = "1"
      maximum     = "1"
      max_surge   = "1"
      availability_zones = ["eu01-3"]
      volume_size = 20
      volume_type = "storage_premium_perf0"
      labels = {}
      taints = []
    }
]

Then we added a new node pool with similar taints, and the terraform stackit provider fails.

  node_pools = [
    {
      name        = "reg-amd64"
      machine_type = "g1a.2d"     # 2 CPU, 8 GB RAM, amd64 (x86)
      os_version_min = "3975.2.1"
      minimum     = "1"
      maximum     = "3"
      max_surge   = "1"
      availability_zones = ["eu01-3"]
      volume_size = 20
      volume_type = "storage_premium_perf0"
      labels = {}
      taints = []
    },
    {
      name        = "reg-arm64"
      machine_type = "g1r.2d"   # 2 CPU, 8 GB RAM arm64
      os_version_min = "3975.2.1"
      minimum     = "1"
      maximum     = "3"
      max_surge   = "1"
      availability_zones = ["eu01-3"]
      volume_size = 20
      volume_type = "storage_premium_perf0"
      labels = {}
      taints = []
    },
    {
      name        = "destroy-arm64"
      machine_type = "c1.2"
      os_version_min = "3975.2.1"
      minimum     = "1"
      maximum     = "1"
      max_surge   = "1"
      availability_zones = ["eu01-3"]
      volume_size = 20
      volume_type = "storage_premium_perf0"
      labels = {
        "licensed-node" = true
      }
      taints = [{
        key = "licensed-node"
        effect = "NoSchedule"
      }]
    }
  ]

Fails with:

│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.ske_cluster.stackit_ske_cluster.cluster, provider "provider[\"registry.terraform.io/stackitcloud/stackit\"]" produced an
│ unexpected new value: .node_pools[2].taints[0].value: was null, but now cty.StringVal("").
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

[jruhe-adesso]

The issue remains even if the taint has a value assigned (taint "value" is optional).

[joaopalet]

Hi @jruhe-adesso,

Thanks for reporting the issue! We will have a look and get back to you soon.