Output block causes diffs on terraform plan for map field

[zli82016]

SDK version

github.com/hashicorp/terraform-plugin-sdk/v2 v2.33.0

Relevant provider source code

provider registry.terraform.io/hashicorp/google v6.0.1
provider registry.terraform.io/hashicorp/google v5.42

Terraform Configuration Files

provider "google" {
  region      = "us-central1"
}

resource "google_storage_bucket" "guins_bucket_tf" {
  name     = "guins-bucket-tf"
  location = "US"
}

output "bucket_url" {
  value = google_storage_bucket.guins_bucket_tf.url
}

output "bucket_labels" {
  value = google_storage_bucket.guins_bucket_tf.labels
}

Debug Output

Expected Behavior

terraform apply should show no diff

Actual Behavior

terraform apply shows diff on labels:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan:

  # google_storage_bucket.guins_bucket_tf has changed
  ~ resource "google_storage_bucket" "guins_bucket_tf" {
        id                          = "guins-bucket-tf"
      + labels                      = {}
        name                        = "guins-bucket-tf"
        # (15 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }
(...)
Changes to Outputs:
  + bucket_labels = {}

The output block causes the diffs. Without the output block, no diffs is observed in the second terraform apply.
Other fields with schema.TypeMap have the same issue. I tested with the resource_tags field in google_bigquery_table resource.
Can you help understand the reason that the output block causes the diffs in the second terraform apply? Is it a bug in Terraform core? Thanks.

Steps to Reproduce

1. terraform apply to create the resource
2. terraform apply to get the diffs in the plan

References

[austinvalle]

Hey there @zli82016 👋🏻, thanks for reporting the bug and sorry you're running into trouble here.

What you're describing here sounds like a data consistency issue in the provider + SDKv2. That data discrepancy is only being shown by adding the output, however it's likely always present for all map types with similar data handling.

The problem here is that in this configuration, labels is null, which Terraform core will always interpret as null:

resource "google_storage_bucket" "guins_bucket_tf" {
  name     = "guins-bucket-tf"
  location = "US"
  # labels is not present in the configuration
}

In the provider, it looks like regardless of the config value for labels, it is always being set to an empty map during create:

• https://github.com/hashicorp/terraform-provider-google/blob/5bce89216324fcf9165ef5fc8d1634e55465282b/google/services/storage/resource_storage_bucket.go#L1862-L1864

This by itself would be a data consistency issue (config value needs to be preserved, aka null), however SDKv2 has some unfortunate legacy data handling during apply that attempts to normalizes empty values with null values. One of the reasons this exists is because SDKv2 cannot natively represent null.

So the overall sequence of events that occur:

1. After your first apply, labels is set to an empty map in the provider Create method, SDKv2 normalizes that to null and the result in state after the first apply for labels is null.
2. During your second apply, Terraform refreshes your resource (via the Read method) and it receives an empty map for labels. Terraform core can sometimes ignore these changes for the legacy SDK, but in this case there is an output that relies on the value of labels, so Terraform prompts that a change has occurred, from null to an empty map. At the end of your second apply, the labels state value is set to an empty map.

Unfortunately, this is something that SDKv2 cannot be easily modified to fix as a good chunk of the provider ecosystem rely on this legacy behavior, compounding on that, null values cannot be represented by SDKv2 provider code to correct the problem either 🙁.

The only real end solution is to eventually migrate the resource over to terraform-plugin-framework, which fully supports Terraform's type system and will allow providers to produce null values.

One note, during migration you'll want to cover the cases of previous versions of the resource having empty maps in state, despite their config values being null, which can be preserved from prior state following Terraform's data consistency rule:

Any attribute that was non-null in the configuration must either preserve the exact configuration value or return the corresponding attribute value from the prior state.

---

Another semi-related issue that suffers from the same problem: #1101

[zli82016]

@austinvalle , thank you for the detailed information. I got the reason. Migrating the resource over to terraform-plugin-framework is on the plan.