Have local google_storage_bucket resource name validation

[rux616]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

When creating buckets, there are some pretty straight forward naming guidelines. However, the provider doesn't seem to do any checks locally that would catch things like capital letters or special characters in the name, instead relying on the Google API to do so. I'd like to request that some of this validation be added so that it's caught in a terraform plan instead of in terraform apply.

Per #3420, I totally understand that adhering strictly to the underlying API may not be feasible, but I think at least having a basic regex validator of something like [a-z0-9][a-z0-9_.-]{1,220}[a-z0-9] wouldn't be outrageous.

New or Affected Resource(s)

• google_storage_bucket

Potential Terraform Configuration

resource "google_storage_bucket" "foo" {
  # `terraform plan` should barf on this, but doesn't
  name = "This-p4$ses-local-VALIdation!"
}

References

• https://cloud.google.com/storage/docs/naming-buckets
• google_storage_bucket should validate fields #3420

[aucampia]

I don't actually understand how it is acceptable to not validate. This wastes a lot of your users time, as the failures now occurs after PRs have been reviewed and merged, so then for every validation error I have to make a new PR, just to find out what is next.

Is there some other solution that is being suggested to do validation? I think issuing warnings would be okay, or maybe calling actual API for validation.

[upodroid]

@rileykarson Dupe of #3580 and #8347

Futher details are in #8347 but plan time validation will never be perfect and end users are expected/required to be reading API documentation to supply valid values to Terraform.

In short Terraform plan-time validation is broken/buggy as explained below:

• Google doesn't have an API for testing values. Failures are detected after a POST API Call is made and this done during the apply step.
• Some resources don't allow certain values to be specified.

# There are 2 illegal configs in this VMs. Both are noted in the Google Docs
resource "google_compute_instance" "default" {
  name         = "test"
  machine_type = "e2-medium"
  zone         = "us-central1-a"

  tags = ["foo", "bar"]

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-10"
    }
  }

  network_interface {
    network = "default"
  }

  scratch_disk {
    interface = "NVME"
  }

  scheduling {
    on_host_maintenance = "TERMINATE"
  }
}

• Some configurations are illegal because of values configured in a resource that it depends on.

# There are 3 illegal configs here. They are documented in the API docs.
resource "google_compute_network" "main" {
  name = "main"
}
resource "google_compute_subnetwork" "two" {
  name          = "two-bar"
  ip_cidr_range = "10.2.0.0/16"
  region        = "us-central1"
  network       = google_compute_network.main.id
}

resource "google_compute_subnetwork" "one" {
  name          = "foo-bar"
  ip_cidr_range = "10.0.0.0/14"
  region        = "us-central3"
  network       = google_compute_network.main.id
}

• IAM Prerequisites for certain resources and IAM permissions of identity running terraform.
• Field values supported by Google change over time. It is double work for the maintainers as the initial validation logic needs to be amended or removed in the future. It also breaks provider versions when the APIs change.
• Validation coverage varies across resources. Some resources have minimal field validations while the older ones have more validation.

Therefore, I advise you not to rely on plan time validation and leverage the API documentation for correct set of values. It will get easier with experience.

[rux616]

Futher details are in #8347 but plan time validation will never be perfect and end users are expected/required to be reading API documentation to supply valid values to Terraform.

That's all well and good, but I would point out that this proposed change is not asking for full plan-time validation. It's asking for a small amount of validation (more akin to sanity-checking actually) using a very simple regex on one of the seemingly more common pain points surrounding buckets.

Therefore, I advise you not to rely on plan time validation and leverage the API documentation for correct set of values. It will get easier with experience.

Advising someone to not rely on plan-time validation is a bit like advising someone to not worry about what the unit tests say. If the unit tests don't work, you fix the unit tests.

[upodroid]

Advising someone to not rely on plan-time validation is a bit like advising someone to not worry about what the unit tests say. If the unit tests don't work, you fix the unit tests.

You are missing the point. For example, how would you even validate the examples I linked above? There isn't a way of validating those right now because of the limitations mentioned above. However, your proposal is valid but IMHO it won't be enough as there will always be resource x not validating field y, which is why I don't recommend relying on plan time validation.

#8437 is for fixing plan time validation and once there are satisfactory solutions we can potentially fix all of these validations problems.

[rileykarson]

Name validation (for syntax, not uniqueness- you'd still fail at apply time if the name was non-unique) is a reasonable thing to add, and 3580/8437 are extremely general and don't really apply. For the provider team it's generally a matter of prioritization, see my reply at #9113 (comment).

[rux616]

You are missing the point. For example, how would you even validate the examples I linked above? There isn't a way of validating those right now because of the limitations mentioned above. However, your proposal is valid but IMHO it won't be enough as there will always be resource x not validating field y, which is why I don't recommend relying on plan time validation.

#8437 is for fixing plan time validation and once there are satisfactory solutions we can potentially fix all of these validations problems.

I do get the point. I completely understand that full validation isn't feasible right now (though like you, I absolutely wish it were). I accept that fact and concur that having the provider dev team spend all their time chasing down each API change and adding specific validation for them all would simply be an exercise in futility. My counterpoint, to continue to use the unit test analogy, was essentially "since full unit tests to catch everything aren't possible at the moment, let's at least get a sanity check in there to catch the most obvious things".

If anything, reliance on plan-time validation (at least for this provider) should be referenced with a caveat emptor clause rather than the blanket "should not be used".

[melinath]

b/237107072

[prateek2408]

Issue has been fixed now

[melinath]

Resolved by GoogleCloudPlatform/magic-modules#6250

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.