EdgeCacheService deployment fails with Error "defaultTtl must not be set when cacheMode is USE_ORIGIN_HEADERS or BYPASS_CACHE."

[CDN-guy]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.5.0
on darwin_arm64

Affected Resource(s)

google_network_services_edge_cache_service

Terraform Configuration Files

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "~> 4.61.0"
    }
  }
}

resource "google_network_services_edge_cache_origin" "dummy-origin" {
  name           = "dummy-origin-TF"
  origin_address = "test.storage.googleapis.com"
}

resource "google_network_services_edge_cache_service" "dummy-service"{
  name                  = "dummy-service-TF"

  routing {
    host_rule {
      description  = "host rule description"
      hosts = ["*"]
      path_matcher = "routes"
    }

    path_matcher {
      name = "routes"

      route_rule {
        priority    = 1

        match_rule {
          prefix_match = "/"
        }

        origin = google_network_services_edge_cache_origin.dummy-origin.name

        route_action {
          cdn_policy {
            cache_mode  = "FORCE_CACHE_ALL"
            default_ttl = "3600s"
            # comment the above 2 lines and uncomment the 1 line below to reproduce the issue
            # cache_mode = "BYPASS_CACHE" 
          }
        }
      }
    }
  }
}

Debug Output

https://gist.github.com/CDN-guy/090e2b47848bd8b53b80c1d21b46f56b

Panic Output

Expected Behavior

1. Terraform removes the defaultTTL line when constructing the actual EdgeCacheService configuration that will be submitted to the EdgeCacheService API
2. The EdgeCacheService API accepts the configuration and updates the cache_mode from FORCE_CACHE_ALL to BYPASS_CACHE

Actual Behavior

1. Terraform somehow retains the defaultTTL line when constructing the EdgeCacheService config, even when the defaultTTL line has been explicitly commented/removed
2. EdgeCacheService API receives a configuration that has cache_mode = BYPASS_CACHE and defaultTTL defined, then responds "googleapi: Error 400: code:3 message:"Config validation failed" complaining about "defaultTtl must not be set when cacheMode is USE_ORIGIN_HEADERS or BYPASS_CACHE."

Steps to Reproduce

1. terraform apply
2. After the resources being created, follow the instruction under cdn_policy
3. Edit the cache_mode from FORCE_CACHE_ALL to BYPASS_CACHE
4. Save the files and do terraform apply again

Important Factoids

References

• #0000

b/309603268

[edwardmedia]

Looking into this but receiving 404 error.

b/287444737

POST /v1/projects/myproject/locations/global/edgeCacheOrigins?alt=json&edgeCacheOriginId=issue14903origin  HTTP/1.1
Host: networkservices.googleapis.com
{
"originAddress": "test.storage.googleapis.com"
}

HTTP/1.1 404 Not Found
{
  "error": {
    "status": "NOT_FOUND", 
    "message": "Method not found.", 
    "code": 404
  }
}

[CDN-guy]

@edwardmedia thanks for looking into this.

Note that you need to do this within an mediacdn allowlisted project.

[slevenick]

This may be an unfortunate side effect of using Computed + Optional on the default_ttl field in the Terraform schema.

Basically what I believe is happening is that default_ttl is set to a value during the initial apply, then when you remove default_ttl from your config the value is kept in state because this field is marked as Computed+Optional. When it creates the PATCH request to update the resource it uses the value in state.

If you recreate the resource entirely it should work.

Do you know if there is a zero value that signifies that default_ttl is unset? Sometimes this is a special string like "0" or the empty string "". Can you try those values out and see if that allows the update?

[CDN-guy]

@slevenick I tried both.. none of them worked out - setting default_ttl filed to zero value or 0 does not help removing this line.

For now, the only workaround I found out is to first remove the the particular route rule and then add them back in a different location - i.e. adding them back at the very bottom of all route rules so that TF will treat them as new_addition and not override any existing route rule.

Again this is not ideal and quite complex for customers who simply want to modify the cache_mode.. It would be great if we can explore some options to address the issue.

[melinath]

It looks like defaultTtl can be set to zero or omitted, and those have different meanings. Additionally, it sounds from the API docs like there's a server-side default of 3600s which is probably why the field is default_from_api (Optional + Computed) in the first place. However, using default_from_api means that zero values in the config will be ignored - setting the field to 0 is the same as not setting it at all.

Ideally, this would probably be an Optional only field (not O + C) with a client-side default value of 3600s and send_empty_value: true. I believe that would match the API behavior most closely. However, that would be a breaking change and would need to wait for a major release.

[rileykarson]

BackendService and BackendBucket needed encoders to mitigate this- basically, the default values are fairly complex and the API pretty picky (incl. rejecting explicit zero values)

[esmaeilram]

any update for this breaking change and google api not found error?
plan is ok but I got this error when I apply. my google provider version is 5.5.0

google_network_services_edge_cache_origin.default: Creating... ╷ │ Error: Error creating EdgeCacheOrigin: googleapi: Error 404: Method not found. │

[melinath]

If this can be handled with encoders I believe it should be possible to fix without a breaking change.

[joelsdc]

any update for this breaking change and google api not found error? plan is ok but I got this error when I apply. my google provider version is 5.5.0

google_network_services_edge_cache_origin.default: Creating... ╷ │ Error: Error creating EdgeCacheOrigin: googleapi: Error 404: Method not found. │

@esmaeilram: I deployed correctly a google_network_services_edge_cache_origin + google_network_services_edge_cache_service on v5.4.0 a few weeks ago, now I'm trying to create the same resources on another project and I'm getting the same error as you are. I also landed on this issue while searching for an answer.

I think that whatever the problem is with google_network_services_edge_cache_service, it's also affecting the google_network_services_edge_cache_origin resource.

Reproduced on version:

terraform v1.6.2
google-provider v5.7.0 (also tried v5.6.0, v5.5.0 and v5.4.0).

[okcompute]

I have a similar error. I cannot create an Edge service origin. This configuration (from the doc) always fails for me:

resource "google_network_services_edge_cache_origin" "default" {
  name                 = "my-origin"
  origin_address       = "gs://media-edge-default"
  description          = "The default bucket for media edge test"
}

With this error:

Error: Error creating EdgeCacheOrigin: googleapi: Error 404: Method not found.
with google_network_services_edge_cache_origin.default
on media-cdn.tf line 14, in resource "google_network_services_edge_cache_origin" "default":
resource "google_network_services_edge_cache_origin" "default" {

I tried with 5.4.0 and 5.7.2 providers. I'm just trying to create the origin. I don't even have the service yet in my Terraform configuration file.

I don't know how to resolve this. Any ideas?

Thank you!

[c2thorn]

removing 6.0.0 assignment since it does not have to wait for a major release according to #14903 (comment)

[arobles-d]

I have the same error. I can't create the "google_network_services_edge_cache_origin" or the "google_network_services_edge_cache_service".

This is my file for the origin:

resource "google_network_services_edge_cache_origin" "test-origin" {
name = "test-origin"
origin_address = "gs://media-edge-default"
description = "The default bucket for media edge test"
project = "project-test"
}

Fails with this error:

google_network_services_edge_cache_origin.test-origin: Creating...

│ Error: Error creating EdgeCacheOrigin: googleapi: Error 404: Method not found.
│
│ with google_network_services_edge_cache_origin.test-origin,
│ on main.tf line 14, in resource "google_network_services_edge_cache_origin" "test-origin":
│ 14: resource "google_network_services_edge_cache_origin" "test-origin" {
│

I even used the Google Cloud Shell option that the Terraform Provider has. https://registry.terraform.io/providers/hashicorp/google/6.6.0/docs/resources/network_services_edge_cache_origin

I tried with 6.6.0, 6.2.0 and some other older providers, and tf configs seems correct. Any ideas on how to solve this?

Thanks in advance!

[CDN-guy]

bumping up this issue.. I am seeing more and more folks running into the challenge

• essentially the Computed + Optional in the Terraform schema is causing the field unremovable.

is there a way to tell terraform to completely remove this field using a special value? for example, can we do something like, if default_ttl = Null, then this field is deem to be removed?

[wyardley]

Note that you need to do this within an mediacdn allowlisted project.

I'd missed this comment earlier, but I guess this explains the issue.