Provider crash invalid memory address

[Klaitos]

Terraform Version

Terraform v1.8.1
But also happening in terraform 1.7.4

Affected Fastly Terraform Resource(s)

It seems to be fastly_tls_subscription but I use several resources (compute and service_vcl)

Get "https://api.fastly.com/service/[redacted]/details": remote error: tls: no application protocol

Terraform Configuration Files

# Copy-paste your Terraform configuration here.

Expected Behavior

No crash

Actual Behavior

Stack trace from the terraform-provider-fastly_v5.8.0 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x58 pc=0x100d958e8]

goroutine 53 [running]:
github.com/fastly/terraform-provider-fastly/fastly.resourceFastlyTLSSubscriptionRead({0x1010e4cf0?, 0x1400056ce10?}, 0x14000528480, {0x100f54080?, 0x14000134c68})
        github.com/fastly/terraform-provider-fastly/fastly/resource_fastly_tls_subscription.go:219 +0x3f8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x140003ffc00, {0x1010e4cf0, 0x1400056ce10}, 0xd?, {0x100f54080, 0x14000134c68})
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/resource.go:795 +0xe8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0x140003ffc00, {0x1010e4cf0, 0x1400056ce10}, 0x1400071a270, {0x100f54080, 0x14000134c68})
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/resource.go:1089 +0x494
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0x1400049a048, {0x1010e4cf0?, 0x1400056ccf0?}, 0x1400046f100)
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/grpc_provider.go:649 +0x400
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0x1400025f680, {0x1010e4cf0?, 0x1400056c330?}, 0x14000430660)
        github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/tf5server/server.go:789 +0x3e8
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x1010a04a0?, 0x1400025f680}, {0x1010e4cf0, 0x1400056c330}, 0x14000529000, 0x0)
        github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:431 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0x140001d41e0, {0x1010e4cf0, 0x1400056c2a0}, {0x1010e97f8, 0x140003f1d40}, 0x140005638c0, 0x14000488810, 0x101642da8, 0x0)
        google.golang.org/grpc@v1.59.0/server.go:1343 +0xc6c
google.golang.org/grpc.(*Server).handleStream(0x140001d41e0, {0x1010e97f8, 0x140003f1d40}, 0x140005638c0)
        google.golang.org/grpc@v1.59.0/server.go:1737 +0x9f0
google.golang.org/grpc.(*Server).serveStreams.func1.1()
        google.golang.org/grpc@v1.59.0/server.go:986 +0x88
created by google.golang.org/grpc.(*Server).serveStreams.func1
        google.golang.org/grpc@v1.59.0/server.go:997 +0x174

Error: The terraform-provider-fastly_v5.8.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

[Integralist]

👋🏻

Thanks for opening this issue. I've implemented a quick "fix" here:
#839

Essentially, the error is caused by the Fastly TF provider trying to reference an ID field on a pointer to a TLSDomain object but the pointer is nil and so dereferencing nil triggers a runtime panic.

The problem I have is that it's unclear to me why the Fastly API would return a TLS Subscription with "relation,tls_domains" where the list of TLS Domains contained a nil value.

Would it be possible to see an example of your TF configuration? I'd ideally like to run this past the Fastly TLS team to better understand the environment in which this runtime panic occurred.

Thanks.

[Klaitos]

Thanks for the quick fix !

I'm not sure this is related but yesterday there was an incident outage on Fastly api returning 500 errors. When they fixed it, this bug suddenly appeared.

I will see if I can share something for you

[Klaitos]

I can confirm that the issue occurs when this ressource is in my state :

resource "fastly_tls_subscription" "domain" {
  domains               = ['domain']
  certificate_authority = "lets-encrypt"
}

is this related to https://www.fastlystatus.com/incident/376455 ?

[bryangm]

👋 Hello! I work on the Fastly TLS team.

Regarding the Let’s Encrypt’s Chain of Trust changes that you referenced, the change will not occur until June 6, 2024...so that is unlikely to be related.

We'd be happy to investigate further. Would you be able to open a support ticket with details on the specific subscription and domains experiencing this issue?

[Klaitos]

Tickets 758522 created. Thanks !

[jonnangle]

We're also seeing this error since the incident:

Get "https://api.fastly.com/service/[redacted]/details": remote error: tls: no application protocol

Though in our case it doesn't seem to be related to having a fastly_tls_subscription resource - the error appears to be coming back in response to the initial TLS Client Hello handshake with api.fastly.com, so we don't get any further than that.

[jonnangle]

Removing force_http2 = true from the provider config seems to be a workaround for us.

[Klaitos]

Hello, I was off for few days and it seems all my issues are resolved somehow. (the provider panic and the one @jonnangle mentioned).

[Integralist]

Thanks for the update @Klaitos 👍🏻

I believe there was an issue with the Fastly API that was underpinning the errors materialising within the Terraform provider. I'll check in with the API team to get a confirmation that all should be resolved and then I'll probably close off this issue.

[Integralist]

👋🏻 @Klaitos

Just wanted to circle back to confirm that the API issue was resolved end of last week.

If you have any other questions regarding the incident then please reach out to https://support.fastly.com/

Thanks!