AWS Route53 record fails to apply when instance is stopped / shutdown

[bmalynovytch]

Dear all,

I'm facing a recurrent issue.
When applying configuration with some AWS instances that are shutdown / stopped, with Route53 records attached, Terraform tries to apply a new (empty) value to the aws_route53_record which fails:

~ aws_route53_record.myhost
    records.0:         "" => ""
    records.200629859: "1.2.3.4" => ""

* aws_route53_record.myhost: InvalidChangeBatch: Invalid Resource Record: FATAL problem: ARRDATAIllegalIPv4Address (Value is not a valid IPv4 address) encountered with ''
    status code: 400, request id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx

Terraform Version

$ terraform -v
Terraform v0.6.16

Affected Resource(s)

• aws_route53_record

Expected Behavior

Do not apply a new value if it does not match the validation pattern.

Actual Behavior

Request fails, trying to apply an invalid (empty) value.

Steps to Reproduce

1. define a resource of type aws_instance
2. define a resource of type aws_route53_record of type A, using the previously created instance's gathered IP address (aws_instance.myhost.public_ip)
3. terraform apply
4. Instance is created, DNS record is created with the corresponding IP address
5. Stop the instance using AWS Console
6. terraform apply
7. Apply fails, trying to change/update the attached IP address

References

This issue is related to comment #1579 (comment)

[aramkarapetian]

I had the same issue, instance was not getting public ip as I have not requested.
You need to have in there:

resource "aws_instance" "resource_name" {
    ....
    associate_public_ip_address = true
    ....
}

[bmalynovytch]

You're right, and this is the case is my setup.
The trouble comes in when the instances are stopped, the public IP is released, therefore, Terraform tries to set an empty value to the DNS record, instead of leaving it alone.

[CpuID]

Confirmed here also.

[stack72]

Hi @bmalynovytch

I am not sure this is a terraform bug. Terraform is reacting to the fact that the IP address has changed. Why would you expect terraform to keep the IP address the same? When the instance comes back up, it will have a different IP and Terraform would need to change it then anyway

Paul

[bmalynovytch]

Well, the fact is, that Terraform is failing to update the DNS, which is quite normal, as the host is down.
Why ignoring the DNS update ? Because, "empty" isn't a valid value, and that updating a DNS entry for a down host is useless.
If you could suggest something better for this specific case (updating a resource to an invalid value, due to a dependency state), I'm fine, but the failure isn't acceptable, as I use TF with a team, with a locking system that doesn't release the lock if a failure occurred.

[farvour]

I am also experiencing this problem. I definitely feel this is a bug. Terraform should not ever try to set route53 records to blank values, and just assume the record should be destroyed. It can be argued that a stopped instance is probably in some transient state so removing the record might not be in my best interest, however, I would rather have that behavior at the very minimum over a complete failure of the run.

[roderickrandolph]

This issue reared its head in v0.8.2 as a result of this bug #10908 in case others experience a similar error.

[farvour]

Is this issue going to get any visibility any time soon? We are up to 0.9.7 and still seeing this. I would expect that the route53 resource really should never attempt setting a blank value on an A record???

Thanks! :)

@stack72 I disagree with the comment above that this isn't a Terraform bug. The whole purpose of terraform's early plan operation is to detect this kind of problems before the apply stage is even executed. Terraform should know better than it can't set a blank IPv4 entry and do one of two things:

1.) Error during plan, which would require us to use some kind of hack to ensure a stopped instance doesn't try to set blank variables.
2.) Utilize a null_resource to notify the route53 record based on similar criteria in item 1.

I would think any time terraform comes back with a "surprise" from the API that it did something that should not have occurred, it tells me that there is improvement that could be made in terraform's plan stage so that it can't even end up in that state. Wouldn't allowing this error, regardless of who caused the bad input, continue to come back from the API instead of detection during plan go against the entire concept of terraform?

I think there needs to be a smarter solution here. @bmalynovytch and I can't be the only ones setting Route53 A records based off of the public_ip of an ec2_instance resource...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.