Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provider/aws: Succeed creating aws_volume_attachment if identical attachment exists #11060

Merged

Conversation

dougneal
Copy link
Contributor

@dougneal dougneal commented Jan 5, 2017

If an aws_volume_attachment is identical to one that already exists in the API, don't attempt to re-create it (which fails), simply act as though the creation command had already been run and continue.

This allows Terraform to cleanly recover from a situation where a volume attachment action hangs indefinitely, possibly due to a bad instance state, requiring manual intervention such as an instance reboot. In such a situation, Terraform believes the attachment has failed, when in fact
it succeeded after the timeout had expired. On the subsequent retry run, attempting to re-create the attachment will fail outright, due to the AttachVolume API call being non-idempotent. This patch implements the idempotency client-side by matching the (name, vID, iID) tuple.

Note that volume attachments are not assigned an ID by the API.

Addresses #8458

If an `aws_volume_attachment` is identical to one that already exists in
the API, don't attempt to re-create it (which fails), simply act as
though the creation command had already been run and continue.

This allows Terraform to cleanly recover from a situation where a volume
attachment action hangs indefinitely, possibly due to a bad instance
state, requiring manual intervention such as an instance reboot. In such
a situation, Terraform believes the attachment has failed, when in fact
it succeeded after the timeout had expired. On the subsequent retry run,
attempting to re-create the attachment will fail outright, due to the
AttachVolume API call being non-idempotent. This patch implements the
idempotency client-side by matching the (name, vID, iID) tuple.

Note that volume attachments are not assigned an ID by the API.
@dougneal dougneal changed the title Succeed creating aws_volume_attachment if identical attachment exists provider/aws: Succeed creating aws_volume_attachment if identical attachment exists Jan 9, 2017
@stack72
Copy link
Contributor

stack72 commented Feb 1, 2017

Hi @dougneal

Thanks so much for this - this actually solves another issue as well that I had just started looking at :)

3 instances, each with a volume attached. When i tainted an instance and ran terraform apply, I got this

Error applying plan:

2 error(s) occurred:

* aws_volume_attachment.data_att.0: [WARN] Error attaching volume (vol-02002007a24d64d42) to instance (i-0c78f99768cd4c07d), message: "vol-02002007a24d64d42 is already attached to an instance", code: "VolumeInUse"
* aws_volume_attachment.data_att.2: [WARN] Error attaching volume (vol-0864a6ca6bf0a3d30) to instance (i-011de5dbfdaae5cf5), message: "vol-0864a6ca6bf0a3d30 is already attached to an instance", code: "VolumeInUse"

With this code fix, I was able to successfully get it running:

% terraform plan                                                                                                                             ✭
[WARN] /Users/stacko/Code/go/bin/terraform-provider-aws overrides an internal plugin for aws-provider.
  If you did not expect to see this message you will need to remove the old plugin.
  See https://www.terraform.io/docs/internals/internal-plugins.html
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

aws_key_pair.demo: Refreshing state... (ID: demo)
data.aws_availability_zones.available: Refreshing state...
aws_ebs_volume.data_volumn.1: Refreshing state... (ID: vol-0e69f9a6310637faf)
aws_instance.main.1: Refreshing state... (ID: i-0dd385053e8f4d8c3)
aws_instance.main.0: Refreshing state... (ID: i-0c78f99768cd4c07d)
aws_ebs_volume.data_volumn.2: Refreshing state... (ID: vol-0864a6ca6bf0a3d30)
aws_instance.main.2: Refreshing state... (ID: i-011de5dbfdaae5cf5)
aws_ebs_volume.data_volumn.0: Refreshing state... (ID: vol-02002007a24d64d42)
aws_volume_attachment.data_att.1: Refreshing state... (ID: vai-392533943)
aws_volume_attachment.data_att.0: Refreshing state... (ID: vai-1418711167)
aws_volume_attachment.data_att.2: Refreshing state... (ID: vai-522740285)
No changes. Infrastructure is up-to-date.

This means that Terraform could not detect any differences between your
configuration and real physical resources that exist. As a result, Terraform
doesn't need to do anything.
```% terraform taint aws_instance.main.1                                                                                                        ✭
[WARN] /Users/stacko/Code/go/bin/terraform-provider-aws overrides an internal plugin for aws-provider.
  If you did not expect to see this message you will need to remove the old plugin.
  See https://www.terraform.io/docs/internals/internal-plugins.html
The resource aws_instance.main.1 in the module root has been marked as tainted!```
% terraform apply                                                     
aws_volume_attachment.data_att.2: Destroying...
aws_volume_attachment.data_att.0: Destroying...
aws_volume_attachment.data_att.1: Destroying...
aws_volume_attachment.data_att.2: Destruction complete
aws_volume_attachment.data_att.0: Destruction complete
aws_volume_attachment.data_att.1: Destruction complete
aws_instance.main.1: Destroying...
aws_instance.main.1: Still destroying... (10s elapsed)
aws_instance.main.1: Still destroying... (20s elapsed)
aws_instance.main.1: Still destroying... (30s elapsed)
aws_instance.main.1: Destruction complete
aws_instance.main.1: Creating...
  ami:                                       "" => "ami-6edd3078"
  associate_public_ip_address:               "" => "<computed>"
  availability_zone:                         "" => "us-east-1a"
  ebs_block_device.#:                        "" => "<computed>"
  ephemeral_block_device.#:                  "" => "<computed>"
  instance_state:                            "" => "<computed>"
  instance_type:                             "" => "m3.medium"
  key_name:                                  "" => "demo"
  network_interface_id:                      "" => "<computed>"
  placement_group:                           "" => "<computed>"
  private_dns:                               "" => "<computed>"
  private_ip:                                "" => "<computed>"
  public_dns:                                "" => "<computed>"
  public_ip:                                 "" => "<computed>"
  root_block_device.#:                       "" => "1"
  root_block_device.0.delete_on_termination: "" => "true"
  root_block_device.0.iops:                  "" => "<computed>"
  root_block_device.0.volume_size:           "" => "8"
  root_block_device.0.volume_type:           "" => "standard"
  security_groups.#:                         "" => "<computed>"
  source_dest_check:                         "" => "true"
  subnet_id:                                 "" => "<computed>"
  tags.%:                                    "" => "1"
  tags.Name:                                 "" => "1"
  tenancy:                                   "" => "<computed>"
  vpc_security_group_ids.#:                  "" => "<computed>"
aws_instance.main.1: Still creating... (8s elapsed)
aws_instance.main.1: Still creating... (10s elapsed)
aws_instance.main.1: Still creating... (18s elapsed)
aws_instance.main.1: Still creating... (20s elapsed)
aws_instance.main.1: Creation complete
aws_volume_attachment.data_att.1: Creating...
  device_name:  "" => "/dev/sdh"
  force_detach: "" => "<computed>"
  instance_id:  "" => "i-058931cad1bf4e518"
  skip_destroy: "" => "true"
  volume_id:    "" => "vol-0e69f9a6310637faf"
aws_volume_attachment.data_att.0: Creating...
  device_name:  "" => "/dev/sdh"
  force_detach: "" => "<computed>"
  instance_id:  "" => "i-0c78f99768cd4c07d"
  skip_destroy: "" => "true"
  volume_id:    "" => "vol-02002007a24d64d42"
aws_volume_attachment.data_att.2: Creating...
  device_name:  "" => "/dev/sdh"
  force_detach: "" => "<computed>"
  instance_id:  "" => "i-011de5dbfdaae5cf5"
  skip_destroy: "" => "true"
  volume_id:    "" => "vol-0864a6ca6bf0a3d30"
aws_volume_attachment.data_att.1: Still creating... (10s elapsed)
aws_volume_attachment.data_att.2: Still creating... (10s elapsed)
aws_volume_attachment.data_att.0: Still creating... (10s elapsed)
aws_volume_attachment.data_att.2: Creation complete
aws_volume_attachment.data_att.1: Creation complete
aws_volume_attachment.data_att.0: Creation complete

Apply complete! Resources: 4 added, 0 changed, 4 destroyed.

State path: terraform.tfstate

@stack72 stack72 merged commit babc522 into hashicorp:master Feb 1, 2017
stack72 pushed a commit that referenced this pull request Feb 1, 2017
…#11060)

If an `aws_volume_attachment` is identical to one that already exists in
the API, don't attempt to re-create it (which fails), simply act as
though the creation command had already been run and continue.

This allows Terraform to cleanly recover from a situation where a volume
attachment action hangs indefinitely, possibly due to a bad instance
state, requiring manual intervention such as an instance reboot. In such
a situation, Terraform believes the attachment has failed, when in fact
it succeeded after the timeout had expired. On the subsequent retry run,
attempting to re-create the attachment will fail outright, due to the
AttachVolume API call being non-idempotent. This patch implements the
idempotency client-side by matching the (name, vID, iID) tuple.

Note that volume attachments are not assigned an ID by the API.
@dougneal
Copy link
Contributor Author

dougneal commented Feb 1, 2017

Nice :)

arcadiatea pushed a commit to ticketmaster/terraform that referenced this pull request Feb 9, 2017
…hashicorp#11060)

If an `aws_volume_attachment` is identical to one that already exists in
the API, don't attempt to re-create it (which fails), simply act as
though the creation command had already been run and continue.

This allows Terraform to cleanly recover from a situation where a volume
attachment action hangs indefinitely, possibly due to a bad instance
state, requiring manual intervention such as an instance reboot. In such
a situation, Terraform believes the attachment has failed, when in fact
it succeeded after the timeout had expired. On the subsequent retry run,
attempting to re-create the attachment will fail outright, due to the
AttachVolume API call being non-idempotent. This patch implements the
idempotency client-side by matching the (name, vID, iID) tuple.

Note that volume attachments are not assigned an ID by the API.
danielelisi added a commit to danielelisi/terraform-provider-aws that referenced this pull request Apr 4, 2019
I used to edit my `tfstate` file manually to import `aws_volume_attachment`. Then I found hashicorp/terraform#11060 which changed Terraform behaviour when an EBS volume is already attached to an instance.

I thought would be a good idea to mention it in the docs.
@ghost
Copy link

ghost commented Apr 17, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants