Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v2.63 and above amending ASG tags when not changed #13635

Closed
andydix opened this issue Jun 5, 2020 · 14 comments · Fixed by #13912
Closed

v2.63 and above amending ASG tags when not changed #13635

andydix opened this issue Jun 5, 2020 · 14 comments · Fixed by #13912
Labels
bug Addresses a defect in current functionality. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. service/autoscaling Issues and PRs that pertain to the autoscaling service.
Milestone

Comments

@andydix
Copy link

andydix commented Jun 5, 2020

I've just noticed that when using any provider above v2.62 a plan is reporting changes to auto scaling group tags when no changes are necessary.

v2.62 reports "No changes. Infrastructure is up-to-date."

Update the provider and make no other TF changes

v2.63 reports...
~ aws_autoscaling_group.build_release_agents
tags.0.key: "Environment" => "Name"
tags.0.propagate_at_launch: "true" => "1"
tags.0.value: "build/release" => "cd-ec2-build_release_agent"
tags.1.key: "Name" => "Environment"
tags.1.propagate_at_launch: "true" => "1"
tags.1.value: "cd-ec2-build_release_agent" => "build/release"
tags.2.propagate_at_launch: "true" => "1"

Both v2.64 and v2.65 report...
~ aws_autoscaling_group.build_release_agents
tags.1177997400.%: "0" => "3"
tags.1177997400.key: "" => "Project"
tags.1177997400.propagate_at_launch: "" => "1"
tags.1177997400.value: "" => "Azure DevOps Continuous Delivery"
tags.1427379085.%: "0" => "3"
tags.1427379085.key: "" => "Name"
tags.1427379085.propagate_at_launch: "" => "1"
tags.1427379085.value: "" => "cd-ec2-build_release_agent"
tags.2875147149.%: "3" => "0"
tags.2875147149.key: "Environment" => ""
tags.2875147149.propagate_at_launch: "true" => ""
tags.2875147149.value: "build/release" => ""
tags.3059725845.%: "3" => "0"
tags.3059725845.key: "Project" => ""
tags.3059725845.propagate_at_launch: "true" => ""
tags.3059725845.value: "Azure DevOps Continuous Delivery" => ""
tags.4203851964.%: "3" => "0"
tags.4203851964.key: "Name" => ""
tags.4203851964.propagate_at_launch: "true" => ""
tags.4203851964.value: "cd-ec2-build_release_agent" => ""
tags.877694087.%: "0" => "3"
tags.877694087.key: "" => "Environment"
tags.877694087.propagate_at_launch: "" => "1"
tags.877694087.value: "" => "build/release"

If I apply changes using 2.63/2.64/2.65 they apply ok but the next plan still reports the same changes are required.

Anyone found similar?

@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Jun 5, 2020
@ewbankkit
Copy link
Contributor

Related:

@ewbankkit
Copy link
Contributor

@andydix Thanks for raising this issue.
Which version of Terraform are you using?

@andydix
Copy link
Author

andydix commented Jun 5, 2020

I'm still using 0.11.14 at the moment.

@ewbankkit ewbankkit added the service/autoscaling Issues and PRs that pertain to the autoscaling service. label Jun 5, 2020
@ewbankkit
Copy link
Contributor

@andydix Is your configuration something like:

  tags = [
    {
      key                 = "Project"
      value               = "Azure DevOps Continuous Delivery"
      propagate_at_launch = 1
    },
    {
      key                 = "Name"
      value               = "cd-ec2-build_release_agent"
      propagate_at_launch = 1
    },
    {
      key                 = "Environment"
      value               = "build/release"
      propagate_at_launch = 1
    },
  ]

with propagate_at_launch a number or numeric string (e.g. "1") rather than a boolean (true)?

@andydix
Copy link
Author

andydix commented Jun 5, 2020

I'm actually using a concat/list/map combo like this...

  tags = [
    "${concat(
      list(
        map("key", "Name", "value", "${var.project_tag_acronym}-ec2-build_release_agent", "propagate_at_launch", true)
      ),
      local.asg_common_tags)}",
  ]

@andydix
Copy link
Author

andydix commented Jun 5, 2020

Oh and the "common" is defined as this...

  asg_common_tags = [
    {
      key                 = "Environment"
      value               = "build/release"
      propagate_at_launch = true
    },
    {
      key                 = "Project"
      value               = "Azure DevOps Continuous Delivery"
      propagate_at_launch = true
    },
  ]

@bflad
Copy link
Contributor

bflad commented Jun 5, 2020

This may be due to some of the quirky pre-Terraform 0.12 behavior with types, does it still show differences if you quote true in any configuration going into the tags argument ("true")?

@ewbankkit
Copy link
Contributor

Verified that changing true to "true" results in no diff.

resource "aws_autoscaling_group" "test" {
  tags = [
    "${concat(
      list(
        map("key", "Name", "value", "cd-ec2-build_release_agent", "propagate_at_launch", "true")
      ),
      local.asg_common_tags)}",
  ]
}

locals {
  asg_common_tags = [
    {
      key                 = "Environment"
      value               = "build/release"
      propagate_at_launch = "true"
    },
    {
      key                 = "Project"
      value               = "Azure DevOps Continuous Delivery"
      propagate_at_launch = "true"
    },
  ]
}

@muthu-vgh
Copy link

any movement on this issue @bflad @ewbankkit @andydix ?

@andydix
Copy link
Author

andydix commented Jun 23, 2020

Not that I know of.

I've also just noticed something similar happening for security groups on an RDS instance...

vpc_security_group_ids.#:          "1" => "2"
vpc_security_group_ids.0:          "" => ""
vpc_security_group_ids.3547069620: "sg-081da9a6ec834b4b0" => "sg-081da9a6ec834b4b0"

The group hasn't changed in the config and looks like the provider is evaluating a change to ids where none is required.

And this is using v2.67.0 for reference.

bflad added a commit that referenced this issue Jun 23, 2020
…gs` for Terraform 0.11 and earlier with boolean `propagate_at_launch` values

Reference: #13312
Reference: #13360
Reference: #13469
Reference: #13549
Reference: #13635

One of the major improvements internally with Terraform 0.12's type system is that providers and the core logic agree on a resource schema and its normalization during configuration parsing so the state and difference handling is more consistent. In Terraform 0.11 and earlier, a quirky behavior with raw boolean values is that they could be converted to string "0" and "1" values based on various factors such as passing through functions. With recent changes to the `aws_autoscaling_group` resource to fix its handling of the `tags` attribute (to prevent `d.Set()` errors and testing panics in the upcoming Terraform Plugin SDK 2.0.0), this behavior was highlighted as unexpected perpetual differences. This cannot be caught in the 0.12 shimmed acceptance testing framework and therefore is manually tested by the below configuration and reproduction steps.

**NOTE:** The `tags` argument will be deprecated and removed in a future major version of the Terraform AWS Provider since it was designed as a workaround for Terraform 0.11 and earlier configurations. There are very few, if any, other Terraform AWS Provider attributes represented as a set of maps. In Terraform 0.12 and later, the `tag` argument more appropriately represents Auto Scaling Group resource tags as configuration blocks with strongly typed nested arguments and can be dynamically built via the `dynamic` configuration language feature. That deprecation is out of scope for this changeset though and likely will not occur until after Terraform AWS Provider version 3.0.0.

Given the following Terraform 0.11 compatible configuration with raw boolean values:

```hcl
terraform {
  required_version = "0.11.14"
}

provider "aws" {
  region  = "us-east-2"
  version = "2.67.0"
}

locals {
  test = [
    {
      key                 = "localkey1"
      value               = "localvalue1"
      propagate_at_launch = true
    },
    {
      key                 = "localkey2"
      value               = "localvalue2"
      propagate_at_launch = true
    },
  ]
}

data "aws_ami" "test" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn-ami-hvm-*-x86_64-gp2"]
  }
}

data "aws_availability_zones" "available" {
  state = "available"

  filter {
    name   = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}

resource "aws_launch_template" "test" {
  name_prefix   = "test"
  image_id      = "${data.aws_ami.test.id}"
  instance_type = "t2.micro"
}

resource "aws_autoscaling_group" "test" {
  availability_zones = ["${data.aws_availability_zones.available.names[0]}"]
  desired_capacity   = 0
  max_size           = 0
  min_size           = 0
  tags               = ["${concat(list(map("key", "resourcekey1", "value", "resourcevalue1", "propagate_at_launch", true)), local.test)}"]

  launch_template {
    id      = "${aws_launch_template.test.id}"
    version = "${aws_launch_template.test.default_version}"
  }
}

output "tags" {
  value = "${aws_autoscaling_group.test.tags}"
}

```

Applying twice has a perpetual difference:

```console
$ terraform0.11.14 init

Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (2.67.0)...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

$ terraform0.11.14 apply
data.aws_ami.test: Refreshing state...
data.aws_availability_zones.available: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + aws_autoscaling_group.test
      id:                                  <computed>
      arn:                                 <computed>
      availability_zones.#:                "1"
      availability_zones.4293815384:       "us-east-2a"
      default_cooldown:                    <computed>
      desired_capacity:                    "0"
      force_delete:                        "false"
      health_check_grace_period:           "300"
      health_check_type:                   <computed>
      launch_template.#:                   "1"
      launch_template.0.id:                "${aws_launch_template.test.id}"
      launch_template.0.name:              <computed>
      launch_template.0.version:           "${aws_launch_template.test.default_version}"
      load_balancers.#:                    <computed>
      max_size:                            "0"
      metrics_granularity:                 "1Minute"
      min_size:                            "0"
      name:                                <computed>
      protect_from_scale_in:               "false"
      service_linked_role_arn:             <computed>
      tags.#:                              "3"
      tags.2825254636.%:                   "3"
      tags.2825254636.key:                 "localkey1"
      tags.2825254636.propagate_at_launch: "1"
      tags.2825254636.value:               "localvalue1"
      tags.2888644505.%:                   "3"
      tags.2888644505.key:                 "resourcekey1"
      tags.2888644505.propagate_at_launch: "1"
      tags.2888644505.value:               "resourcevalue1"
      tags.3299348900.%:                   "3"
      tags.3299348900.key:                 "localkey2"
      tags.3299348900.propagate_at_launch: "1"
      tags.3299348900.value:               "localvalue2"
      target_group_arns.#:                 <computed>
      vpc_zone_identifier.#:               <computed>
      wait_for_capacity_timeout:           "10m"

  + aws_launch_template.test
      id:                                  <computed>
      arn:                                 <computed>
      default_version:                     <computed>
      image_id:                            "ami-083ebc5a49573896a"
      instance_type:                       "t2.micro"
      latest_version:                      "0"
      metadata_options.#:                  <computed>
      name:                                <computed>
      name_prefix:                         "test"

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_launch_template.test: Creating...
  arn:                "" => "<computed>"
  default_version:    "" => "<computed>"
  image_id:           "" => "ami-083ebc5a49573896a"
  instance_type:      "" => "t2.micro"
  latest_version:     "" => "<computed>"
  metadata_options.#: "" => "<computed>"
  name:               "" => "<computed>"
  name_prefix:        "" => "test"
aws_launch_template.test: Creation complete after 0s (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Creating...
  arn:                                 "" => "<computed>"
  availability_zones.#:                "" => "1"
  availability_zones.4293815384:       "" => "us-east-2a"
  default_cooldown:                    "" => "<computed>"
  desired_capacity:                    "" => "0"
  force_delete:                        "" => "false"
  health_check_grace_period:           "" => "300"
  health_check_type:                   "" => "<computed>"
  launch_template.#:                   "" => "1"
  launch_template.0.id:                "" => "lt-0d8e6f4f94a683778"
  launch_template.0.name:              "" => "<computed>"
  launch_template.0.version:           "" => "1"
  load_balancers.#:                    "" => "<computed>"
  max_size:                            "" => "0"
  metrics_granularity:                 "" => "1Minute"
  min_size:                            "" => "0"
  name:                                "" => "<computed>"
  protect_from_scale_in:               "" => "false"
  service_linked_role_arn:             "" => "<computed>"
  tags.#:                              "" => "3"
  tags.2825254636.%:                   "" => "3"
  tags.2825254636.key:                 "" => "localkey1"
  tags.2825254636.propagate_at_launch: "" => "1"
  tags.2825254636.value:               "" => "localvalue1"
  tags.2888644505.%:                   "" => "3"
  tags.2888644505.key:                 "" => "resourcekey1"
  tags.2888644505.propagate_at_launch: "" => "1"
  tags.2888644505.value:               "" => "resourcevalue1"
  tags.3299348900.%:                   "" => "3"
  tags.3299348900.key:                 "" => "localkey2"
  tags.3299348900.propagate_at_launch: "" => "1"
  tags.3299348900.value:               "" => "localvalue2"
  target_group_arns.#:                 "" => "<computed>"
  vpc_zone_identifier.#:               "" => "<computed>"
  wait_for_capacity_timeout:           "" => "10m"
aws_autoscaling_group.test: Creation complete after 2s (ID: tf-asg-20200623194008467200000003)

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

tags = [
    {
        key = localkey2,
        propagate_at_launch = 1,
        value = localvalue2
    },
    {
        key = localkey1,
        propagate_at_launch = 1,
        value = localvalue1
    },
    {
        key = resourcekey1,
        propagate_at_launch = 1,
        value = resourcevalue1
    }
]

$ terraform0.11.14 apply
data.aws_ami.test: Refreshing state...
data.aws_availability_zones.available: Refreshing state...
aws_launch_template.test: Refreshing state... (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Refreshing state... (ID: tf-asg-20200623194008467200000003)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ aws_autoscaling_group.test
      tags.2131493069.%:                   "3" => "0"
      tags.2131493069.key:                 "localkey2" => ""
      tags.2131493069.propagate_at_launch: "true" => ""
      tags.2131493069.value:               "localvalue2" => ""
      tags.2825254636.%:                   "0" => "3"
      tags.2825254636.key:                 "" => "localkey1"
      tags.2825254636.propagate_at_launch: "" => "1"
      tags.2825254636.value:               "" => "localvalue1"
      tags.2888644505.%:                   "0" => "3"
      tags.2888644505.key:                 "" => "resourcekey1"
      tags.2888644505.propagate_at_launch: "" => "1"
      tags.2888644505.value:               "" => "resourcevalue1"
      tags.3299348900.%:                   "0" => "3"
      tags.3299348900.key:                 "" => "localkey2"
      tags.3299348900.propagate_at_launch: "" => "1"
      tags.3299348900.value:               "" => "localvalue2"
      tags.3593646732.%:                   "3" => "0"
      tags.3593646732.key:                 "localkey1" => ""
      tags.3593646732.propagate_at_launch: "true" => ""
      tags.3593646732.value:               "localvalue1" => ""
      tags.3707627218.%:                   "3" => "0"
      tags.3707627218.key:                 "resourcekey1" => ""
      tags.3707627218.propagate_at_launch: "true" => ""
      tags.3707627218.value:               "resourcevalue1" => ""

Plan: 0 to add, 1 to change, 0 to destroy.
```

Building this version of the provider, the difference is no longer present:

```console
$ cp ~/go/bin/terraform-provider-aws .terraform/plugins/darwin_amd64/terraform-provider-aws_v2.67.0_x4; terraform0.11.14 init; terraform0.11.14 apply

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
data.aws_availability_zones.available: Refreshing state...
data.aws_ami.test: Refreshing state...
aws_launch_template.test: Refreshing state... (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Refreshing state... (ID: tf-asg-20200623194008467200000003)

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

tags = [
    {
        key = localkey1,
        propagate_at_launch = 1,
        value = localvalue1
    },
    {
        key = resourcekey1,
        propagate_at_launch = 1,
        value = resourcevalue1
    },
    {
        key = localkey2,
        propagate_at_launch = 1,
        value = localvalue2
    }
]
```

Output from acceptance testing (Terraform 0.12 compatible):

```
--- PASS: TestAccAWSAutoScalingGroup_ALB_TargetGroups (176.05s)
--- PASS: TestAccAWSAutoScalingGroup_ALB_TargetGroups_ELBCapacity (342.40s)
--- PASS: TestAccAWSAutoScalingGroup_autoGeneratedName (89.91s)
--- PASS: TestAccAWSAutoScalingGroup_basic (259.25s)
--- PASS: TestAccAWSAutoScalingGroup_classicVpcZoneIdentifier (93.45s)
--- PASS: TestAccAWSAutoScalingGroup_emptyAvailabilityZones (92.72s)
--- PASS: TestAccAWSAutoScalingGroup_enablingMetrics (174.98s)
--- PASS: TestAccAWSAutoScalingGroup_initialLifecycleHook (271.36s)
--- PASS: TestAccAWSAutoScalingGroup_launchTemplate (62.35s)
--- PASS: TestAccAWSAutoScalingGroup_LaunchTemplate_IAMInstanceProfile (61.69s)
--- PASS: TestAccAWSAutoScalingGroup_launchTemplate_update (143.24s)
--- PASS: TestAccAWSAutoScalingGroup_launchTempPartitionNum (77.30s)
--- PASS: TestAccAWSAutoScalingGroup_LoadBalancers (695.77s)
--- PASS: TestAccAWSAutoScalingGroup_MaxInstanceLifetime (80.89s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy (108.93s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandAllocationStrategy (49.98s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandBaseCapacity (82.14s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandPercentageAboveBaseCapacity (96.59s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotAllocationStrategy (83.60s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotInstancePools (90.32s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotMaxPrice (112.90s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_UpdateToZeroOnDemandBaseCapacity (73.62s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_LaunchTemplateSpecification_LaunchTemplateName (86.69s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_LaunchTemplateSpecification_Version (85.84s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_Override_InstanceType (85.29s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_Override_WeightedCapacity (205.06s)
--- PASS: TestAccAWSAutoScalingGroup_namePrefix (45.70s)
--- PASS: TestAccAWSAutoScalingGroup_serviceLinkedRoleARN (83.67s)
--- PASS: TestAccAWSAutoScalingGroup_suspendingProcesses (267.51s)
--- PASS: TestAccAWSAutoScalingGroup_tags (299.92s)
--- PASS: TestAccAWSAutoScalingGroup_TargetGroupArns (249.92s)
--- PASS: TestAccAWSAutoScalingGroup_terminationPolicies (151.45s)
--- PASS: TestAccAWSAutoScalingGroup_VpcUpdates (236.40s)
--- PASS: TestAccAWSAutoScalingGroup_WithLoadBalancer (389.18s)
--- PASS: TestAccAWSAutoScalingGroup_WithLoadBalancer_ToTargetGroup (384.27s)
--- PASS: TestAccAWSAutoScalingGroup_withMetrics (96.85s)
--- PASS: TestAccAWSAutoScalingGroup_withPlacementGroup (200.69s)
```
@bflad
Copy link
Contributor

bflad commented Jun 23, 2020

Fix submitted for the aws_autoscaling_group resource tags handling: #13912

If there are issues with other attributes in the resource, please file a separate bug report for triage. Thanks.

@bflad bflad added bug Addresses a defect in current functionality. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. and removed needs-triage Waiting for first response or review from a maintainer. labels Jun 23, 2020
@bflad bflad added this to the v2.69.0 milestone Jul 1, 2020
bflad added a commit that referenced this issue Jul 1, 2020
…gs` for Terraform 0.11 and earlier with boolean `propagate_at_launch` values (#13912)

Reference: #13312
Reference: #13360
Reference: #13469
Reference: #13549
Reference: #13635

One of the major improvements internally with Terraform 0.12's type system is that providers and the core logic agree on a resource schema and its normalization during configuration parsing so the state and difference handling is more consistent. In Terraform 0.11 and earlier, a quirky behavior with raw boolean values is that they could be converted to string "0" and "1" values based on various factors such as passing through functions. With recent changes to the `aws_autoscaling_group` resource to fix its handling of the `tags` attribute (to prevent `d.Set()` errors and testing panics in the upcoming Terraform Plugin SDK 2.0.0), this behavior was highlighted as unexpected perpetual differences. This cannot be caught in the 0.12 shimmed acceptance testing framework and therefore is manually tested by the below configuration and reproduction steps.

**NOTE:** The `tags` argument will be deprecated and removed in a future major version of the Terraform AWS Provider since it was designed as a workaround for Terraform 0.11 and earlier configurations. There are very few, if any, other Terraform AWS Provider attributes represented as a set of maps. In Terraform 0.12 and later, the `tag` argument more appropriately represents Auto Scaling Group resource tags as configuration blocks with strongly typed nested arguments and can be dynamically built via the `dynamic` configuration language feature. That deprecation is out of scope for this changeset though and likely will not occur until after Terraform AWS Provider version 3.0.0.

Given the following Terraform 0.11 compatible configuration with raw boolean values:

```hcl
terraform {
  required_version = "0.11.14"
}

provider "aws" {
  region  = "us-east-2"
  version = "2.67.0"
}

locals {
  test = [
    {
      key                 = "localkey1"
      value               = "localvalue1"
      propagate_at_launch = true
    },
    {
      key                 = "localkey2"
      value               = "localvalue2"
      propagate_at_launch = true
    },
  ]
}

data "aws_ami" "test" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn-ami-hvm-*-x86_64-gp2"]
  }
}

data "aws_availability_zones" "available" {
  state = "available"

  filter {
    name   = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}

resource "aws_launch_template" "test" {
  name_prefix   = "test"
  image_id      = "${data.aws_ami.test.id}"
  instance_type = "t2.micro"
}

resource "aws_autoscaling_group" "test" {
  availability_zones = ["${data.aws_availability_zones.available.names[0]}"]
  desired_capacity   = 0
  max_size           = 0
  min_size           = 0
  tags               = ["${concat(list(map("key", "resourcekey1", "value", "resourcevalue1", "propagate_at_launch", true)), local.test)}"]

  launch_template {
    id      = "${aws_launch_template.test.id}"
    version = "${aws_launch_template.test.default_version}"
  }
}

output "tags" {
  value = "${aws_autoscaling_group.test.tags}"
}

```

Applying twice has a perpetual difference:

```console
$ terraform0.11.14 init

Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (2.67.0)...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

$ terraform0.11.14 apply
data.aws_ami.test: Refreshing state...
data.aws_availability_zones.available: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + aws_autoscaling_group.test
      id:                                  <computed>
      arn:                                 <computed>
      availability_zones.#:                "1"
      availability_zones.4293815384:       "us-east-2a"
      default_cooldown:                    <computed>
      desired_capacity:                    "0"
      force_delete:                        "false"
      health_check_grace_period:           "300"
      health_check_type:                   <computed>
      launch_template.#:                   "1"
      launch_template.0.id:                "${aws_launch_template.test.id}"
      launch_template.0.name:              <computed>
      launch_template.0.version:           "${aws_launch_template.test.default_version}"
      load_balancers.#:                    <computed>
      max_size:                            "0"
      metrics_granularity:                 "1Minute"
      min_size:                            "0"
      name:                                <computed>
      protect_from_scale_in:               "false"
      service_linked_role_arn:             <computed>
      tags.#:                              "3"
      tags.2825254636.%:                   "3"
      tags.2825254636.key:                 "localkey1"
      tags.2825254636.propagate_at_launch: "1"
      tags.2825254636.value:               "localvalue1"
      tags.2888644505.%:                   "3"
      tags.2888644505.key:                 "resourcekey1"
      tags.2888644505.propagate_at_launch: "1"
      tags.2888644505.value:               "resourcevalue1"
      tags.3299348900.%:                   "3"
      tags.3299348900.key:                 "localkey2"
      tags.3299348900.propagate_at_launch: "1"
      tags.3299348900.value:               "localvalue2"
      target_group_arns.#:                 <computed>
      vpc_zone_identifier.#:               <computed>
      wait_for_capacity_timeout:           "10m"

  + aws_launch_template.test
      id:                                  <computed>
      arn:                                 <computed>
      default_version:                     <computed>
      image_id:                            "ami-083ebc5a49573896a"
      instance_type:                       "t2.micro"
      latest_version:                      "0"
      metadata_options.#:                  <computed>
      name:                                <computed>
      name_prefix:                         "test"

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_launch_template.test: Creating...
  arn:                "" => "<computed>"
  default_version:    "" => "<computed>"
  image_id:           "" => "ami-083ebc5a49573896a"
  instance_type:      "" => "t2.micro"
  latest_version:     "" => "<computed>"
  metadata_options.#: "" => "<computed>"
  name:               "" => "<computed>"
  name_prefix:        "" => "test"
aws_launch_template.test: Creation complete after 0s (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Creating...
  arn:                                 "" => "<computed>"
  availability_zones.#:                "" => "1"
  availability_zones.4293815384:       "" => "us-east-2a"
  default_cooldown:                    "" => "<computed>"
  desired_capacity:                    "" => "0"
  force_delete:                        "" => "false"
  health_check_grace_period:           "" => "300"
  health_check_type:                   "" => "<computed>"
  launch_template.#:                   "" => "1"
  launch_template.0.id:                "" => "lt-0d8e6f4f94a683778"
  launch_template.0.name:              "" => "<computed>"
  launch_template.0.version:           "" => "1"
  load_balancers.#:                    "" => "<computed>"
  max_size:                            "" => "0"
  metrics_granularity:                 "" => "1Minute"
  min_size:                            "" => "0"
  name:                                "" => "<computed>"
  protect_from_scale_in:               "" => "false"
  service_linked_role_arn:             "" => "<computed>"
  tags.#:                              "" => "3"
  tags.2825254636.%:                   "" => "3"
  tags.2825254636.key:                 "" => "localkey1"
  tags.2825254636.propagate_at_launch: "" => "1"
  tags.2825254636.value:               "" => "localvalue1"
  tags.2888644505.%:                   "" => "3"
  tags.2888644505.key:                 "" => "resourcekey1"
  tags.2888644505.propagate_at_launch: "" => "1"
  tags.2888644505.value:               "" => "resourcevalue1"
  tags.3299348900.%:                   "" => "3"
  tags.3299348900.key:                 "" => "localkey2"
  tags.3299348900.propagate_at_launch: "" => "1"
  tags.3299348900.value:               "" => "localvalue2"
  target_group_arns.#:                 "" => "<computed>"
  vpc_zone_identifier.#:               "" => "<computed>"
  wait_for_capacity_timeout:           "" => "10m"
aws_autoscaling_group.test: Creation complete after 2s (ID: tf-asg-20200623194008467200000003)

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

tags = [
    {
        key = localkey2,
        propagate_at_launch = 1,
        value = localvalue2
    },
    {
        key = localkey1,
        propagate_at_launch = 1,
        value = localvalue1
    },
    {
        key = resourcekey1,
        propagate_at_launch = 1,
        value = resourcevalue1
    }
]

$ terraform0.11.14 apply
data.aws_ami.test: Refreshing state...
data.aws_availability_zones.available: Refreshing state...
aws_launch_template.test: Refreshing state... (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Refreshing state... (ID: tf-asg-20200623194008467200000003)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ aws_autoscaling_group.test
      tags.2131493069.%:                   "3" => "0"
      tags.2131493069.key:                 "localkey2" => ""
      tags.2131493069.propagate_at_launch: "true" => ""
      tags.2131493069.value:               "localvalue2" => ""
      tags.2825254636.%:                   "0" => "3"
      tags.2825254636.key:                 "" => "localkey1"
      tags.2825254636.propagate_at_launch: "" => "1"
      tags.2825254636.value:               "" => "localvalue1"
      tags.2888644505.%:                   "0" => "3"
      tags.2888644505.key:                 "" => "resourcekey1"
      tags.2888644505.propagate_at_launch: "" => "1"
      tags.2888644505.value:               "" => "resourcevalue1"
      tags.3299348900.%:                   "0" => "3"
      tags.3299348900.key:                 "" => "localkey2"
      tags.3299348900.propagate_at_launch: "" => "1"
      tags.3299348900.value:               "" => "localvalue2"
      tags.3593646732.%:                   "3" => "0"
      tags.3593646732.key:                 "localkey1" => ""
      tags.3593646732.propagate_at_launch: "true" => ""
      tags.3593646732.value:               "localvalue1" => ""
      tags.3707627218.%:                   "3" => "0"
      tags.3707627218.key:                 "resourcekey1" => ""
      tags.3707627218.propagate_at_launch: "true" => ""
      tags.3707627218.value:               "resourcevalue1" => ""

Plan: 0 to add, 1 to change, 0 to destroy.
```

Building this version of the provider, the difference is no longer present:

```console
$ cp ~/go/bin/terraform-provider-aws .terraform/plugins/darwin_amd64/terraform-provider-aws_v2.67.0_x4; terraform0.11.14 init; terraform0.11.14 apply

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
data.aws_availability_zones.available: Refreshing state...
data.aws_ami.test: Refreshing state...
aws_launch_template.test: Refreshing state... (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Refreshing state... (ID: tf-asg-20200623194008467200000003)

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

tags = [
    {
        key = localkey1,
        propagate_at_launch = 1,
        value = localvalue1
    },
    {
        key = resourcekey1,
        propagate_at_launch = 1,
        value = resourcevalue1
    },
    {
        key = localkey2,
        propagate_at_launch = 1,
        value = localvalue2
    }
]
```

Output from acceptance testing (Terraform 0.12 compatible):

```
--- PASS: TestAccAWSAutoScalingGroup_ALB_TargetGroups (176.05s)
--- PASS: TestAccAWSAutoScalingGroup_ALB_TargetGroups_ELBCapacity (342.40s)
--- PASS: TestAccAWSAutoScalingGroup_autoGeneratedName (89.91s)
--- PASS: TestAccAWSAutoScalingGroup_basic (259.25s)
--- PASS: TestAccAWSAutoScalingGroup_classicVpcZoneIdentifier (93.45s)
--- PASS: TestAccAWSAutoScalingGroup_emptyAvailabilityZones (92.72s)
--- PASS: TestAccAWSAutoScalingGroup_enablingMetrics (174.98s)
--- PASS: TestAccAWSAutoScalingGroup_initialLifecycleHook (271.36s)
--- PASS: TestAccAWSAutoScalingGroup_launchTemplate (62.35s)
--- PASS: TestAccAWSAutoScalingGroup_LaunchTemplate_IAMInstanceProfile (61.69s)
--- PASS: TestAccAWSAutoScalingGroup_launchTemplate_update (143.24s)
--- PASS: TestAccAWSAutoScalingGroup_launchTempPartitionNum (77.30s)
--- PASS: TestAccAWSAutoScalingGroup_LoadBalancers (695.77s)
--- PASS: TestAccAWSAutoScalingGroup_MaxInstanceLifetime (80.89s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy (108.93s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandAllocationStrategy (49.98s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandBaseCapacity (82.14s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandPercentageAboveBaseCapacity (96.59s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotAllocationStrategy (83.60s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotInstancePools (90.32s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotMaxPrice (112.90s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_UpdateToZeroOnDemandBaseCapacity (73.62s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_LaunchTemplateSpecification_LaunchTemplateName (86.69s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_LaunchTemplateSpecification_Version (85.84s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_Override_InstanceType (85.29s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_Override_WeightedCapacity (205.06s)
--- PASS: TestAccAWSAutoScalingGroup_namePrefix (45.70s)
--- PASS: TestAccAWSAutoScalingGroup_serviceLinkedRoleARN (83.67s)
--- PASS: TestAccAWSAutoScalingGroup_suspendingProcesses (267.51s)
--- PASS: TestAccAWSAutoScalingGroup_tags (299.92s)
--- PASS: TestAccAWSAutoScalingGroup_TargetGroupArns (249.92s)
--- PASS: TestAccAWSAutoScalingGroup_terminationPolicies (151.45s)
--- PASS: TestAccAWSAutoScalingGroup_VpcUpdates (236.40s)
--- PASS: TestAccAWSAutoScalingGroup_WithLoadBalancer (389.18s)
--- PASS: TestAccAWSAutoScalingGroup_WithLoadBalancer_ToTargetGroup (384.27s)
--- PASS: TestAccAWSAutoScalingGroup_withMetrics (96.85s)
--- PASS: TestAccAWSAutoScalingGroup_withPlacementGroup (200.69s)
```
@bflad
Copy link
Contributor

bflad commented Jul 1, 2020

The fix for this specific scenario has been merged and will release with version 2.69.0 of the Terraform AWS Provider, likely on Thursday. Unfortunately these cases involving Terraform 0.11 and earlier behaviors are harder to test (since our acceptance testing inherently runs using Terraform 0.12's mock handling of the older version), so please do reach out in followup bug reports there are still lingering issues and we will try to address them.

@ghost
Copy link

ghost commented Jul 3, 2020

This has been released in version 2.69.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Jul 31, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Jul 31, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. regression Pertains to a degraded workflow resulting from an upstream patch or internal enhancement. service/autoscaling Issues and PRs that pertain to the autoscaling service.
Projects
None yet
4 participants