dynamic counts not persisted in plan output

[ktham]

Terraform Version

0.11.7

Expected Behavior

Terraform should execute the plan as follows:

➜  terraform plan -out tf.out

# ...

Terraform will perform the following actions:

  + module.private_subnet.aws_route.nat_gateway[0]
      id:                         <computed>
      destination_cidr_block:     "0.0.0.0/0"
      destination_prefix_list_id: <computed>
      egress_only_gateway_id:     <computed>
      gateway_id:                 <computed>
      instance_id:                <computed>
      instance_owner_id:          <computed>
      nat_gateway_id:             "nat-0b52769ddb6d3f640"
      network_interface_id:       <computed>
      origin:                     <computed>
      route_table_id:             "rtb-657e3f19"
      state:                      <computed>

  + module.private_subnet.aws_route.nat_gateway[1]
      id:                         <computed>
      destination_cidr_block:     "0.0.0.0/0"
      destination_prefix_list_id: <computed>
      egress_only_gateway_id:     <computed>
      gateway_id:                 <computed>
      instance_id:                <computed>
      instance_owner_id:          <computed>
      nat_gateway_id:             "nat-0b52769ddb6d3f640"
      network_interface_id:       <computed>
      origin:                     <computed>
      route_table_id:             "rtb-d87839a4"
      state:                      <computed>

  + module.private_subnet.aws_route.nat_gateway[2]
      id:                         <computed>
      destination_cidr_block:     "0.0.0.0/0"
      destination_prefix_list_id: <computed>
      egress_only_gateway_id:     <computed>
      gateway_id:                 <computed>
      instance_id:                <computed>
      instance_owner_id:          <computed>
      nat_gateway_id:             "nat-0b52769ddb6d3f640"
      network_interface_id:       <computed>
      origin:                     <computed>
      route_table_id:             "rtb-ac7233d0"
      state:                      <computed>


Plan: 3 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

This plan was saved to: tf.out

To perform exactly these actions, run the following command to apply:
    terraform apply "tf.out"

Actual Behavior

Terraform silently fails executing the plan.

➜  terraform apply "tf.out"
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Steps to Reproduce

1. terraform init
2. terraform plan -out tf.out
3. terraform apply "tf.out"

Terraform Configuration Files

locals {
  legacy_vpc_id     = "vpc-xxx"
  legacy_aws_region = "us-east-1"
  azs               = [
    "${local.legacy_aws_region}a",
    "${local.legacy_aws_region}b",
    "${local.legacy_aws_region}c"
  ]
  public_subnet_cidrs = [
    "10.0.8.0/24",
    "10.0.9.0/24",
    "10.0.10.0/24"
  ]
}

provider "aws" {
  version = "~> 1.15.0"
  region  = "${local.legacy_aws_region}"
}
resource "aws_subnet" "public" {
  vpc_id                  = "${local.legacy_vpc_id}"
  cidr_block              = "${element(local.public_subnet_cidrs, count.index)}"
  availability_zone       = "${element(local.azs, count.index)}"
  count                   = "${length(local.public_subnet_cidrs)}"

  tags {
    Name = "legacy-public.${element(local.azs, count.index)}"
  }

  map_public_ip_on_launch = true
}

## NAT

module "nat" {
  source            = "../../terraform_modules/aws/vpc/nat"

  name              = "legacy-nat"
  azs               = "${local.azs}"
  public_subnet_ids = "${aws_subnet.public.*.id}"
}

## Private Subnet

module "private_subnet" {
  source          = "../../terraform_modules/aws/vpc/private_subnet"

  name            = "legacy-private"
  vpc_id          = "${local.legacy_vpc_id}"
  azs             = "${local.azs}"
  cidrs           = [
    "10.0.16.0/20",
    "10.0.32.0/20",
    "10.0.48.0/20"
  ]
  nat_gateway_ids = "${module.nat.nat_gateway_ids}"
}

private subnet module

variable "name" {
  type    = "string"
  default = "private"
}

variable "vpc_id" {
  type = "string"
}

variable "cidrs" {
  type    = "list"
  default = []
}

variable "azs" {
  type    = "list"
  default = []
}

variable "nat_gateway_ids" {
  type    = "list"
  default = []
}

resource "aws_subnet" "private" {
  vpc_id            = "${var.vpc_id}"
  cidr_block        = "${element(var.cidrs, count.index)}"
  availability_zone = "${element(var.azs, count.index)}"
  count             = "${length(var.cidrs)}"

  tags {
    Name = "${var.name}.${element(var.azs, count.index)}"
  }
}

resource "aws_route_table" "private" {
  vpc_id = "${var.vpc_id}"
  count  = "${length(var.cidrs)}"

  tags {
    Name = "${var.name}.${element(var.azs, count.index)}"
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_route" "nat_gateway" {
  route_table_id         = "${element(aws_route_table.private.*.id, count.index)}"
  count                  = "${aws_route_table.private.count}"

  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = "${element(var.nat_gateway_ids, count.index)}"
}

resource "aws_route_table_association" "private" {
  count          = "${length(var.cidrs)}"
  subnet_id      = "${element(aws_subnet.private.*.id, count.index)}"
  route_table_id = "${element(aws_route_table.private.*.id, count.index)}"

  lifecycle {
    create_before_destroy = true
  }
}

// Note: Resources that depend on the subnet actually depends on the subnet AND the route table association with it
output "subnet_ids" {
  value = ["${aws_route_table_association.private.*.subnet_id}"]
}

output "route_table_ids" {
  value = ["${aws_route_table.private.*.id}"]
}

NAT module

variable "name" {
  type = "string"
}

variable "azs" {
  type = "list"
  default = []
}

variable "public_subnet_ids" {
  type = "list"
  default = []
}

resource "aws_eip" "nat" {
  vpc   = true

  # count = "${length(var.azs)}" # Comment out count to only have 1 NAT

  lifecycle { create_before_destroy = true }
}

resource "aws_nat_gateway" "nat" {
  allocation_id = "${element(aws_eip.nat.*.id, count.index)}"
  subnet_id     = "${element(var.public_subnet_ids, count.index)}"

  # count = "${length(var.azs)}" # Comment out count to only have 1 NAT

  lifecycle { create_before_destroy = true }
}

# Note: Id list is comma-separated
output "nat_gateway_ids" {
  value = ["${aws_nat_gateway.nat.*.id}"]
}

Debug Output

2018/04/24 11:26:36 [INFO] Terraform version: 0.11.7
2018/04/24 11:26:36 [INFO] Go runtime version: go1.10.1

# ...

2018/04/24 11:26:40 [DEBUG] Resource state not found for "module.private_subnet.aws_route.nat_gateway[2]": module.private_subnet.aws_route.nat_gateway[2]
2018/04/24 11:26:40 [DEBUG] Resource state not found for "module.private_subnet.aws_route.nat_gateway[0]": module.private_subnet.aws_route.nat_gateway[0]
2018/04/24 11:26:40 [DEBUG] Resource state not found for "module.private_subnet.aws_route.nat_gateway[1]": module.private_subnet.aws_route.nat_gateway[1]
2018/04/24 11:26:40 [TRACE] Graph after step *terraform.AttachStateTransformer:

module.private_subnet.aws_route.nat_gateway[0] - *terraform.NodeApplyableResource
module.private_subnet.aws_route.nat_gateway[1] - *terraform.NodeApplyableResource
module.private_subnet.aws_route.nat_gateway[2] - *terraform.NodeApplyableResource
2018/04/24 11:26:40 [TRACE] ProviderConfigTransformer: Starting for path: []
2018/04/24 11:26:40 [DEBUG] resource module.private_subnet.aws_route.nat_gateway[0] using provider provider.aws
2018/04/24 11:26:40 [DEBUG] resource module.private_subnet.aws_route.nat_gateway[1] using provider provider.aws
2018/04/24 11:26:40 [DEBUG] resource module.private_subnet.aws_route.nat_gateway[2] using provider provider.aws
2018/04/24 11:26:40 [TRACE] Graph after step *terraform.ProviderTransformer:

module.private_subnet.aws_route.nat_gateway[0] - *terraform.NodeApplyableResource
  provider.aws - *terraform.NodeApplyableProvider
module.private_subnet.aws_route.nat_gateway[1] - *terraform.NodeApplyableResource
  provider.aws - *terraform.NodeApplyableProvider
module.private_subnet.aws_route.nat_gateway[2] - *terraform.NodeApplyableResource
  provider.aws - *terraform.NodeApplyableProvider
provider.aws - *terraform.NodeApplyableProvider
2018/04/24 11:26:40 [INFO] Not including "module.nat.var.name" in graph, nothing depends on it
2018/04/24 11:26:40 [INFO] Not including "module.nat.var.azs" in graph, nothing depends on it
2018/04/24 11:26:40 [INFO] Not including "module.nat.var.public_subnet_ids" in graph, nothing depends on it
2018/04/24 11:26:40 [INFO] Not including "module.private_subnet.var.name" in graph, nothing depends on it
2018/04/24 11:26:40 [INFO] Not including "module.private_subnet.var.vpc_id" in graph, nothing depends on it
2018/04/24 11:26:40 [INFO] Not including "module.private_subnet.var.cidrs" in graph, nothing depends on it
2018/04/24 11:26:40 [INFO] Not including "module.private_subnet.var.azs" in graph, nothing depends on it
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "module.private_subnet.aws_route.nat_gateway[1]" references: [module.private_subnet.var.nat_gateway_ids]
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "local.legacy_vpc_id" references: []
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "local.legacy_aws_region" references: []
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "local.azs" references: [local.legacy_aws_region]
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "module.private_subnet.aws_route.nat_gateway[2]" references: [module.private_subnet.var.nat_gateway_ids]
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "module.private_subnet.aws_route.nat_gateway[0]" references: [module.private_subnet.var.nat_gateway_ids]
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "provider.aws" references: [local.legacy_aws_region]
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "local.public_subnet_cidrs" references: []
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "module.nat.output.nat_gateway_ids" references: []
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "module.private_subnet.output.subnet_ids" references: []
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "module.private_subnet.output.route_table_ids" references: []
2018/04/24 11:26:40 [DEBUG] ReferenceTransformer: "module.private_subnet.var.nat_gateway_ids" references: [module.nat.output.nat_gateway_ids]
vate_subnet.var.nat_gateway_ids - *terraform.NodeApplyableModuleVariable
  module.nat.output.nat_gateway_ids - *terraform.NodeApplyableOutput
provider.aws - *terraform.NodeApplyableProvider
  local.legacy_aws_region - *terraform.NodeLocal
ubnet.var.nat_gateway_ids - *terraform.NodeApplyableModuleVariable
  module.nat.output.nat_gateway_ids - *terraform.NodeApplyableOutput
provider.aws - *terraform.NodeApplyableProvider
  local.legacy_aws_region - *terraform.NodeLocal
vate_subnet.var.nat_gateway_ids - *terraform.NodeApplyableModuleVariable
  module.nat.output.nat_gateway_ids - *terraform.NodeApplyableOutput
provider.aws - *terraform.NodeApplyableProvider
  local.legacy_aws_region - *terraform.NodeLocal
provider.aws (close) - *terraform.graphNodeCloseProvider
  module.private_subnet.aws_route.nat_gateway[0] - *terraform.NodeApplyableResource
  module.private_subnet.aws_route.nat_gateway[1] - *terraform.NodeApplyableResource
  module.private_subnet.aws_route.nat_gateway[2] - *terraform.NodeApplyableResource
  provider.aws - *terraform.NodeApplyableProvider
private_subnet.var.nat_gateway_ids - *terraform.NodeApplyableModuleVariable
  module.nat.output.nat_gateway_ids - *terraform.NodeApplyableOutput
provider.aws - *terraform.NodeApplyableProvider
  local.legacy_aws_region - *terraform.NodeLocal
provider.aws (close) - *terraform.graphNodeCloseProvider
  module.private_subnet.aws_route.nat_gateway[0] - *terraform.NodeApplyableResource
  module.private_subnet.aws_route.nat_gateway[1] - *terraform.NodeApplyableResource
  module.private_subnet.aws_route.nat_gateway[2] - *terraform.NodeApplyableResource
  provider.aws - *terraform.NodeApplyableProvider
et.var.nat_gateway_ids - *terraform.NodeApplyableModuleVariable
  module.nat.output.nat_gateway_ids - *terraform.NodeApplyableOutput
provider.aws - *terraform.NodeApplyableProvider
  local.legacy_aws_region - *terraform.NodeLocal
provider.aws (close) - *terraform.graphNodeCloseProvider
  module.private_subnet.aws_route.nat_gateway[0] - *terraform.NodeApplyableResource
  module.private_subnet.aws_route.nat_gateway[1] - *terraform.NodeApplyableResource
  module.private_subnet.aws_route.nat_gateway[2] - *terraform.NodeApplyableResource
  provider.aws - *terraform.NodeApplyableProvider
root - terraform.graphNodeRoot
  meta.count-boundary (count boundary fixup) - *terraform.NodeCountBoundary
  provider.aws (close) - *terraform.graphNodeCloseProvider
.aws (close) - *terraform.graphNodeCloseProvider
  module.private_subnet.aws_route.nat_gateway[0] - *terraform.NodeApplyableResource
  module.private_subnet.aws_route.nat_gateway[1] - *terraform.NodeApplyableResource
  module.private_subnet.aws_route.nat_gateway[2] - *terraform.NodeApplyableResource
root - terraform.graphNodeRoot
  meta.count-boundary (count boundary fixup) - *terraform.NodeCountBoundary
  provider.aws (close) - *terraform.graphNodeCloseProvider
2018/04/24 11:26:40 [DEBUG] Starting graph walk: walkApply


# ...

2018/04/24 11:26:44 [WARN] Interpolation "count" failed: Error reading aws_route_table.private count: cannot parse "${length(var.cidrs)}" as an integer
2018/04/24 11:26:44 [TRACE] [walkApply] Entering eval tree: module.private_subnet.aws_route.nat_gateway[0]
2018/04/24 11:26:44 [WARN] Interpolation "count" failed: Error reading aws_route_table.private count: cannot parse "${length(var.cidrs)}" as an integer
2018/04/24 11:26:44 [TRACE] root.private_subnet: eval: *terraform.EvalInterpolate, err: early exit
2018/04/24 11:26:44 [WARN] Interpolation "count" failed: Error reading aws_route_table.private count: cannot parse "${length(var.cidrs)}" as an integer
2018/04/24 11:26:44 [TRACE] root.private_subnet: eval: *terraform.EvalInterpolate, err: early exit

[jbardin]

Hi @ktham,

Thanks for the very complete bug report.

This is a known issue at the moment, because of the way count is handled by the configuration.
It was somewhat mitigated by #17133 and #17548, but this scenario is one that isn't covered quite yet.

We intend change the plan serialization format in the next major release, which would allow us to add the required data to allow interpolated counts to work. If this isn't being run in automation, running terraform apply and confirming the plan interactively should work around the issue.

I think that you may also be able to avoid this in aws_route.nat_gateway by setting count to "${length(var.cidrs)}", avoiding the indirectly interpolated "${aws_route_table.private.count}" value which is causing the issue.

[ktham]

I see, thanks @jbardin . We are ok with doing the interactive terraform apply as a one-off thing outside of our TFE workflow, but ideally we want to avoid doing this outside of our automation. Any estimate on how far out this serialization format change is? (is that the hcl2 stuff?)

I'll try that last suggestion to see if that works.

[jbardin]

The plan output changes are directly related to the HCL2 work, so a complete fix will need to wait until the next major release.

[mildwonkey]

Hi! With the release of Terraform 0.12, there have been several bug fixes, enhancements, and syntax changes which impact this example.

I was able to get your example code running with only a few modifications. The terraform 0.12upgrade command should be able to take care of most of them for you, or at least offer some suggestions.

The "count" attribute is no longer supported in terraform 0.12:

Error: Invalid resource count attribute

  on modules/private_subnet/main.tf line 51, in resource "aws_route" "nat_gateway":
  51:   count                  = "${aws_route_table.private.count}"

The special "count" attribute is no longer supported after Terraform v0.12.
Instead, use length(aws_route_table.private) to count resource instances.

I am going to close this since the main issue is fixed in terraform 0.12. If you upgrade to terraform 0.12 and see any odd behaviors, please open a new issue. Thank you!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.