Incorrect indexing of a splat with more than 10 elements because of lexigraphical sorting

[dang3r]

Salut,

Terraform Version

Terraform v0.7.8

Affected Resource(s)

• all resources will be affected as this is a terraform core issue, not a resource-specific one

Terraform Configuration Files

# user names
variable "users" {
  type = "list"

  default = [
    "foo",
    "bar",
    "baz",
  ]
}

# policy name => policy, users
variable "policies" {
  type = "list"

  default = [
    {
      name   = "policy0"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "foo bar"
    },
    {
      name   = "policy1"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "foo bar"
    },
    {
      name   = "policy2"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "foo bar"
    },
    {
      name   = "policy3"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
       users  = "foo baz"
    },
    {
      name   = "policy4"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
       users  = "foo baz"

    },
    {
      name   = "policy5"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
       users  = "foo baz"
    },
    {
      name   = "policy6"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "bar"
    },
    {
      name   = "policy7"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "foo"
    },
    {
      name   = "policy8"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "baz"
    },
    {
      name   = "policy9"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "bar"
    },
    {
      name   = "policy10"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "foo"
    },
    {
      name   = "policy11"
      policy = "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": \"*\",\"Resource\": \"*\"}]}"
      users  = "foo"
    },
  ]
}

# --- users ---
resource "aws_iam_user" "user" {
  count = "${length(var.users)}"
  name  = "${element(var.users, count.index)}"
}

# --- policies ---
resource "aws_iam_policy" "policy" {
  count  = "${length(var.policies)}"
  name   = "${lookup(var.policies[count.index], "name")}"
  policy = "${lookup(var.policies[count.index], "policy")}"
}

resource "aws_iam_policy_attachment" "policy_attachment" {
  depends_on = ["aws_iam_user.user", "aws_iam_policy.policy"]
  count      = "${length(var.policies)}"
  name       = "${lookup(var.policies[count.index], "name")}"
  users      = ["${compact(split(" ", lookup(var.policies[count.index], "users")))}"]
  policy_arn = "${aws_iam_policy.policy.*.arn[count.index]}"
}

Debug Output

https://gist.github.com/dang3r/02a50747ffae7d7f323d319df1ac944e

Expected Behavior

When executing a terraform plan ., the plan is correct and maps the correct policy to the correct policy_arn. The resulting sample output lines were retrieved after executing terraform apply. && terraform plan . Observe that the policy_arn for each policy attachment is being modified. The expected behaviour is that after the initial apply, because the files haven't been changed, a plan should produce no changes.

-/+ aws_iam_policy_attachment.policy_attachment.10
    name:             "policy10" => "policy10"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy8" => "arn:aws:iam::068555316154:policy/policy10" (forces new resource)
    users.#:          "1" => "1"
    users.2356372769: "foo" => "foo"

-/+ aws_iam_policy_attachment.policy_attachment.11
    name:             "policy11" => "policy11"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy9" => "arn:aws:iam::068555316154:policy/policy11" (forces new resource)
    users.#:          "1" => "1"
    users.2356372769: "foo" => "foo"

-/+ aws_iam_policy_attachment.policy_attachment.2
    name:             "policy2" => "policy2"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy10" => "arn:aws:iam::068555316154:policy/policy2" (forces new resource)
    users.#:          "2" => "2"
    users.1996459178: "bar" => "bar"
    users.2356372769: "foo" => "foo"

-/+ aws_iam_policy_attachment.policy_attachment.3
    name:             "policy3" => "policy3"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy11" => "arn:aws:iam::068555316154:policy/policy3" (forces new resource)
    users.#:          "2" => "2"
    users.2015626392: "baz" => "baz"
    users.2356372769: "foo" => "foo"

-/+ aws_iam_policy_attachment.policy_attachment.4
    name:             "policy4" => "policy4"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy2" => "arn:aws:iam::068555316154:policy/policy4" (forces new resource)
    users.#:          "2" => "2"
    users.2015626392: "baz" => "baz"
    users.2356372769: "foo" => "foo"

-/+ aws_iam_policy_attachment.policy_attachment.5
    name:             "policy5" => "policy5"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy3" => "arn:aws:iam::068555316154:policy/policy5" (forces new resource)
    users.#:          "2" => "2"
    users.2015626392: "baz" => "baz"
    users.2356372769: "foo" => "foo"

-/+ aws_iam_policy_attachment.policy_attachment.6
    name:             "policy6" => "policy6"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy4" => "arn:aws:iam::068555316154:policy/policy6" (forces new resource)
    users.#:          "1" => "1"
    users.1996459178: "bar" => "bar"

-/+ aws_iam_policy_attachment.policy_attachment.7
    name:             "policy7" => "policy7"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy5" => "arn:aws:iam::068555316154:policy/policy7" (forces new resource)
    users.#:          "1" => "1"
    users.2356372769: "foo" => "foo"

-/+ aws_iam_policy_attachment.policy_attachment.8
    name:             "policy8" => "policy8"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy6" => "arn:aws:iam::068555316154:policy/policy8" (forces new resource)
    users.#:          "1" => "1"
    users.2015626392: "baz" => "baz"

-/+ aws_iam_policy_attachment.policy_attachment.9
    name:             "policy9" => "policy9"
    policy_arn:       "arn:aws:iam::068555316154:policy/policy7" => "arn:aws:iam::068555316154:policy/policy9" (forces new resource)
    users.#:          "1" => "1"
    users.1996459178: "bar" => "bar"

Actual Behavior

The policy_arn that is being mapped to is correct when executing a plan, but incorrect after an apply has been peformed.

Steps to Reproduce

1. terraform apply - create initial resources
2. terraform apply - see changes despite the files being static

Important Factoids

I believe this issue is a result of depending on the order of an exported splat from a resource. Observe that in the example above the policy_arn for aws_iam_policy_attachment.policy_attachment.2 is being modified to go from 10->2 during the plan phase (policy_arn: "arn:aws:iam::068555316154:policy/policy10" => "arn:aws:iam::068555316154:policy/policy2" (forces new resource)).

I believe the issue is that after terraform creates the initial aws_iam_policy resources, they are ordered in the following fashion:

• aws_iam_policy.policy.0
• aws_iam_policy.policy.1
• aws_iam_policy.policy.10
• aws_iam_policy.policy.11
• aws_iam_policy.policy.2

When indexing into this splat to retrieve the associated policy_arn, this causes the aws_iam_policy_attachment.policy_attachment.2 resource to get 10's policy_arn. This is because each resource is probably given a unique ID using its index, and the sorting algorithm orders it alphanumerically. Is this expected behaviour as a result of how the unique string ids associated with resources are generated?

Edit : Workaround

Our current workaround is to change the interpolation for the policy_arn to
policy_arn = "${element(aws_iam_policy.policy.*.arn, index(aws_iam_policy.policy.*.name, lookup(var.policies[count.index], "name")))}"
The downfall of this approach is that if our policy list is modified, all resources will be destroyed and brought up again because of the interpolation.

[mitchellh]

Oh! I just fixed this with #9883. It also references a dup there. :)

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.