Terraform AWS KMS Key Policy fails when used with AWS IAM Policy Document on AWS Provider >= 3.68.0

[jabouchleih]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Starting with version 3.68.0 of the AWS provider, the code snippet shown below fails with the mentioned error.

Terraform CLI and Terraform AWS Provider Version

• Terraform v1.1.4 on darwin_arm64
• aws 3.74.0

Affected Resource(s)

• aws_kms_key
• aws_iam_policy_document

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_kms_key" "sqs-key" {
  description             = "key to encrypt sqs data for ${var.app_name}"
  deletion_window_in_days = 14
  policy = data.aws_iam_policy_document.optional-sqs-kms-key-policy.json
}

data "aws_iam_policy_document" "optional-sqs-kms-key-policy" {
  statement {
    sid    = "AllowSQSAccessToKey"
    effect = "Allow"
    principals {
      identifiers = ["sqs.amazonaws.com"]
      type        = "Service"
    }
    actions = [
      "kms:CreateGrant",
      "kms:ListGrants",
      "kms:RevokeGrant"
    ]
    resources = [
      "*"
    ]

    condition {
      test     = "Bool"
      values   = ["true"]
      variable = "kms:GrantIsForAWSResource"
    }
    condition {
      test     = "StringEquals"
      values   = ["sqs.${var.region}.amazonaws.com"]
      variable = "kms:ViaService"
    }
  }
}

Panic Output

│ Error: while setting policy ({ "Statement": [ { "Action": [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ], "Condition": { "Bool": { "kms:GrantIsForAWSResource": "true" }, "StringEquals": { "kms:ViaService": "sqs.eu-west-1.amazonaws.com" } }, "Effect": "Allow", "Principal": { "Service": "sqs.amazonaws.com" }, "Resource": "*", "Sid": "AllowSQSAccessToKey" } ], "Version": "2012-10-17" }), encountered: Error parsing policy: Error parsing statement: 1 error(s) decoding:
│ 
│ * '[0].Condition' expected a map, got 'slice'
│ 
│   with module.appdeployment.module.sqs["0"].aws_kms_key.sqs-key,
│   on .terraform/modules/appdeployment/modules/sqs/main.tf line 21, in resource "aws_kms_key" "sqs-key":
│   21: resource "aws_kms_key" "sqs-key" {

Further information

Only AWS provider versions >= 3.68.0 are affected by this bug.

Expected Behavior

Terraform should apply the AWS KMS Key Policy correctly

Actual Behavior

Terraform apply fails with the Go output:

 '[0].Condition' expected a map, got 'slice'

Steps to Reproduce

1. Create a KMS Key resource and reference the aws_iam_policy_document as shown above
2. terraform apply

[ewbankkit]

IAM policy Conditions.

[koyedele]

A possible workaround for this is to specify the policy directly inline instead of using the aws_iam_policy_document data source. So something like:

{
  Version = "2012-10-17"
  Statement = [
    {
      Sid    = "AllowSQSAccessToKey"
      Effect = "Allow"
      Principal = {
        Service = "sqs.amazonaws.com"
      }
      Action = [
        "kms:CreateGrant",
        "kms:ListGrants",
        "kms:RevokeGrant"
      ]
      Resource = "*"
    }
  ]
  Condition = {
    "Bool" = {
      "kms:GrantIsForAWSResource" = "true"
    }
    "StringEquals" = {
      "kms:ViaService" = "sqs.${var.aws_region}.amazonaws.com"
    }
  }
}

[jamesboehmer]

A possible workaround for this is to specify the policy directly inline instead of using the aws_iam_policy_document data source. So something like:

I'm having the same issue with 3.70 (and have tried 4.19.0 as well), and inlining the policy doesn't help. I get the same error. My condition block:

  Condition = {
    "StringLike" = {
      "aws:sourceArn" = "arn:aws:lambda:${var.aws_region}:${local.account_id}:function:*"
    }
  } 

The error message:

Error: while setting policy (), encountered: Error parsing policy: Error parsing statement: 3 error(s) decoding: * '[1].Condition[Test]' expected a map, got 'string' * '[1].Condition[Values]' expected a map, got 'slice' * '[1].Condition[Variable]' expected a map, got 'string'

I have tried using a policy doc as well as a JSON heredoc. Same error in both cases.

[applike-ss]

This issue also exists when using the basic example for data.iam_policy_document from the docs (https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document). Even without a condition being set.

[applike-ss]

While i was facing the same error message, the root cause for me was in fact that i was able to screw up the state.

Terraform saved an invalid iam policy json in the state when i did an apply (which did not succeed, because the aws api declied the update of the resource).

Removing the resource and importing it again fixed the state for me and allowed me to terraform apply the resource again.

[paololazzari]

@applike-ss facing the same issue; how did you remove it from the state? I don't have neither the document nor the policy in the state

[applike-ss]

i used terraform state rm path.to.resource. Are you sure you didn't apply it at least once? Maybe it is two separate issues then.

[paololazzari]

I couldn't see the resource in terraform state list, but after setting TF_LOG=error I was able to find the corrupt resource in the terraform plan output,. and then removed it with terraform state rm

[jar-b]

I believe this has been fixed. Attempting to reproduce with the latest provider results in a successful apply.

Note: Minor modifications were made to the original config to remove variable references and add permissions cover a KMS safety check.

I'll leave this open with the waiting response label to allow time to confirm whether the bug still exists with the original configuration using the latest provider.

Terraform Config

Show

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

provider "aws" {}

resource "aws_kms_key" "sqs-key" {
  description             = "key to encrypt sqs data for a thing"
  deletion_window_in_days = 14
  policy                  = data.aws_iam_policy_document.optional-sqs-kms-key-policy.json
}

data "aws_iam_policy_document" "optional-sqs-kms-key-policy" {
  # added to prevent "missing principal permissions" error:
  #   Error: error creating KMS Key: MalformedPolicyDocumentException: The new key policy will not allow you to update the key policy in the future.
  #   
  # ref: https://aws.amazon.com/premiumsupport/knowledge-center/update-key-policy-future/
  statement {
    sid    = "RequiredManagement"
    effect = "Allow"
    principals {
      type        = "AWS"
      identifiers = ["*"]
    }
    actions   = ["kms:*"]
    resources = ["*"]
  }

  statement {
    sid    = "AllowSQSAccessToKey"
    effect = "Allow"
    principals {
      identifiers = ["sqs.amazonaws.com"]
      type        = "Service"
    }
    actions = [
      "kms:CreateGrant",
      "kms:ListGrants",
      "kms:RevokeGrant"
    ]
    resources = [
      "*"
    ]

    condition {
      test     = "Bool"
      values   = ["true"]
      variable = "kms:GrantIsForAWSResource"
    }
    condition {
      test     = "StringEquals"
      values   = ["sqs.us-west-2.amazonaws.com"]
      variable = "kms:ViaService"
    }
  }
}

Reproduction Attempt

$ terraform -v
Terraform v1.3.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.37.0

$ terraform apply -auto-approve
data.aws_iam_policy_document.optional-sqs-kms-key-policy: Reading...
data.aws_iam_policy_document.optional-sqs-kms-key-policy: Read complete after 1s [id=3381652016]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_kms_key.sqs-key will be created
  + resource "aws_kms_key" "sqs-key" {
      + arn                                = (known after apply)
      + bypass_policy_lockout_safety_check = false
      + customer_master_key_spec           = "SYMMETRIC_DEFAULT"
      + deletion_window_in_days            = 14
      + description                        = "key to encrypt sqs data for a thing"
      + enable_key_rotation                = false
      + id                                 = (known after apply)
      + is_enabled                         = true
      + key_id                             = (known after apply)
      + key_usage                          = "ENCRYPT_DECRYPT"
      + multi_region                       = (known after apply)
      + policy                             = jsonencode(
            {
              + Statement = [
                  + {
                      + Action    = "kms:*"
                      + Effect    = "Allow"
                      + Principal = {
                          + AWS = "*"
                        }
                      + Resource  = "*"
                      + Sid       = "RequiredManagement"
                    },
                  + {
                      + Action    = [
                          + "kms:RevokeGrant",
                          + "kms:ListGrants",
                          + "kms:CreateGrant",
                        ]
                      + Condition = {
                          + Bool         = {
                              + "kms:GrantIsForAWSResource" = "true"
                            }
                          + StringEquals = {
                              + "kms:ViaService" = "sqs.us-west-2.amazonaws.com"
                            }
                        }
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "sqs.amazonaws.com"
                        }
                      + Resource  = "*"
                      + Sid       = "AllowSQSAccessToKey"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + tags_all                           = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.
aws_kms_key.sqs-key: Creating...
aws_kms_key.sqs-key: Creation complete after 5s [id=1ff4a026-c7a3-4abe-a8e1-525f4fd140f0]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

[jar-b]

Closing given no further response to reproduction.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.