r/s3_bucket_lifecycle_configuration: Provider produced inconsistent final plan

[Nuru]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.6
on darwin_amd64

• provider registry.terraform.io/hashicorp/aws v4.8.0

Affected Resource(s)

• s3_bucket_lifecycle_configuration

Terraform Configuration Files

main.tf (click to reveal)

variable "trigger" {
  type    = bool
  default = false
}

resource "aws_s3_bucket" "this" {
  bucket = "eg-test-aws-issue-23883"
}

resource "aws_s3_bucket_versioning" "this" {
  bucket = aws_s3_bucket.this.id

  versioning_configuration {
    status = "Enabled"
  }
}

locals {
  lc_rules_list = [
    {
      abort_incomplete_multipart_upload_days = null
      enabled                                = true
      expiration = {
        days                         = 1
        expired_object_delete_marker = null
      }

      # test no filter
      filter_prefix_only = null
      filter_and = {
        prefix                   = null
        object_size_greater_than = null
        tags = {
          temp = "true"
        }
      }

      id = "rule-1"
    },
    {
      abort_incomplete_multipart_upload_days = 1
      enabled                                = true
      expiration = {
        days                         = 1464
        expired_object_delete_marker = null
      }

      # test no filter
      filter_prefix_only = null
      filter_and         = null
      id                 = "rule-2"
    },

    {
      abort_incomplete_multipart_upload_days = 1
      enabled                                = true
      expiration = {
        days                         = null
        expired_object_delete_marker = null
      }

      # test no filter
      filter_prefix_only = null
      filter_and         = null
      id                 = "nofilter"
    },
    {
      abort_incomplete_multipart_upload_days = 1
      enabled                                = true
      expiration = {
        days                         = null
        expired_object_delete_marker = null
      }

      # test prefix only
      filter_prefix_only = {
        prefix = "prefix1"
      }
      filter_and = null
      id         = "prefix1"
    },
    {
      abort_incomplete_multipart_upload_days = null
      enabled                                = true
      expiration = {
        days                         = 1461
        expired_object_delete_marker = false
      }
      # test prefix with other filter
      filter_prefix_only = null
      filter_and = {
        prefix                   = "prefix2"
        object_size_greater_than = 128 * 1024
        tags                     = null
      }
      id = "prefix2"
    },

    {
      abort_incomplete_multipart_upload_days = null
      enabled                                = true
      expiration = {
        days                         = 93
        expired_object_delete_marker = false
      }
      # test filter without prefix
      filter_prefix_only = null
      filter_and = {
        prefix                   = ""
        object_size_greater_than = 256 * 1024
        tags                     = null
      }
      id = "big"
    }
  ]

  lc_rules_map = {
    base    = [local.lc_rules_list[0], local.lc_rules_list[1], local.lc_rules_list[3], local.lc_rules_list[4]]
    trigger = local.lc_rules_list
  }

  lc_rules = local.lc_rules_map[var.trigger ? "trigger" : "base"]
}

resource "aws_s3_bucket_lifecycle_configuration" "default" {
  bucket = aws_s3_bucket.this.id

  dynamic "rule" {
    for_each = local.lc_rules
    content {
      id     = rule.value.id
      status = rule.value.enabled == true ? "Enabled" : "Disabled"

      dynamic "expiration" {
        for_each = rule.value.expiration == null ? [] : [rule.value.expiration]
        content {
          days                         = expiration.value.days
          expired_object_delete_marker = expiration.value.expired_object_delete_marker
        }
      }

      # Filter is always required due to https://github.com/hashicorp/terraform-provider-aws/issues/23299
      dynamic "filter" {
        for_each = rule.value.filter_prefix_only == null && rule.value.filter_and == null ? ["empty"] : []
        content {}
      }

      dynamic "filter" {
        for_each = rule.value.filter_prefix_only == null ? [] : ["prefix"]
        content {
          prefix = rule.value.filter_prefix_only.prefix
        }
      }

      dynamic "filter" {
        for_each = rule.value.filter_and == null ? [] : ["and"]
        content {
          and {
            object_size_greater_than = try(rule.value.filter_and.object_size_greater_than, null)
            object_size_less_than    = try(rule.value.filter_and.object_size_less_than, null)
            prefix                   = rule.value.filter_and.prefix
            tags                     = try(rule.value.filter_and.tags, null)
          }
        }
      }
    }
  }
}

The issue appears to be that when the new list of rules doesn't line up with the old list of rules, and a rule goes from having

filter {
  and {
    prefix = "prefix2"
    object_size_greater_than = 131072
  }
}

to

filter {
  prefix = "prefix1"
}

the provider gets confused.

Debug Output

Debug output contains sensitive information like API keys. How do I redact it?

Terraform `plan` output

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # aws_s3_bucket_lifecycle_configuration.default has changed
  ~ resource "aws_s3_bucket_lifecycle_configuration" "default" {
        id     = "eg-test-aws-issue-23883"
        # (1 unchanged attribute hidden)

      ~ rule {
            id     = "rule-2"
            # (1 unchanged attribute hidden)


          ~ filter {
            }
            # (1 unchanged block hidden)
        }
      ~ rule {
          ~ id     = "nofilter" -> "prefix1"
            # (1 unchanged attribute hidden)


          ~ filter {
              + prefix = "prefix1"
            }
            # (1 unchanged block hidden)
        }
      ~ rule {
          ~ id     = "prefix1" -> "prefix2"
            # (1 unchanged attribute hidden)

          ~ expiration {
              ~ days                         = 0 -> 1461
                # (1 unchanged attribute hidden)
            }

          ~ filter {
              - prefix = "prefix1" -> null

              ~ and {
                  ~ object_size_greater_than = 0 -> 131072
                  + prefix                   = "prefix2"
                    tags                     = {}
                    # (1 unchanged attribute hidden)
                }
            }
        }
      - rule {
          - id     = "prefix2" -> null
          - status = "Enabled" -> null

          - expiration {
              - days                         = 1461 -> null
              - expired_object_delete_marker = false -> null
            }

          - filter {
              - and {
                  - object_size_greater_than = 131072 -> null
                  - object_size_less_than    = 0 -> null
                  - prefix                   = "prefix2" -> null
                }
            }
        }
      - rule {
          - id     = "big" -> null
          - status = "Enabled" -> null

          - expiration {
              - days                         = 93 -> null
              - expired_object_delete_marker = false -> null
            }

          - filter {
              - and {
                  - object_size_greater_than = 262144 -> null
                  - object_size_less_than    = 0 -> null
                }
            }
        }
        # (1 unchanged block hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may
include actions to undo or respond to these changes.

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_s3_bucket_lifecycle_configuration.default will be updated in-place
  ~ resource "aws_s3_bucket_lifecycle_configuration" "default" {
        id     = "eg-test-aws-issue-23883"
        # (1 unchanged attribute hidden)

      ~ rule {
          ~ id     = "prefix1" -> "nofilter"
            # (1 unchanged attribute hidden)


          ~ filter {
              - prefix = "prefix1" -> null
            }
            # (1 unchanged block hidden)
        }
      ~ rule {
          ~ id     = "prefix2" -> "prefix1"
            # (1 unchanged attribute hidden)

          ~ expiration {
              ~ days                         = 1461 -> 0
                # (1 unchanged attribute hidden)
            }

          ~ filter {
              + prefix = "prefix1"

              ~ and {
                  - object_size_greater_than = 131072 -> null
                  - prefix                   = "prefix2" -> null
                    tags                     = {}
                    # (1 unchanged attribute hidden)
                }
            }
        }
      + rule {
          + id     = "prefix2"
          + status = "Enabled"

          + expiration {
              + days                         = 1461
              + expired_object_delete_marker = false
            }

          + filter {
              + and {
                  + object_size_greater_than = 131072
                  + prefix                   = "prefix2"
                }
            }
        }
      + rule {
          + id     = "big"
          + status = "Enabled"

          + expiration {
              + days                         = 93
              + expired_object_delete_marker = false
            }

          + filter {
              + and {
                  + object_size_greater_than = 262144
                }
            }
        }
        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Actual Behavior

On terraform apply

│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for module.s3_bucket.aws_s3_bucket_lifecycle_configuration.default[0] to include new values learned so far during apply,
│ provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .rule[1].filter[0].object_size_greater_than: was null, but now
│ cty.StringVal("").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Note that the same error message repeats but with object_size_less_than and prefix attributes.

Steps to Reproduce

1. terraform init
2. terraform apply -auto-approve
3. terraform apply -auto-approve -var trigger=true # Triggers r/s3_bucket_lifecycle_configuration: Rule change generates MalformedXML #23884 MalformedXML, fails with error
4. terraform apply -auto-approve -var trigger=true # Triggers this issue

References

This complex setup of the filter configuration is made necessary by:

• Bad error (message?) aws_s3_bucket_lifecycle_configuration complains about "Base level prefix" when there is none #23299
• r/s3_bucket_lifecycle_configuration: providing filter "and" clause with only "prefix" causes MalformedXML error #23882

[anGie44]

Hi @Nuru , I've tried reproducing this error on update of the aws_s3_bucket_lifecycle_configuration resource but haven't had luck just yet with a similar example. Are you by any chance running terraform apply after upgrading your provider version to v4.8.0 or after importing the resources into terraform state?

The example i've tried so far only hits the MalformedXML error described in #23884 .

1. I've first applied the following config with v4.8.0 of the provider.

resource "aws_s3_bucket" "this" {
  bucket = "tf-test-abc-20220317"
}

locals {
  lc_rules = [
    {
      abort_incomplete_multipart_upload_days = 1
      enabled                                = true
      expiration = {
        days                         = null
        expired_object_delete_marker = null
      }
      filter_prefix_only = {
        prefix = "prefix1"
      }
      filter_and = null
      id         = "prefix1"
      noncurrent_version_expiration = {
        newer_noncurrent_versions = 2
        noncurrent_days           = 30
      }
      noncurrent_version_transition = []
      transition = [
        {
          days          = 7
          storage_class = "GLACIER"
        },
      ]
    },
    {
      abort_incomplete_multipart_upload_days = 1
      enabled                                = true
      expiration = {
        days                         = 93
        expired_object_delete_marker = false
      }

      filter_prefix_only = null

      filter_and = {
        prefix                   = "prefix2"
        object_size_greater_than = 131072
      }

      id = "prefix2"
      noncurrent_version_expiration = {
        newer_noncurrent_versions = 2
        noncurrent_days           = 14
      }
      noncurrent_version_transition = []
      transition = [
        {
          days          = 3
          storage_class = "GLACIER"
        },
      ]
    }
  ]
}

resource "aws_s3_bucket_lifecycle_configuration" "default" {
  bucket = aws_s3_bucket.this.id

  dynamic "rule" {
    for_each = local.lc_rules
    content {
      id     = rule.value.id
      status = rule.value.enabled == true ? "Enabled" : "Disabled"

      expiration {
        days = 90
      }

      # Filter is always required due to https://github.com/hashicorp/terraform-provider-aws/issues/23299
      dynamic "filter" {
        for_each = rule.value.filter_prefix_only == null && rule.value.filter_and == null ? ["empty"] : []
        content {}
      }

      dynamic "filter" {
        for_each = rule.value.filter_prefix_only == null ? [] : ["prefix"]
        content {
          prefix = rule.value.filter_prefix_only.prefix
        }
      }

      dynamic "filter" {
        for_each = rule.value.filter_and == null ? [] : ["and"]
        content {
          and {
            object_size_greater_than = try(rule.value.filter_and.object_size_greater_than, null)
            object_size_less_than    = try(rule.value.filter_and.object_size_less_than, null)
            prefix                   = rule.value.filter_and.prefix
            tags                     = try(rule.value.filter_and.tags, null)
          }
        }
      }
    }
  }
}

2. Then updated the local to:

  lc_rules = [
    {
      abort_incomplete_multipart_upload_days = 1
      enabled                                = true
      expiration = {
        days                         = null
        expired_object_delete_marker = null
      }
      filter_prefix_only = null
      filter_and         = null
      id                 = "nofilter"
      noncurrent_version_expiration = {
        newer_noncurrent_versions = 2
        noncurrent_days           = 30
      }
      noncurrent_version_transition = []
      transition = [
        {
          days          = 7
          storage_class = "GLACIER"
        },
      ]
    },
    {
      abort_incomplete_multipart_upload_days = 1
      enabled                                = true
      expiration = {
        days                         = 93
        expired_object_delete_marker = false
      }

      filter_prefix_only = {
        prefix = "prefix1"
      }

      filter_and = null

      id = "prefix1"
      noncurrent_version_expiration = {
        newer_noncurrent_versions = 2
        noncurrent_days           = 14
      }
      noncurrent_version_transition = []
      transition = [
        {
          days          = 3
          storage_class = "GLACIER"
        },
      ]
    }
  ]
}

which produced a similar update:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_s3_bucket_lifecycle_configuration.default will be updated in-place
  ~ resource "aws_s3_bucket_lifecycle_configuration" "default" {
        id     = "tf-test-abc-20220317"
        # (1 unchanged attribute hidden)

      ~ rule {
          ~ id     = "prefix1" -> "nofilter"
            # (1 unchanged attribute hidden)


          ~ filter {
              - prefix = "prefix1" -> null
            }
            # (1 unchanged block hidden)
        }
      ~ rule {
          ~ id     = "prefix2" -> "prefix1"
            # (1 unchanged attribute hidden)


          ~ filter {
              + prefix = "prefix1"

              ~ and {
                  - object_size_greater_than = 131072 -> null
                  - prefix                   = "prefix2" -> null
                    tags                     = {}
                    # (1 unchanged attribute hidden)
                }
            }
            # (1 unchanged block hidden)
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.
aws_s3_bucket_lifecycle_configuration.default: Modifying... [id=tf-test-abc-20220317]
╷
│ Error: error updating S3 Bucket Lifecycle Configuration (tf-test-abc-20220317): MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
│ 	status code: 400, request id: A22D07RDD6F5STNG, host id: SN9nviwFSksLIFFkpUI1oCPpWET8MLaMPmhR16rl+1ZmLiV8L2qPzZb6r4mfpgoOQ0XUqmSjzAw=
│
│   with aws_s3_bucket_lifecycle_configuration.default,
│   on main.tf line 84, in resource "aws_s3_bucket_lifecycle_configuration" "default":
│   84: resource "aws_s3_bucket_lifecycle_configuration" "default" {
│
╵

[Nuru]

@anGie44 I updated the issue description with a complete configuration. Note that in my testing, the first time I try to run with trigger=true I get the MalformedXML error, but if I then immediately run it again, I get this provider error.

[anGie44]

Thanks @Nuru , I'm able to reproduce with those set of instructions. Good news is that if I build the provider with the fix in #23893, I don't hit the MalformedXML error nor the Provider produced inconsistent final plan error as it's possible the latter was also a side-effect of the DiffSuppressFunc previously used.

[anGie44]

Hi @Nuru , just noting here that with the fix for #23884 merged, it may be safe to assume this issue will no longer appear once it's released in v4.9.0 of the AWS provider (likely out next week) but if you could confirm once you upgrade, it would be greatly appreciated!

[Nuru]

I can no longer reproduce this bug now that v4.10.0 is out. Probably fixed in v4.9.0. Thank you.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.