aws_s3control_bucket_lifecycle_configuration throws "unknown format"

[jcreyf]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

$ terraform -v
Terraform v0.14.7
+ provider registry.terraform.io/hashicorp/aws v3.25.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

Affected Resource(s)

• aws_s3control_bucket_lifecycle_configuration

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_s3control_bucket_lifecycle_configuration" "log_retention" {
  bucket = "arn:aws:s3:::fusion-nonprd"
  rule {
    status = "Enabled"
    id     = "log_retention"
    filter {
      prefix = "logs/"
    }
    expiration {
      days = 7
    }
  }
}

Debug Output

Panic Output

Expected Behavior

This should create a new S3 Bucket Lifecycle Rule and configure it with a filter and number of days for objects to expire in the configured S3 Bucket / Prefix.

Actual Behavior

Fails to apply with error: error parsing S3 Control Bucket ARN (): unknown format

$ terraform plan
..snip..
  # aws_s3control_bucket_lifecycle_configuration.alb_external_log_retention will be created
  + resource "aws_s3control_bucket_lifecycle_configuration" "alb_external_log_retention" {
      + bucket = "arn:aws:s3:::fusion-nonprd"
      + id     = (known after apply)
      + rule {
          + id     = "log_retention"
          + status = "Enabled"

          + expiration {
              + days                         = 7
              + expired_object_delete_marker = false
            }

          + filter {
              + prefix = "logs/"
            }
        }
    }
..snip..
Plan: 1 to add, 0 to change, 0 to destroy.

$ terraform apply
..snip..
aws_s3control_bucket_lifecycle_configuration.log_retention: Creating...
Error: error parsing S3 Control Bucket ARN (): unknown format
..snip..

Steps to Reproduce

The provided Terraform Configuration File in this ticket is pretty straight forward and stands on its own.
Just replace my S3 Bucket arn with another arn that you have access to and set the filter prefix to some location in the bucket.

Important Factoids

• The error is thrown here (when the AccountID in the parsed S3 ARN is empty):
  aws/resource_aws_s3control_bucket_policy.go
  

  terraform-provider-aws/aws/resource_aws_s3control_bucket_policy.go

  Line 74 in 1194e7a

  return fmt.Errorf("error parsing S3 Control Bucket ARN (%s): unknown format", d.Id())

67: 	parsedArn, err := arn.Parse(d.Id())
...
73:	if parsedArn.AccountID == "" {
74:		return fmt.Errorf("error parsing S3 Control Bucket ARN (%s): unknown format", d.Id())
75:	}

S3 resources are global though and don't need accountIDs.
I tried by adding the accountID to the S3 ARN anyways and am then getting a different exception thrown in Terraform Apply:

aws_s3control_bucket_lifecycle_configuration.alb_external_log_retention: Creating...
Error: error creating S3 Control Lifecycle Configuration (arn:aws:s3::289036058743:fusion-nonprd): error while fetching endpoint ARN: invalid Amazon s3 ARN, unknown resource type, arn:aws:s3::289036058743:fusion-nonprd

• Resource type aws_s3control_bucket_lifecycle_configuration was added to terraform-provider-aws on Oct 27, 2020 and released for the first time in version v3.13.0 (Oct 29, 2020):
  https://github.com/hashicorp/terraform-provider-aws/releases/tag/v3.13.0

  FEATURES
    New Resource: aws_s3control_bucket (#15510)
    New Resource: aws_s3control_bucket_lifecycle_configuration (#15604)

The most current version of terraform-provider-aws at this point in time is v3.59.0 and no code changes have been made to aws/resource_aws_s3control_bucket_policy.go since it was first introduced. So upgrading the provider on my end to a newer version is not going to resolve my problem.

References

• the suspect block of code:
  

  terraform-provider-aws/aws/resource_aws_s3control_bucket_policy.go

  Line 74 in 1194e7a

  return fmt.Errorf("error parsing S3 Control Bucket ARN (%s): unknown format", d.Id())

• release doc when this resouce type was first introduced:
  https://github.com/hashicorp/terraform-provider-aws/releases/tag/v3.13.0

[anGie44]

Hi @jcreyf , thank you for raising this issue. Looking at the bucket arn being provided as input, arn:aws:s3:::fusion-nonprd, I believe the resource you want to use instead is s3_bucket_policy since the aws_s3control_bucket_policy resource is intended for buckets on Outposts. Though the error mentioned above Error: error creating S3 Control Lifecycle Configuration refers to a different resource which is also only intended for S3 buckets on Outposts. Do you mind confirming which type of S3 bucket you'd like to create a policy/lifecycle configuration for?

[jcreyf]

Hi @anGie44 , you are absolutely correct! Turns out I'm not trying to set a lifecycle on an Outpost S3 Bucket but on regular S3 Buckets.
I've done some more digging and turns out there's no independent resource type available yet like aws_s3control_bucket_lifecycle_configuration for regular S3 Buckets. I will have to manage my S3 Bucket Lifecycle rules in my aws_s3_bucket resource code where I create and manage the actual bucket.
This is going to make my life quite a pain to maintain, but guess there's no other way for now.
Turns out issue #6188 is what I need (created in 2018 and got added to the v4.0.0 milestone list earlier this year).
So guess we can close this issue and I'll have to revisit my hacks after resource aws_s3_bucket_lifecycle_configuration becomes available at some point.
Thank you very much for your help!!!

[jcreyf]

What I need is part of the aws_s3_bucket refactor that is planned for version 4.0 (#20433)

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.