provider/aws: Add support S3 Object Lifecycle Rule #6220

kjmkznr · 2016-04-18T15:51:30Z

This pull request adds S3 object lifecycle management configuration.

resource "aws_s3_bucket" "bucket" {
    bucket = "my-bucket"
    acl = "private"

    lifecycle_rule {
        id = "log"
        prefix = "log/"
        enabled = true

        transition {
            days = 30
            storage_class = "STANDARD_IA"
        }
        transition {
            days = 60
            storage_class = "GLACIER"
        }
        expiration {
            days = 90
        }
    }
    lifecycle_rule {
        id = "tmp"
        prefix = "tmp/"
        enabled = true

        expiration {
            date = "2016-01-12"
        }
    }
}

Acceptance test

$ make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSS3Bucket_Lifecycle' 
==> Checking that code complies with gofmt requirements...
/home/kjmkznr/go/bin/stringer
go generate $(go list ./... | grep -v /vendor/)
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSS3Bucket_Lifecycle -timeout 120m
=== RUN   TestAccAWSS3Bucket_Lifecycle
--- PASS: TestAccAWSS3Bucket_Lifecycle (23.39s)
PASS
ok      github.com/hashicorp/terraform/builtin/providers/aws    23.407s
$ make test TEST=./builtin/providers/aws 
==> Checking that code complies with gofmt requirements...
/home/kjmkznr/go/bin/stringer
go generate $(go list ./... | grep -v /vendor/)
TF_ACC= go test ./builtin/providers/aws  -timeout=30s -parallel=4
ok      github.com/hashicorp/terraform/builtin/providers/aws    0.354s

kjmkznr · 2016-04-18T16:02:56Z

Sorry, I forgot to check for the vet command.
I will fix soon.

ajlanghorn · 2016-04-18T16:12:34Z

<3

stack72 · 2016-04-18T16:45:49Z

builtin/providers/aws/resource_aws_s3_bucket.go

+	if len(lifecycle.Rules) > 0 {
+		rules := make([]map[string]interface{}, 0, len(lifecycle.Rules))
+
+		for _, lifecycleRule := range lifecycle.Rules {


Is there a chance any of these dereferenced properties may not exist? If so then, when dereferencing, we will get a panic

These should basically exists in response, but I added check code just in case.

stack72 · 2016-04-18T16:51:25Z

just a quick point here - when I was building the Azure provider - 3 levels of nesting sometimes gave some interesting results. So we need to keep an eye on this :)

aviflax · 2016-04-18T17:06:57Z

Semantics look fantastic! Thank you!

kjmkznr · 2016-04-19T14:33:02Z

@stack72
Thanks reviews!
I fixed as was pointed out in comment.

just a quick point here - when I was building the Azure provider - 3 levels of nesting sometimes gave some interesting results. So we need to keep an eye on this :)

I'm sorry, I don't understand.
Do I need to fix something?
What is interest results?

stack72 · 2016-04-19T18:54:04Z

Hi @kjmkznr

Ignore my comments about the nesting - it's not an issue. BUT when i run all of the S3 bucket tests, I now see the following:

make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSS3Bucket_' 2>~/tf.log
==> Checking that code complies with gofmt requirements...
/Users/stacko/Code/go/bin/stringer
go generate $(go list ./... | grep -v /vendor/)
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSS3Bucket_ -timeout 120m
=== RUN   TestAccAWSS3Bucket_Notification
--- FAIL: TestAccAWSS3Bucket_Notification (22.63s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 796BA42E16CABD2F
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 6DC4F0C56A308207

        State: <nil>
=== RUN   TestAccAWSS3Bucket_NotificationWithoutFilter
--- FAIL: TestAccAWSS3Bucket_NotificationWithoutFilter (20.72s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 8A803B0235F45AF4
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 54D63AD66FFB82D5

        State: <nil>
=== RUN   TestAccAWSS3Bucket_basic
--- FAIL: TestAccAWSS3Bucket_basic (21.54s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 532767AA9682C193
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: E6FB35C32C931C5E

        State: <nil>
=== RUN   TestAccAWSS3Bucket_Policy
--- FAIL: TestAccAWSS3Bucket_Policy (22.87s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: B61BEAA467EE28F4
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 403B3CDABE43A28F

        State: <nil>
=== RUN   TestAccAWSS3Bucket_UpdateAcl
--- FAIL: TestAccAWSS3Bucket_UpdateAcl (22.04s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 1332CA491E3AE1B2
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: FA02515E8AE637EE

        State: <nil>
=== RUN   TestAccAWSS3Bucket_Website_Simple
--- FAIL: TestAccAWSS3Bucket_Website_Simple (24.81s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 7E25E5BDBD9F48ED
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 630E9F25BC785915

        State: <nil>
=== RUN   TestAccAWSS3Bucket_WebsiteRedirect
--- FAIL: TestAccAWSS3Bucket_WebsiteRedirect (28.52s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 2F93706E561BAC07
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 26FE97469B0AC003

        State: <nil>
=== RUN   TestAccAWSS3Bucket_WebsiteRoutingRules
--- FAIL: TestAccAWSS3Bucket_WebsiteRoutingRules (23.24s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 981441D2C0C9EA0F
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 6FB2653631814E1B

        State: <nil>
=== RUN   TestAccAWSS3Bucket_shouldFailNotFound
--- FAIL: TestAccAWSS3Bucket_shouldFailNotFound (24.82s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 1C2DAC5C96A068BA
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: F558276F6C15994E

        State: <nil>
=== RUN   TestAccAWSS3Bucket_Versioning
--- FAIL: TestAccAWSS3Bucket_Versioning (21.13s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 8DFE8967311791B7
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: F27FC80250CBA25F

        State: <nil>
=== RUN   TestAccAWSS3Bucket_Cors
--- FAIL: TestAccAWSS3Bucket_Cors (26.24s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: C6E17788D25E2A69
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 14CEA0E8FFAAD3F9

        State: <nil>
=== RUN   TestAccAWSS3Bucket_Logging
--- FAIL: TestAccAWSS3Bucket_Logging (23.07s)
    testing.go:154: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_s3_bucket.log_bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: 6BD0B8723977C63B
    testing.go:175: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Error refreshing: 1 error(s) occurred:

        * aws_s3_bucket.log_bucket: NoSuchLifecycleConfiguration: The lifecycle configuration does not exist
            status code: 404, request id: ED125DEFA295B715

        State: <nil>
=== RUN   TestAccAWSS3Bucket_Lifecycle
--- PASS: TestAccAWSS3Bucket_Lifecycle (92.03s)
FAIL
exit status 1
FAIL    github.com/hashicorp/terraform/builtin/providers/aws    373.675s

kjmkznr · 2016-04-20T00:40:02Z

@stack72

I forgat check other acceptance tests.
I fixed it.

stack72 · 2016-04-20T10:15:56Z

Hi @kjmkznr

this looks great now - thanks so much for the fix on the 404!

The new test run confirmed it worked as expected:

make testacc TEST=./builtin/providers/aws TESTARGS='-run=TestAccAWSS3Bucket_' 2>~/tf.log
==> Checking that code complies with gofmt requirements...
/Users/stacko/Code/go/bin/stringer
go generate $(go list ./... | grep -v /vendor/)
TF_ACC=1 go test ./builtin/providers/aws -v -run=TestAccAWSS3Bucket_ -timeout 120m
=== RUN   TestAccAWSS3Bucket_Notification
--- PASS: TestAccAWSS3Bucket_Notification (126.44s)
=== RUN   TestAccAWSS3Bucket_NotificationWithoutFilter
--- PASS: TestAccAWSS3Bucket_NotificationWithoutFilter (55.94s)
=== RUN   TestAccAWSS3Bucket_basic
--- PASS: TestAccAWSS3Bucket_basic (34.73s)
=== RUN   TestAccAWSS3Bucket_Policy
--- PASS: TestAccAWSS3Bucket_Policy (88.65s)
=== RUN   TestAccAWSS3Bucket_UpdateAcl
--- PASS: TestAccAWSS3Bucket_UpdateAcl (65.63s)
=== RUN   TestAccAWSS3Bucket_Website_Simple
--- PASS: TestAccAWSS3Bucket_Website_Simple (101.24s)
=== RUN   TestAccAWSS3Bucket_WebsiteRedirect
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (100.66s)
=== RUN   TestAccAWSS3Bucket_WebsiteRoutingRules
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (70.57s)
=== RUN   TestAccAWSS3Bucket_shouldFailNotFound
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (19.91s)
=== RUN   TestAccAWSS3Bucket_Versioning
--- PASS: TestAccAWSS3Bucket_Versioning (97.40s)
=== RUN   TestAccAWSS3Bucket_Cors
--- PASS: TestAccAWSS3Bucket_Cors (35.75s)
=== RUN   TestAccAWSS3Bucket_Logging
--- PASS: TestAccAWSS3Bucket_Logging (63.50s)
=== RUN   TestAccAWSS3Bucket_Lifecycle
--- PASS: TestAccAWSS3Bucket_Lifecycle (66.68s)
PASS
ok      github.com/hashicorp/terraform/builtin/providers/aws    918.631s

Thanks again :)

Paul

kjmkznr · 2016-04-20T17:00:02Z

Thanks!

ghost · 2020-04-25T02:12:06Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

providers/aws: Add support S3 Object Lifecycle Rule

73e3817

radeksimko added enhancement provider/aws labels Apr 18, 2016

radeksimko mentioned this pull request Apr 18, 2016

AWS provider: aws_s3_bucket resource; missing versioning, lifecycle, logging, etc #3108

Closed

kjmkznr added 2 commits April 19, 2016 01:10

Fix failed vet command

df19f5e

Fix failed acceptance tests

55d3f46

stack72 reviewed Apr 18, 2016
View reviewed changes

kjmkznr added 2 commits April 19, 2016 10:55

Check nil pointer before dereference.

b1d9efe

Move S3 lifecycle rule id validator func to validators.go

1c87ef9

stack72 self-assigned this Apr 19, 2016

stack72 added the waiting-response An issue/pull request is waiting for a response from the community label Apr 19, 2016

Don't fail when get lifecycle rule's response code is 404

00ae47f

stack72 merged commit 5e33517 into hashicorp:master Apr 20, 2016

kjmkznr deleted the aws-s3-lifecycle branch June 7, 2016 12:27

ghost locked and limited conversation to collaborators Apr 25, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

provider/aws: Add support S3 Object Lifecycle Rule #6220

provider/aws: Add support S3 Object Lifecycle Rule #6220

kjmkznr commented Apr 18, 2016 •

edited by radeksimko

Loading

kjmkznr commented Apr 18, 2016

ajlanghorn commented Apr 18, 2016

stack72 Apr 18, 2016

kjmkznr Apr 19, 2016

stack72 commented Apr 18, 2016

aviflax commented Apr 18, 2016 •

edited

Loading

kjmkznr commented Apr 19, 2016

stack72 commented Apr 19, 2016

kjmkznr commented Apr 20, 2016

stack72 commented Apr 20, 2016

kjmkznr commented Apr 20, 2016

ghost commented Apr 25, 2020

provider/aws: Add support S3 Object Lifecycle Rule #6220

provider/aws: Add support S3 Object Lifecycle Rule #6220

Conversation

kjmkznr commented Apr 18, 2016 • edited by radeksimko Loading

Acceptance test

kjmkznr commented Apr 18, 2016

ajlanghorn commented Apr 18, 2016

stack72 Apr 18, 2016

Choose a reason for hiding this comment

kjmkznr Apr 19, 2016

Choose a reason for hiding this comment

stack72 commented Apr 18, 2016

aviflax commented Apr 18, 2016 • edited Loading

kjmkznr commented Apr 19, 2016

stack72 commented Apr 19, 2016

kjmkznr commented Apr 20, 2016

stack72 commented Apr 20, 2016

kjmkznr commented Apr 20, 2016

ghost commented Apr 25, 2020

kjmkznr commented Apr 18, 2016 •

edited by radeksimko

Loading

aviflax commented Apr 18, 2016 •

edited

Loading