Encrypt the bucket and log access on the AWS S3 Bucket of the TFState #645

Xtigyro · 2019-02-03T10:55:46Z

Encrypt the bucket and log the access on the AWS S3 Bucket of the Terraform State - both bucket-wide.

brikis98

This looks great, thank you!

Two questions:

Could you update the README to indicate that encryption and access logging will be turned on by default and can be enabled with the new skip_xxx params?
How did you test these changes?

brikis98 · 2019-02-03T13:06:50Z

remote/remote_state_s3.go

+		Bucket:                  aws.String(config.Bucket),
+		BucketLoggingStatus: &s3.BucketLoggingStatus{
+		LoggingEnabled: &s3.LoggingEnabled{
+			TargetBucket: aws.String(config.Bucket),


Are you writing the access logs back to the same bucket? If so, may be worth mentioning that in the log statement.

Yes, in the same bucket. OK - will do.

@brikis98 Done.

brikis98 · 2019-02-03T13:07:20Z

remote/remote_state_s3.go

+	DynamotableTags					[]map[string]string `mapstructure:"dynamodb_table_tags"`
+	SkipBucketVersioning		bool                `mapstructure:"skip_bucket_versioning"`
+	SkipBucketSSEncryption	bool                `mapstructure:"skip_bucket_ssencryption"`
+	SkipBucketAccessLogging bool                `mapstructure:"skip_bucket_accesslogging"`


Is this just GitHub rendering or is the indentation not quite right here? Try running make fmt if you haven't already.

Weird - I use "Atom" v1.34.0 on Win10Home - looks nicely there. Installed MS VSC (latest) - modified it there, looks nicely - but again it's not fine in GitHub.
Will try make fmt

@brikis98 Should be fixed.
Used the - https://github.com/Microsoft/vscode-go/wiki/Go-tools-that-the-Go-extension-depends-on - for correct formatting in MS VSC 1.30.2 (latest).

goreturns or goimports for formatting code

On GitHub it still looks wrong though.

Tried using GitHub and then paste the code from there - but the Go tools revert it back locally to the currently uploaded code.

Try making whitespace visible in your editor? In vscode, View -> Toggle Render Whitespace

Fixed it remotely on GitHub already.
Thank you for the nice advice though - it'll be useful in the future.

Nah - reverting - I'll use the formatting from the Go Tools. Checked the whitespaces as well - it should be fine when reverted.

It doesn't sound right to use GitHub remotely in a browser to fix such formatting issues.

The funny part - after reverting it looks fine on GitHub as well... #GitHubSaySorryPlease

Xtigyro · 2019-02-03T15:52:16Z

This looks great, thank you!

Two questions:

1. Could you update the README to indicate that encryption and access logging will be turned on by default and can be enabled with the new `skip_xxx` params?

2. How did you test these changes?

(1) OK - will do.
(2) I'm not familiar with the way a PR should be done in GoLang - thought that the maintainers do the testing. Could you remind me please where the procedure was for testing your code within Terragrunt?

brikis98 · 2019-02-03T18:25:21Z

https://circleci.com/docs/2.0/oss/#pass-secrets-to-builds-from-forked-pull-requests

brikis98 · 2019-02-03T18:27:03Z

Whoops, hit enter too soon! What I was trying to write:

I'm not familiar with the way a PR should be done in GoLang - thought that the maintainers do the testing. Is there a procedure for testing your code within Terragrunt?

Since Terragrunt talks to AWS, and our CI builds have credentials for our AWS account, we can't automatically run tests on external PRs: https://circleci.com/docs/2.0/oss/#pass-secrets-to-builds-from-forked-pull-requests

So, we ask contributors to run tests themselves. You can test it manually and/or automatically. Once I merge, all the CI tests will run, but I want to have some sense of confidence so I don't merge broken code into master.

[root@localhost remote]# go test -v -parallel 128 # _/root/go-project/terragrunt/remote [_/root/go-project/terragrunt/remote.test] ./remote_state_s3.go:449:4: unknown field 'ServerSideEncryption' in struct literal of type s3.ServerSideEncryptionConfiguration ./remote_state_s3.go:450:4: unknown field 'SSEKMSKeyId' in struct literal of type s3.ServerSideEncryptionConfiguration FAIL _/root/go-project/terragrunt/remote [build failed]

[root@localhost remote]# go test -v -parallel 128 # _/root/go-project/terragrunt/remote ./remote_state_s3.go:459: Logger.Printf format %s reads arg #2, but call has 1 arg FAIL _/root/go-project/terragrunt/remote [build failed]

Xtigyro · 2019-02-03T21:42:04Z

Whoops, hit enter too soon! What I was trying to write:

I'm not familiar with the way a PR should be done in GoLang - thought that the maintainers do the testing. Is there a procedure for testing your code within Terragrunt?

Since Terragrunt talks to AWS, and our CI builds have credentials for our AWS account, we can't automatically run tests on external PRs: https://circleci.com/docs/2.0/oss/#pass-secrets-to-builds-from-forked-pull-requests

So, we ask contributors to run tests themselves. You can test it manually and/or automatically. Once I merge, all the CI tests will run, but I want to have some sense of confidence so I don't merge broken code into master.

@brikis98 Tested with:

[root@localhost go-project]# cat /usr/local/bin/re-go.sh
#!/usr/bin/env bash

rm -rf /root/go-project/terragrunt/
cp -r /root/shares/GitHub/Xtigyro/terragrunt/ /root/go-project/
cd /root/go-project/terragrunt/remote/
go test -v -parallel 128

[root@localhost go-project]# re-go.sh
=== RUN   TestConfigValuesEqual
=== PAUSE TestConfigValuesEqual
=== RUN   TestForcePathStyleClientSession
=== PAUSE TestForcePathStyleClientSession
=== RUN   TestGetAwsSessionConfig
=== PAUSE TestGetAwsSessionConfig
=== RUN   TestToTerraformInitArgs
=== PAUSE TestToTerraformInitArgs
=== RUN   TestToTerraformInitArgsUnknownBackend
=== PAUSE TestToTerraformInitArgsUnknownBackend
=== RUN   TestToTerraformInitArgsNoBackendConfigs
=== PAUSE TestToTerraformInitArgsNoBackendConfigs
=== RUN   TestDiffersFrom
=== PAUSE TestDiffersFrom
=== RUN   TestParseTerraformStateLocal
=== PAUSE TestParseTerraformStateLocal
=== RUN   TestParseTerraformStateRemote
=== PAUSE TestParseTerraformStateRemote
=== RUN   TestParseTerraformStateRemoteFull
=== PAUSE TestParseTerraformStateRemoteFull
=== RUN   TestParseTerraformStateEmpty
=== PAUSE TestParseTerraformStateEmpty
=== RUN   TestParseTerraformStateInvalid
=== PAUSE TestParseTerraformStateInvalid
=== CONT  TestConfigValuesEqual
=== CONT  TestParseTerraformStateLocal
=== CONT  TestParseTerraformStateInvalid
--- PASS: TestParseTerraformStateInvalid (0.00s)
=== RUN   TestConfigValuesEqual/equal-both-empty
=== PAUSE TestConfigValuesEqual/equal-both-empty
=== RUN   TestConfigValuesEqual/equal-empty-and-nil
=== PAUSE TestConfigValuesEqual/equal-empty-and-nil
=== RUN   TestConfigValuesEqual/equal-empty-and-nil-backend-config
=== PAUSE TestConfigValuesEqual/equal-empty-and-nil-backend-config
=== RUN   TestConfigValuesEqual/equal-one-key
=== PAUSE TestConfigValuesEqual/equal-one-key
=== RUN   TestConfigValuesEqual/equal-multiple-keys
=== PAUSE TestConfigValuesEqual/equal-multiple-keys
=== RUN   TestConfigValuesEqual/equal-encrypt-bool-handling
=== PAUSE TestConfigValuesEqual/equal-encrypt-bool-handling
=== RUN   TestConfigValuesEqual/equal-general-bool-handling
=== PAUSE TestConfigValuesEqual/equal-general-bool-handling
=== RUN   TestConfigValuesEqual/equal-ignore-s3-tags
=== PAUSE TestConfigValuesEqual/equal-ignore-s3-tags
=== RUN   TestConfigValuesEqual/equal-ignore-dynamodb-tags
=== PAUSE TestConfigValuesEqual/equal-ignore-dynamodb-tags
=== RUN   TestConfigValuesEqual/unequal-wrong-backend
=== PAUSE TestConfigValuesEqual/unequal-wrong-backend
=== RUN   TestConfigValuesEqual/unequal-values
=== CONT  TestParseTerraformStateRemoteFull
--- PASS: TestParseTerraformStateRemoteFull (0.00s)
--- PASS: TestParseTerraformStateLocal (0.00s)
=== PAUSE TestConfigValuesEqual/unequal-values
=== RUN   TestConfigValuesEqual/unequal-non-empty-config-nil
=== PAUSE TestConfigValuesEqual/unequal-non-empty-config-nil
=== RUN   TestConfigValuesEqual/unequal-general-bool-handling
=== PAUSE TestConfigValuesEqual/unequal-general-bool-handling
=== CONT  TestConfigValuesEqual/equal-both-empty
=== CONT  TestConfigValuesEqual/equal-ignore-s3-tags
=== CONT  TestConfigValuesEqual/equal-general-bool-handling
=== CONT  TestConfigValuesEqual/equal-encrypt-bool-handling
=== CONT  TestConfigValuesEqual/unequal-general-bool-handling
=== CONT  TestConfigValuesEqual/unequal-values
=== CONT  TestToTerraformInitArgsNoBackendConfigs
--- PASS: TestToTerraformInitArgsNoBackendConfigs (0.00s)
=== CONT  TestToTerraformInitArgsUnknownBackend
[terragrunt] 2019/02/03 23:34:40 Backend config has changed from map[foo:different] to map[foo:bar]
--- PASS: TestToTerraformInitArgsUnknownBackend (0.00s)
[terragrunt] 2019/02/03 23:34:40 Backend config has changed from map[something:%!s(bool=false)] to map[something:%!s(bool=true)]
=== CONT  TestGetAwsSessionConfig
=== CONT  TestConfigValuesEqual/unequal-non-empty-config-nil
=== CONT  TestConfigValuesEqual/equal-ignore-dynamodb-tags
=== RUN   TestGetAwsSessionConfig/all-values
=== PAUSE TestGetAwsSessionConfig/all-values
=== CONT  TestConfigValuesEqual/unequal-wrong-backend
=== RUN   TestGetAwsSessionConfig/no-values
=== PAUSE TestGetAwsSessionConfig/no-values
[terragrunt] 2019/02/03 23:34:40 Backend type has changed from s3 to wrong
=== RUN   TestGetAwsSessionConfig/extra-values
=== PAUSE TestGetAwsSessionConfig/extra-values
=== CONT  TestToTerraformInitArgs
=== CONT  TestParseTerraformStateEmpty
=== CONT  TestDiffersFrom
[terragrunt] 2019/02/03 23:34:40 Backend  has not changed.
[terragrunt] 2019/02/03 23:34:40 Backend s3 has not changed.
[terragrunt] 2019/02/03 23:34:40 Backend type has changed from s3 to atlas
[terragrunt] 2019/02/03 23:34:40 Backend s3 has not changed.
[terragrunt] 2019/02/03 23:34:40 Backend config has changed from map[region:us-east-1 bucket:foo key:bar] to map[bucket:different key:bar region:us-east-1]
[terragrunt] 2019/02/03 23:34:40 Backend config has changed from map[bucket:foo key:bar region:us-east-1] to map[bucket:foo key:different region:us-east-1]
[terragrunt] 2019/02/03 23:34:40 Backend config has changed from map[bucket:foo key:bar region:us-east-1] to map[bucket:foo key:bar region:different]
[terragrunt] 2019/02/03 23:34:40 Backend config has changed from map[something:true] to map[something:%!s(bool=false)]
=== CONT  TestParseTerraformStateRemote
=== CONT  TestConfigValuesEqual/equal-multiple-keys
=== CONT  TestConfigValuesEqual/equal-one-key
=== CONT  TestConfigValuesEqual/equal-empty-and-nil-backend-config
=== CONT  TestConfigValuesEqual/equal-empty-and-nil
--- PASS: TestConfigValuesEqual (0.00s)
    --- PASS: TestConfigValuesEqual/equal-both-empty (0.00s)
    --- PASS: TestConfigValuesEqual/equal-general-bool-handling (0.00s)
    --- PASS: TestConfigValuesEqual/equal-ignore-s3-tags (0.00s)
    --- PASS: TestConfigValuesEqual/equal-encrypt-bool-handling (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-values (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-general-bool-handling (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-non-empty-config-nil (0.00s)
    --- PASS: TestConfigValuesEqual/equal-ignore-dynamodb-tags (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-wrong-backend (0.00s)
    --- PASS: TestConfigValuesEqual/equal-multiple-keys (0.00s)
    --- PASS: TestConfigValuesEqual/equal-one-key (0.00s)
    --- PASS: TestConfigValuesEqual/equal-empty-and-nil-backend-config (0.00s)
    --- PASS: TestConfigValuesEqual/equal-empty-and-nil (0.00s)
=== CONT  TestForcePathStyleClientSession
=== RUN   TestForcePathStyleClientSession/path-style-true
=== PAUSE TestForcePathStyleClientSession/path-style-true
=== RUN   TestForcePathStyleClientSession/path-style-false
=== PAUSE TestForcePathStyleClientSession/path-style-false
=== RUN   TestForcePathStyleClientSession/path-style-non-existent
=== PAUSE TestForcePathStyleClientSession/path-style-non-existent
=== CONT  TestForcePathStyleClientSession/path-style-true
=== CONT  TestForcePathStyleClientSession/path-style-non-existent
=== CONT  TestForcePathStyleClientSession/path-style-false
=== CONT  TestGetAwsSessionConfig/all-values
=== CONT  TestGetAwsSessionConfig/extra-values
=== CONT  TestGetAwsSessionConfig/no-values
--- PASS: TestGetAwsSessionConfig (0.00s)
    --- PASS: TestGetAwsSessionConfig/all-values (0.00s)
    --- PASS: TestGetAwsSessionConfig/extra-values (0.00s)
    --- PASS: TestGetAwsSessionConfig/no-values (0.00s)
--- PASS: TestToTerraformInitArgs (0.00s)
--- PASS: TestParseTerraformStateEmpty (0.00s)
--- PASS: TestDiffersFrom (0.00s)
--- PASS: TestParseTerraformStateRemote (0.00s)
--- FAIL: TestForcePathStyleClientSession (0.00s)
    --- FAIL: TestForcePathStyleClientSession/path-style-non-existent (0.28s)
        require.go:765:
                Error Trace:    remote_state_s3_test.go:154
                Error:          Expected nil, but got: &errors.Error{Err:(*awserr.baseError)(0xc00005a980), stack:[]uintptr{0x766e41, 0x79d339, 0x7a3eeb, 0x4e2c5f, 0x45d551}, frames:[]errors.StackFrame(nil), prefix:"Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?)"}
                Test:           TestForcePathStyleClientSession/path-style-non-existent
                Messages:       Unexpected error creating client for test: Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): NoCredentialProviders: no valid providers in chain. Deprecated.
                                        For verbose messaging see aws.Config.CredentialsChainVerboseErrors
    --- FAIL: TestForcePathStyleClientSession/path-style-true (0.31s)
        require.go:765:
                Error Trace:    remote_state_s3_test.go:154
                Error:          Expected nil, but got: &errors.Error{Err:(*awserr.baseError)(0xc00005a980), stack:[]uintptr{0x766e41, 0x79d339, 0x7a3eeb, 0x4e2c5f, 0x45d551}, frames:[]errors.StackFrame(nil), prefix:"Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?)"}
                Test:           TestForcePathStyleClientSession/path-style-true
                Messages:       Unexpected error creating client for test: Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): NoCredentialProviders: no valid providers in chain. Deprecated.
                                        For verbose messaging see aws.Config.CredentialsChainVerboseErrors
    --- FAIL: TestForcePathStyleClientSession/path-style-false (0.33s)
        require.go:765:
                Error Trace:    remote_state_s3_test.go:154
                Error:          Expected nil, but got: &errors.Error{Err:(*awserr.baseError)(0xc00005a980), stack:[]uintptr{0x766e41, 0x79d339, 0x7a3eeb, 0x4e2c5f, 0x45d551}, frames:[]errors.StackFrame(nil), prefix:"Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?)"}
                Test:           TestForcePathStyleClientSession/path-style-false
                Messages:       Unexpected error creating client for test: Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): NoCredentialProviders: no valid providers in chain. Deprecated.
                                        For verbose messaging see aws.Config.CredentialsChainVerboseErrors
FAIL
exit status 1
FAIL    _/root/go-project/terragrunt/remote     0.335s

Haven't tested it with real AWS account yet - fixed 3 bugs and it should be fine now.
Will try and test it through a real AWS account probably tomorrow evening after my work day is finished and get home.

Xtigyro · 2019-02-03T21:50:57Z

@brikis98 I guess I might be a geek... that couldn't resist the temptation - tested with an AWS account:

[root@localhost go-project]# cat /usr/local/bin/re-go.sh
#!/usr/bin/env bash

rm -rf /root/go-project/terragrunt/
cp -r /root/shares/GitHub/Xtigyro/terragrunt/ /root/go-project/
cd /root/go-project/terragrunt/remote/
go test -v -parallel 128

[root@localhost go-project]# re-go.sh
=== RUN   TestConfigValuesEqual
=== PAUSE TestConfigValuesEqual
=== RUN   TestForcePathStyleClientSession
=== PAUSE TestForcePathStyleClientSession
=== RUN   TestGetAwsSessionConfig
=== PAUSE TestGetAwsSessionConfig
=== RUN   TestToTerraformInitArgs
=== PAUSE TestToTerraformInitArgs
=== RUN   TestToTerraformInitArgsUnknownBackend
=== PAUSE TestToTerraformInitArgsUnknownBackend
=== RUN   TestToTerraformInitArgsNoBackendConfigs
=== PAUSE TestToTerraformInitArgsNoBackendConfigs
=== RUN   TestDiffersFrom
=== PAUSE TestDiffersFrom
=== RUN   TestParseTerraformStateLocal
=== PAUSE TestParseTerraformStateLocal
=== RUN   TestParseTerraformStateRemote
=== PAUSE TestParseTerraformStateRemote
=== RUN   TestParseTerraformStateRemoteFull
=== PAUSE TestParseTerraformStateRemoteFull
=== RUN   TestParseTerraformStateEmpty
=== PAUSE TestParseTerraformStateEmpty
=== RUN   TestParseTerraformStateInvalid
=== PAUSE TestParseTerraformStateInvalid
=== CONT  TestConfigValuesEqual
=== RUN   TestConfigValuesEqual/equal-both-empty
=== CONT  TestParseTerraformStateLocal
=== CONT  TestParseTerraformStateRemoteFull
--- PASS: TestParseTerraformStateLocal (0.00s)
=== PAUSE TestConfigValuesEqual/equal-both-empty
=== RUN   TestConfigValuesEqual/equal-empty-and-nil
=== PAUSE TestConfigValuesEqual/equal-empty-and-nil
=== RUN   TestConfigValuesEqual/equal-empty-and-nil-backend-config
=== PAUSE TestConfigValuesEqual/equal-empty-and-nil-backend-config
=== RUN   TestConfigValuesEqual/equal-one-key
=== PAUSE TestConfigValuesEqual/equal-one-key
--- PASS: TestParseTerraformStateRemoteFull (0.00s)
=== RUN   TestConfigValuesEqual/equal-multiple-keys
=== PAUSE TestConfigValuesEqual/equal-multiple-keys
=== RUN   TestConfigValuesEqual/equal-encrypt-bool-handling
=== PAUSE TestConfigValuesEqual/equal-encrypt-bool-handling
=== RUN   TestConfigValuesEqual/equal-general-bool-handling
=== PAUSE TestConfigValuesEqual/equal-general-bool-handling
=== RUN   TestConfigValuesEqual/equal-ignore-s3-tags
=== PAUSE TestConfigValuesEqual/equal-ignore-s3-tags
=== RUN   TestConfigValuesEqual/equal-ignore-dynamodb-tags
=== PAUSE TestConfigValuesEqual/equal-ignore-dynamodb-tags
=== RUN   TestConfigValuesEqual/unequal-wrong-backend
=== CONT  TestToTerraformInitArgsNoBackendConfigs
--- PASS: TestToTerraformInitArgsNoBackendConfigs (0.00s)
=== CONT  TestToTerraformInitArgsUnknownBackend
--- PASS: TestToTerraformInitArgsUnknownBackend (0.00s)
=== CONT  TestToTerraformInitArgs
--- PASS: TestToTerraformInitArgs (0.00s)
=== CONT  TestGetAwsSessionConfig
=== RUN   TestGetAwsSessionConfig/all-values
=== PAUSE TestGetAwsSessionConfig/all-values
=== RUN   TestGetAwsSessionConfig/no-values
=== CONT  TestForcePathStyleClientSession
=== CONT  TestParseTerraformStateInvalid
=== CONT  TestParseTerraformStateRemote
--- PASS: TestParseTerraformStateRemote (0.00s)
=== CONT  TestDiffersFrom
[terragrunt] 2019/02/03 23:49:26 Backend  has not changed.
[terragrunt] 2019/02/03 23:49:26 Backend s3 has not changed.
[terragrunt] 2019/02/03 23:49:26 Backend type has changed from s3 to atlas
[terragrunt] 2019/02/03 23:49:26 Backend s3 has not changed.
[terragrunt] 2019/02/03 23:49:26 Backend config has changed from map[bucket:foo key:bar region:us-east-1] to map[bucket:different key:bar region:us-east-1]
[terragrunt] 2019/02/03 23:49:26 Backend config has changed from map[region:us-east-1 bucket:foo key:bar] to map[bucket:foo key:different region:us-east-1]
[terragrunt] 2019/02/03 23:49:26 Backend config has changed from map[bucket:foo key:bar region:us-east-1] to map[key:bar region:different bucket:foo]
[terragrunt] 2019/02/03 23:49:26 Backend config has changed from map[something:true] to map[something:%!s(bool=false)]
--- PASS: TestDiffersFrom (0.00s)
=== CONT  TestParseTerraformStateEmpty
--- PASS: TestParseTerraformStateEmpty (0.00s)
=== PAUSE TestConfigValuesEqual/unequal-wrong-backend
=== RUN   TestConfigValuesEqual/unequal-values
=== PAUSE TestConfigValuesEqual/unequal-values
=== RUN   TestConfigValuesEqual/unequal-non-empty-config-nil
=== PAUSE TestConfigValuesEqual/unequal-non-empty-config-nil
=== RUN   TestConfigValuesEqual/unequal-general-bool-handling
=== PAUSE TestConfigValuesEqual/unequal-general-bool-handling
=== CONT  TestConfigValuesEqual/equal-both-empty
=== PAUSE TestGetAwsSessionConfig/no-values
=== RUN   TestGetAwsSessionConfig/extra-values
=== PAUSE TestGetAwsSessionConfig/extra-values
=== CONT  TestGetAwsSessionConfig/all-values
=== CONT  TestConfigValuesEqual/unequal-general-bool-handling
[terragrunt] 2019/02/03 23:49:26 Backend config has changed from map[something:%!s(bool=false)] to map[something:%!s(bool=true)]
=== CONT  TestConfigValuesEqual/unequal-non-empty-config-nil
=== CONT  TestConfigValuesEqual/unequal-values
[terragrunt] 2019/02/03 23:49:26 Backend config has changed from map[foo:different] to map[foo:bar]
=== CONT  TestConfigValuesEqual/equal-multiple-keys
=== CONT  TestConfigValuesEqual/equal-one-key
=== CONT  TestConfigValuesEqual/equal-empty-and-nil-backend-config
--- PASS: TestParseTerraformStateInvalid (0.00s)
=== CONT  TestConfigValuesEqual/unequal-wrong-backend
=== CONT  TestConfigValuesEqual/equal-empty-and-nil
[terragrunt] 2019/02/03 23:49:26 Backend type has changed from s3 to wrong
=== RUN   TestForcePathStyleClientSession/path-style-true
=== CONT  TestConfigValuesEqual/equal-encrypt-bool-handling
=== CONT  TestGetAwsSessionConfig/extra-values
=== CONT  TestConfigValuesEqual/equal-ignore-dynamodb-tags
=== CONT  TestConfigValuesEqual/equal-ignore-s3-tags
=== CONT  TestConfigValuesEqual/equal-general-bool-handling
--- PASS: TestConfigValuesEqual (0.00s)
    --- PASS: TestConfigValuesEqual/equal-both-empty (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-general-bool-handling (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-non-empty-config-nil (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-values (0.00s)
    --- PASS: TestConfigValuesEqual/equal-one-key (0.00s)
    --- PASS: TestConfigValuesEqual/equal-multiple-keys (0.00s)
    --- PASS: TestConfigValuesEqual/equal-empty-and-nil-backend-config (0.00s)
    --- PASS: TestConfigValuesEqual/equal-empty-and-nil (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-wrong-backend (0.00s)
    --- PASS: TestConfigValuesEqual/equal-encrypt-bool-handling (0.00s)
    --- PASS: TestConfigValuesEqual/equal-ignore-dynamodb-tags (0.00s)
    --- PASS: TestConfigValuesEqual/equal-ignore-s3-tags (0.00s)
    --- PASS: TestConfigValuesEqual/equal-general-bool-handling (0.00s)
=== PAUSE TestForcePathStyleClientSession/path-style-true
=== RUN   TestForcePathStyleClientSession/path-style-false
=== PAUSE TestForcePathStyleClientSession/path-style-false
=== RUN   TestForcePathStyleClientSession/path-style-non-existent
=== PAUSE TestForcePathStyleClientSession/path-style-non-existent
=== CONT  TestForcePathStyleClientSession/path-style-true
=== CONT  TestGetAwsSessionConfig/no-values
--- PASS: TestGetAwsSessionConfig (0.00s)
    --- PASS: TestGetAwsSessionConfig/all-values (0.00s)
    --- PASS: TestGetAwsSessionConfig/extra-values (0.00s)
    --- PASS: TestGetAwsSessionConfig/no-values (0.00s)
=== CONT  TestForcePathStyleClientSession/path-style-non-existent
=== CONT  TestForcePathStyleClientSession/path-style-false
--- PASS: TestForcePathStyleClientSession (0.00s)
    --- PASS: TestForcePathStyleClientSession/path-style-true (0.00s)
    --- PASS: TestForcePathStyleClientSession/path-style-non-existent (0.00s)
    --- PASS: TestForcePathStyleClientSession/path-style-false (0.00s)
PASS
ok      _/root/go-project/terragrunt/remote     0.009s

brikis98

OK, code changes and tests look great!

If you can update the README accordingly, this is good to merge. Thanks!

remote/remote_state_s3.go

Xtigyro · 2019-02-04T12:38:40Z

OK, code changes and tests look great!

If you can update the README accordingly, this is good to merge. Thanks!

@brikis98 Perfect - will update the README accordingly today in the evening. Thank you / Спасибо!

Co-Authored-By: Xtigyro <Xtigyro@users.noreply.github.com>

Xtigyro · 2019-02-04T12:51:28Z

I'll retest it as well today in the evening - before coming back here and letting you know the status.

Xtigyro · 2019-02-04T21:40:22Z

[root@localhost ~]# cat /usr/local/bin/re-go.sh
#!/usr/bin/env bash

rm -rf /root/go-project/terragrunt/
cp -r /root/shares/GitHub/Xtigyro/terragrunt/ /root/go-project/
cd /root/go-project/terragrunt/remote/
go test -v -parallel 128

[root@localhost ~]# re-go.sh
=== RUN   TestConfigValuesEqual
=== PAUSE TestConfigValuesEqual
=== RUN   TestForcePathStyleClientSession
=== PAUSE TestForcePathStyleClientSession
=== RUN   TestGetAwsSessionConfig
=== PAUSE TestGetAwsSessionConfig
=== RUN   TestToTerraformInitArgs
=== PAUSE TestToTerraformInitArgs
=== RUN   TestToTerraformInitArgsUnknownBackend
=== PAUSE TestToTerraformInitArgsUnknownBackend
=== RUN   TestToTerraformInitArgsNoBackendConfigs
=== PAUSE TestToTerraformInitArgsNoBackendConfigs
=== RUN   TestDiffersFrom
=== PAUSE TestDiffersFrom
=== RUN   TestParseTerraformStateLocal
=== PAUSE TestParseTerraformStateLocal
=== RUN   TestParseTerraformStateRemote
=== PAUSE TestParseTerraformStateRemote
=== RUN   TestParseTerraformStateRemoteFull
=== PAUSE TestParseTerraformStateRemoteFull
=== RUN   TestParseTerraformStateEmpty
=== PAUSE TestParseTerraformStateEmpty
=== RUN   TestParseTerraformStateInvalid
=== PAUSE TestParseTerraformStateInvalid
=== CONT  TestConfigValuesEqual
=== CONT  TestParseTerraformStateLocal
--- PASS: TestParseTerraformStateLocal (0.00s)
=== RUN   TestConfigValuesEqual/equal-both-empty
=== PAUSE TestConfigValuesEqual/equal-both-empty
=== CONT  TestDiffersFrom
=== RUN   TestConfigValuesEqual/equal-empty-and-nil
=== PAUSE TestConfigValuesEqual/equal-empty-and-nil
=== RUN   TestConfigValuesEqual/equal-empty-and-nil-backend-config
=== PAUSE TestConfigValuesEqual/equal-empty-and-nil-backend-config
=== RUN   TestConfigValuesEqual/equal-one-key
=== PAUSE TestConfigValuesEqual/equal-one-key
=== RUN   TestConfigValuesEqual/equal-multiple-keys
[terragrunt] 2019/02/04 23:32:58 Backend  has not changed.
=== PAUSE TestConfigValuesEqual/equal-multiple-keys
=== RUN   TestConfigValuesEqual/equal-encrypt-bool-handling
[terragrunt] 2019/02/04 23:32:58 Backend s3 has not changed.
=== PAUSE TestConfigValuesEqual/equal-encrypt-bool-handling
[terragrunt] 2019/02/04 23:32:58 Backend type has changed from s3 to atlas
=== RUN   TestConfigValuesEqual/equal-general-bool-handling
=== PAUSE TestConfigValuesEqual/equal-general-bool-handling
[terragrunt] 2019/02/04 23:32:58 Backend s3 has not changed.
=== RUN   TestConfigValuesEqual/equal-ignore-s3-tags
=== PAUSE TestConfigValuesEqual/equal-ignore-s3-tags
=== RUN   TestConfigValuesEqual/equal-ignore-dynamodb-tags
[terragrunt] 2019/02/04 23:32:58 Backend config has changed from map[bucket:foo key:bar region:us-east-1] to map[bucket:different key:bar region:us-east-1]
=== PAUSE TestConfigValuesEqual/equal-ignore-dynamodb-tags
=== RUN   TestConfigValuesEqual/unequal-wrong-backend
[terragrunt] 2019/02/04 23:32:58 Backend config has changed from map[bucket:foo key:bar region:us-east-1] to map[bucket:foo key:different region:us-east-1]
=== PAUSE TestConfigValuesEqual/unequal-wrong-backend
=== RUN   TestConfigValuesEqual/unequal-values
=== PAUSE TestConfigValuesEqual/unequal-values
[terragrunt] 2019/02/04 23:32:58 Backend config has changed from map[bucket:foo key:bar region:us-east-1] to map[bucket:foo key:bar region:different]
=== RUN   TestConfigValuesEqual/unequal-non-empty-config-nil
=== PAUSE TestConfigValuesEqual/unequal-non-empty-config-nil
[terragrunt] 2019/02/04 23:32:58 Backend config has changed from map[something:true] to map[something:%!s(bool=false)]
=== RUN   TestConfigValuesEqual/unequal-general-bool-handling
=== PAUSE TestConfigValuesEqual/unequal-general-bool-handling
=== CONT  TestToTerraformInitArgsNoBackendConfigs
--- PASS: TestDiffersFrom (0.00s)
=== CONT  TestToTerraformInitArgs
--- PASS: TestToTerraformInitArgsNoBackendConfigs (0.00s)
=== CONT  TestToTerraformInitArgsUnknownBackend
--- PASS: TestToTerraformInitArgsUnknownBackend (0.00s)
=== CONT  TestConfigValuesEqual/equal-ignore-dynamodb-tags
=== CONT  TestConfigValuesEqual/equal-ignore-s3-tags
=== CONT  TestConfigValuesEqual/equal-general-bool-handling
=== CONT  TestConfigValuesEqual/equal-encrypt-bool-handling
=== CONT  TestConfigValuesEqual/equal-one-key
=== CONT  TestParseTerraformStateEmpty
--- PASS: TestParseTerraformStateEmpty (0.00s)
--- PASS: TestToTerraformInitArgs (0.00s)
=== CONT  TestParseTerraformStateInvalid
--- PASS: TestParseTerraformStateInvalid (0.00s)
=== CONT  TestParseTerraformStateRemoteFull
--- PASS: TestParseTerraformStateRemoteFull (0.00s)
=== CONT  TestParseTerraformStateRemote
--- PASS: TestParseTerraformStateRemote (0.00s)
=== CONT  TestConfigValuesEqual/equal-both-empty
=== CONT  TestGetAwsSessionConfig
=== RUN   TestGetAwsSessionConfig/all-values
=== PAUSE TestGetAwsSessionConfig/all-values
=== RUN   TestGetAwsSessionConfig/no-values
=== PAUSE TestGetAwsSessionConfig/no-values
=== RUN   TestGetAwsSessionConfig/extra-values
=== PAUSE TestGetAwsSessionConfig/extra-values
=== CONT  TestGetAwsSessionConfig/all-values
=== CONT  TestConfigValuesEqual/unequal-general-bool-handling
[terragrunt] 2019/02/04 23:32:58 Backend config has changed from map[something:%!s(bool=false)] to map[something:%!s(bool=true)]
=== CONT  TestConfigValuesEqual/unequal-non-empty-config-nil
=== CONT  TestConfigValuesEqual/unequal-values
[terragrunt] 2019/02/04 23:32:58 Backend config has changed from map[foo:different] to map[foo:bar]
=== CONT  TestConfigValuesEqual/unequal-wrong-backend
[terragrunt] 2019/02/04 23:32:58 Backend type has changed from s3 to wrong
=== CONT  TestForcePathStyleClientSession
=== RUN   TestForcePathStyleClientSession/path-style-true
=== PAUSE TestForcePathStyleClientSession/path-style-true
=== RUN   TestForcePathStyleClientSession/path-style-false
=== PAUSE TestForcePathStyleClientSession/path-style-false
=== RUN   TestForcePathStyleClientSession/path-style-non-existent
=== PAUSE TestForcePathStyleClientSession/path-style-non-existent
=== CONT  TestForcePathStyleClientSession/path-style-true
=== CONT  TestGetAwsSessionConfig/extra-values
=== CONT  TestGetAwsSessionConfig/no-values
--- PASS: TestGetAwsSessionConfig (0.00s)
    --- PASS: TestGetAwsSessionConfig/all-values (0.00s)
    --- PASS: TestGetAwsSessionConfig/extra-values (0.00s)
    --- PASS: TestGetAwsSessionConfig/no-values (0.00s)
=== CONT  TestConfigValuesEqual/equal-multiple-keys
=== CONT  TestConfigValuesEqual/equal-empty-and-nil-backend-config
=== CONT  TestConfigValuesEqual/equal-empty-and-nil
--- PASS: TestConfigValuesEqual (0.00s)
    --- PASS: TestConfigValuesEqual/equal-ignore-s3-tags (0.00s)
    --- PASS: TestConfigValuesEqual/equal-ignore-dynamodb-tags (0.00s)
    --- PASS: TestConfigValuesEqual/equal-general-bool-handling (0.00s)
    --- PASS: TestConfigValuesEqual/equal-encrypt-bool-handling (0.00s)
    --- PASS: TestConfigValuesEqual/equal-one-key (0.00s)
    --- PASS: TestConfigValuesEqual/equal-both-empty (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-general-bool-handling (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-non-empty-config-nil (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-values (0.00s)
    --- PASS: TestConfigValuesEqual/unequal-wrong-backend (0.00s)
    --- PASS: TestConfigValuesEqual/equal-multiple-keys (0.00s)
    --- PASS: TestConfigValuesEqual/equal-empty-and-nil-backend-config (0.00s)
    --- PASS: TestConfigValuesEqual/equal-empty-and-nil (0.00s)
=== CONT  TestForcePathStyleClientSession/path-style-non-existent
=== CONT  TestForcePathStyleClientSession/path-style-false
--- PASS: TestForcePathStyleClientSession (0.00s)
    --- PASS: TestForcePathStyleClientSession/path-style-true (0.00s)
    --- PASS: TestForcePathStyleClientSession/path-style-non-existent (0.00s)
    --- PASS: TestForcePathStyleClientSession/path-style-false (0.00s)
PASS
ok      _/root/go-project/terragrunt/remote     0.014s

@brikis98 Tests - still OK. README.md - updated.

brikis98

Fantastic, thank you. I'll merge now and let the tests run. If they pass, I'll issue a new release and share the link here.

This is a follow-up to #645. The tests were failing with the error: ``` InvalidTargetBucketForLogging: You must give the log-delivery group WRITE and READ_ACP permissions to the target bucket ``` This PR is an attempt to grant those permissions.

Xtigyro added 2 commits February 3, 2019 01:07

Update remote_state_s3.go

c9988ff

Add AWS S3 Bucket-Wide Access Logging of the TFState

1a3ea95

Xtigyro mentioned this pull request Feb 3, 2019

Encrypt the bucket and log the access on the AWS S3 Bucket and encrypt the DynamoDB Lock Table of the TF State #643

Closed

brikis98 reviewed Feb 3, 2019

View reviewed changes

brikis98 mentioned this pull request Feb 3, 2019

Add AWS S3 Bucket-Wide KMS SSEncryption of the TFState #644

Closed

Xtigyro changed the title ~~And Add AWS S3 Bucket-Wide Access Logging of the TFState~~ Encrypt the bucket and log access on the AWS S3 Bucket of the TFState Feb 3, 2019

Xtigyro and others added 3 commits February 3, 2019 18:22

Fix wrong indentation

5d5e23c

Fix Wrong Indentation Remotely on GitHub

2934273

Fix multiple indentation issues through the GoLang Tools

350ccfd

Xtigyro added 2 commits February 3, 2019 23:25

Fix "/remote_state_s3.go:459"

9091786

[root@localhost remote]# go test -v -parallel 128 # _/root/go-project/terragrunt/remote ./remote_state_s3.go:459: Logger.Printf format %s reads arg #2, but call has 1 arg FAIL _/root/go-project/terragrunt/remote [build failed]

brikis98 reviewed Feb 4, 2019

View reviewed changes

lorengordon reviewed Feb 4, 2019

View reviewed changes

remote/remote_state_s3.go Outdated Show resolved Hide resolved

lorengordon reviewed Feb 4, 2019

View reviewed changes

remote/remote_state_s3.go Outdated Show resolved Hide resolved

lorengordon and others added 2 commits February 4, 2019 14:49

Update remote/remote_state_s3.go

323ab4a

Co-Authored-By: Xtigyro <Xtigyro@users.noreply.github.com>

Update remote/remote_state_s3.go

bfa1cd3

Co-Authored-By: Xtigyro <Xtigyro@users.noreply.github.com>

Update README.md

553b991

Fix link syntax in README

6136961

brikis98 approved these changes Feb 5, 2019

View reviewed changes

brikis98 merged commit f3efca3 into gruntwork-io:master Feb 5, 2019

brikis98 mentioned this pull request Feb 5, 2019

Add access logging permissions #646

Merged

yorinasub17 mentioned this pull request May 30, 2019

AWS S3 Backend with Explicit KMS key #715

Closed

tony-harverson-moonpig mentioned this pull request Sep 9, 2020

Access Logging setup causes logging loop. #1333

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Encrypt the bucket and log access on the AWS S3 Bucket of the TFState #645

Encrypt the bucket and log access on the AWS S3 Bucket of the TFState #645

Xtigyro commented Feb 3, 2019 •

edited

Loading

brikis98 left a comment

brikis98 Feb 3, 2019

Xtigyro Feb 3, 2019

Xtigyro Feb 4, 2019

brikis98 Feb 3, 2019

Xtigyro Feb 3, 2019

Xtigyro Feb 3, 2019 •

edited

Loading

Xtigyro Feb 3, 2019 •

edited

Loading

lorengordon Feb 3, 2019

Xtigyro Feb 3, 2019

Xtigyro Feb 3, 2019 •

edited

Loading

Xtigyro Feb 3, 2019 •

edited

Loading

Xtigyro commented Feb 3, 2019 •

edited

Loading

brikis98 commented Feb 3, 2019

brikis98 commented Feb 3, 2019

Xtigyro commented Feb 3, 2019 •

edited

Loading

Xtigyro commented Feb 3, 2019 •

edited

Loading

brikis98 left a comment

Xtigyro commented Feb 4, 2019 •

edited

Loading

Xtigyro commented Feb 4, 2019

Xtigyro commented Feb 4, 2019

brikis98 left a comment

Encrypt the bucket and log access on the AWS S3 Bucket of the TFState #645

Encrypt the bucket and log access on the AWS S3 Bucket of the TFState #645

Conversation

Xtigyro commented Feb 3, 2019 • edited Loading

brikis98 left a comment

Choose a reason for hiding this comment

brikis98 Feb 3, 2019

Choose a reason for hiding this comment

Xtigyro Feb 3, 2019

Choose a reason for hiding this comment

Xtigyro Feb 4, 2019

Choose a reason for hiding this comment

brikis98 Feb 3, 2019

Choose a reason for hiding this comment

Xtigyro Feb 3, 2019

Choose a reason for hiding this comment

Xtigyro Feb 3, 2019 • edited Loading

Choose a reason for hiding this comment

Xtigyro Feb 3, 2019 • edited Loading

Choose a reason for hiding this comment

lorengordon Feb 3, 2019

Choose a reason for hiding this comment

Xtigyro Feb 3, 2019

Choose a reason for hiding this comment

Xtigyro Feb 3, 2019 • edited Loading

Choose a reason for hiding this comment

Xtigyro Feb 3, 2019 • edited Loading

Choose a reason for hiding this comment

Xtigyro commented Feb 3, 2019 • edited Loading

brikis98 commented Feb 3, 2019

brikis98 commented Feb 3, 2019

Xtigyro commented Feb 3, 2019 • edited Loading

Xtigyro commented Feb 3, 2019 • edited Loading

brikis98 left a comment

Choose a reason for hiding this comment

Xtigyro commented Feb 4, 2019 • edited Loading

Xtigyro commented Feb 4, 2019

Xtigyro commented Feb 4, 2019

brikis98 left a comment

Choose a reason for hiding this comment

Xtigyro commented Feb 3, 2019 •

edited

Loading

Xtigyro Feb 3, 2019 •

edited

Loading

Xtigyro Feb 3, 2019 •

edited

Loading

Xtigyro Feb 3, 2019 •

edited

Loading

Xtigyro Feb 3, 2019 •

edited

Loading

Xtigyro commented Feb 3, 2019 •

edited

Loading

Xtigyro commented Feb 3, 2019 •

edited

Loading

Xtigyro commented Feb 3, 2019 •

edited

Loading

Xtigyro commented Feb 4, 2019 •

edited

Loading