Support for AWS assume roles when configuring S3 remote state storage

[adhodgson1]

Hi,

Since Terraform 0.7.3 we have been able to successfully use assume roles when dealing with the AWS provider.

It would be good if this functionality was also added to the Terraform state storage in S3, so we can use a combination of AWS credentials and assume role to access the S3 bucket.

[adhodgson1]

This also needs to apply to the remote state resources that use S3, as it is not possible to assume roles there either.

[jpluscplusm]

It was a surprise to me to discover that S3 remote state was pulled down using local AWS creds, even if an assume_role was configured. I suggest that the necessary IAM roles/rules/etc could be somewhat cleaner if an assume_role was enacted before remote state was fetched.

[damacus]

Looks like the same is true for the terraform importcommand.

[daveadams]

Yeah this is going to bite us, too, as we are starting to roll out more cross-account stuff and eventually we hope to move our IAM creds out of the account where our resources are.

[mikereinhold]

I see the same things as @adhodgson1 and @damacus - both the state provider and the import command have the same problem

[dtserekhman]

Yes, please consider this enhancement/fix. Running into the same problem. In the "provider" section I use "assume_role" but this "assume_role" specification does not seem to apply to the "terraform_remote_state" where backend="s3".

[andrewerickson]

Just ran into this as well.

[yohanndulong]

need assume role with import command. Possible ?

[darend]

Also ran into this when following AWS Identity Account Structure. We have multiple AWS accounts and the one that users login to just provides a means of assuming a role within the various sub-accounts. With local state the TF commands used the assumed role, ie targeted the sub account. However when we switch to s3 for state, it would always try to store it in a bucket under identity account, ignoring the assumed role.

If it helps, we are doing auth using Okta following this guide and this tool for cli auth: https://github.com/oktadeveloper/okta-aws-cli-assume-role. It performs auth using MFA and writes the profile + token to ~/.aws/config and ~/.aws/credentials

Finally, we found that with TF_LOG=trace we do not see any messages about whats causing the AccessDenied. Only through experimentation did we find out what was happening:

TF_LOG=trace ./terraform remote config -backend=s3 -backend-config="bucket=daren-tf-test" -backend-config="key=terraform.tfstate"  -backend-config="region=us-east-1" -backend-config="profile=admin"
2016/11/10 21:39:39 [INFO] Terraform version: 0.7.10  fcf12bc46a34716652a5b9a4d7905361003293e7
2016/11/10 21:39:39 [INFO] CLI args: []string{"/Users/daren/development/sagan/tmp/terraform", "remote", "config", "-backend=s3", "-backend-config=bucket=daren-tf-test", "-backend-config=key=terraform.tfstate", "-backend-config=region=us-east-1", "-backend-config=profile=admin"}
2016/11/10 21:39:39 [DEBUG] Detected home directory from env var: /Users/daren
2016/11/10 21:39:39 [DEBUG] Detected home directory from env var: /Users/daren
2016/11/10 21:39:39 [DEBUG] Attempting to open CLI config file: /Users/daren/.terraformrc
2016/11/10 21:39:39 [DEBUG] Detected home directory from env var: /Users/daren
2016/11/10 21:39:39 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2016/11/10 21:39:39 [TRACE] Preserving existing state lineage "cc8e59fc-eb88-46fa-a713-f74a6176fbfa"
2016/11/10 21:39:39 [TRACE] Preserving existing state lineage "cc8e59fc-eb88-46fa-a713-f74a6176fbfa"
2016/11/10 21:39:39 [TRACE] Preserving existing state lineage "cc8e59fc-eb88-46fa-a713-f74a6176fbfa"
2016/11/10 21:39:39 [TRACE] Preserving existing state lineage "cc8e59fc-eb88-46fa-a713-f74a6176fbfa"
2016/11/10 21:39:39 [TRACE] Preserving existing state lineage "cc8e59fc-eb88-46fa-a713-f74a6176fbfa"
2016/11/10 21:39:39 [WARN] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2016/11/10 21:39:39 [TRACE] Preserving existing state lineage "cc8e59fc-eb88-46fa-a713-f74a6176fbfa"
Remote configuration updated
2016/11/10 21:39:39 [TRACE] Preserving existing state lineage "cc8e59fc-eb88-46fa-a713-f74a6176fbfa"
2016/11/10 21:39:39 [TRACE] Preserving existing state lineage "cc8e59fc-eb88-46fa-a713-f74a6176fbfa"
2016/11/10 21:39:39 [WARN] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2016/11/10 21:39:40 [DEBUG] plugin: waiting for all plugin processes to complete...
Error while performing the initial pull. The error message is shown
below. Note that remote state was properly configured, so you don't
need to reconfigure. You can now use `push` and `pull` directly.

Error reloading remote state: AccessDenied: Access Denied
    status code: 403, request id: 6EB6BEE47E66FDBD

edit: realized that normal tf operations when run with local state were still not using the correct role unless we specified role_arn in the provider config. Providing it in the provider config does not work for remote state. Related issue: #8911

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.