Unable to use AWS SDK Profiles: Access Denied

[osterman]

Terraform Version

Terraform v0.7.10
Terraform v0.7.8

I've tried downgrading to older v0.7.x with no success.

Affected Resource(s)

Unable to use valid profile as defined in ~/.aws/credentials with the TF provider type of aws.

https://www.terraform.io/docs/providers/aws/

Using aws cli with same profile & environment variables works fine.

http://docs.aws.amazon.com/cli/latest/topic/config-vars.html

Terraform Configuration Files

I've tried using the below, both with and without expressing explicit profile that should be used.

provider "aws" {
  region = "${var.aws_region}"
  profile = "default" # tried with/without
  shared_credentials_file = "/Users/e/.aws/credentials" # tried with/without
}

Debug Output

AWS_SDK_LOAD_CONFIG=1 AWS_SHARED_CREDENTIALS_FILE=/Users/e/.aws/credentials AWS_DEFAULT_PROFILE=default TF_LOG=debug bin/terraform-0.7.10/terraform plan
2016/11/09 15:31:49 [INFO] Terraform version: 0.7.10  fcf12bc46a34716652a5b9a4d7905361003293e7
2016/11/09 15:31:49 [INFO] CLI args: []string{"/Users/e/Dropbox/Dev/sagan/cloud-platform/bin/terraform-0.7.10/terraform", "plan"}
2016/11/09 15:31:49 [DEBUG] Detected home directory from env var: /Users/e
2016/11/09 15:31:49 [DEBUG] Detected home directory from env var: /Users/e
2016/11/09 15:31:49 [DEBUG] Attempting to open CLI config file: /Users/e/.terraformrc
2016/11/09 15:31:49 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2016/11/09 15:31:49 [DEBUG] Detected home directory from env var: /Users/e
2016/11/09 15:31:49 [WARN] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2016/11/09 15:31:49 [DEBUG] plugin: waiting for all plugin processes to complete...
Error reloading remote state: AccessDenied: Access Denied
	status code: 403, request id: 6E2C5E46C80CE049

Panic Output

N/A

Expected Behavior

We should be able to leverage the credentials stored in our ~/.aws/credentials.

Defining AWS_DEFAULT_PROFILE to a valid profile name (with non-expired STS tokens) should be adequate as that's all we need to set for the aws cli to work with AWS_SDK_LOAD_CONFIG is true and AWS_SHARED_CREDENTIALS_FILE is a valid path.

Actual Behavior

AccessDenied

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:
populate ~/.aws/credentials with current STS tokens:

[default]
aws_access_key_id=
aws_secret_access_key=
aws_session_token=

populate ~/.aws/config to with appropriate role_arn mapped to profile in credentials.

[default]
role_arn=arn:aws:iam::1234567890:role/Admin
source_profile=default
region = us-east-1

(make sure to update aws account id and role name)

Set the profile you want to use: export AWS_DEFAULT_PROFILE=default

Verify aws cli happy to confirm proper configuration;
aws ec2 describe-instances (assuming you have "admin" privileges)

Try running terraform plan:
terraform plan

Important Factoids

• We are using Okta as our SSO.
• We assume an admin role with carte blanche access to rule out problems with having insufficient privileges
• We use https://github.com/oktadeveloper/okta-aws-cli-assume-role to populate ~/.aws/credentials and ~/.aws/config (I renamed the dynamic profile names to just default to reduce complexity; aws cli happy / terraform unhappy)

References

• AWS_DEFAULT_PROFILE does not work with profiles containing Role ARNs aws/aws-cli#1748 (we have unset AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_SECURITY_TOKEN)
• AuthorizationHeaderMalformed for S3 remote #2774 (we have tried setting AWS_DEFAULT_REGION and AWS_REGION)
• Feature Request: Support for sts:assumerole/source_profile aws/aws-sdk-go#472 (we have tried setting AWS_SDK_LOAD_CONFIG)

[osterman]

Sorry - turned out to be a permissions problem :-/

[osterman]

Related: #10067

[rbuels]

@osterman I'm running into something similar, what did the permissions problem turn out to be?

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.