Proposal: Integrate better with the shared credentials file

[jamesls]

Version 1.3.13 of the CLI added support for loading credentials from the ~/.aws/credentials file. With this change, if credentials are found in the ~/.aws/credentials file, those credentials will be used over the credentials in the ~/.aws/config file.

The benefit of using the ~/.aws/credentials file is two fold:

1. It’s a file that only contains credentials (i.e secrets). This makes a clean separation between credentials and configuration data.
2. All of the AWS SDKs support the ~/.aws/credentials file, whereas the ~/.aws/config is specific to the AWS CLI.

So by using the ~/.aws/credentials file, you won’t have to worry about configuration credentials for any other AWS SDKs, which makes interoperability nice.

However, none of the aws configure commands (including aws configure get/set) were updated. This means that all of our control plane commands for dealing with configuration operated solely on the ~/.aws/config file. This is simple and easy to understand.

This issue is broadly about what we can do with our aws configure suite of commands to integrate better with the ~/.aws/credentials file. On one of the spectrum, we can completely switch over to ~/.aws/credentials and not allow credentials to be written into the ~/.aws/config via the configure commands (though we’d still support loading credentials from ~/.aws/config for backwards compatibility). On the other end we can leave things the way they are and users need to manually create/edit the ~/.aws/credentials file. Below are a few proposals. Feedback would be much appreciated.

Option 1: Do Nothing

Simple, don’t make any changes. Keep everything as it is now.

Option 2: Add options to use shared credentials

Add a --use-shared-credentials/--no-use-shared-credentials option that allows the user to explicitly specify which file they want to use. So for example:

aws configure —use-shared-credentials
AWS Access Key Id []: k1           <—— written to ~/.aws/credentials
AWS Secret Access Key []: k2       <—- written to ~/.aws/credentials
Default region name []: us-west-2  <—- written to ~/.aws/config
Default output format: json        <—- written to ~/.aws/config

aws configure --no-use-shared-credentials

AWS Access Key Id []: k1           <—— written to ~/.aws/config
AWS Secret Access Key []: k2       <—- written to ~/.aws/config
Default region name []: us-west-2  <—- written to ~/.aws/config
Default output format: json        <—- written to ~/.aws/config 

There’s also the question of what to do about the default. Having a default of --no-use-shared-credentials would make using the shared credentials file completely opt in. Defaulting to --use-shared-credentials would be a change in behavior and would mean you’d need to opt out of using the shared credentials file.

Option 3: Automatically Switch Over

This option would always use used the shared credentials file whenever possible for access/secret/token vars. This is basically —use-shared-credentials, but always enabled. With this option the user doesn’t have to do anything, we’ll just automatically always write out access/secret/session key information to ~/.aws/credentials. If a user wants credentials in ~/.aws/config they will need to manually edit their config file.

It would be really helpful to get feedback about which (if any) of these options people prefer.

[skyzyx]

Option 3 👍. Using a single set of credentials between the CLI and my favorite SDK is a no-brainer.

[irfan-syed]

+1 for option 3.

[ehammond]

3++

Makes for a small risk of confusion short term, but that's better than long term inconsistency and confusion, which is what we've lived with for too many years with AWS command line tools.

[davidski]

Another 👍 for option 3.

[garnaat]

Just want to make sure I understand.

Option 3 would automatically move credentials to ~/.aws/credentials but there would still be a ~/.aws/config file and it would still contain any non-credential attributes defined in ~/.aws/config, right? And those non-credentail attributes would still be accessible if I did a get_scoped_config() call? Would the credential also appear in the dict returned by a get_scoped_config() call?

[ankushnarula]

+1 option 3

[jamesls]

@garnaat Yes that's correct. Ideally the change is only to the aws configure commands so that anytime you access credential variables it will use the ~/.aws/credentials file. Botocore should remain unchanged.

So it sounds like option 3 is the option we should go with. To flesh this out a bit more:

• The list of variables considered to be credentials are: aws_access_key_id/aws_secret_access_key/aws_session_token
• Anytime you set credential variables (either interactively via aws configure or via aws configure set, any credentials variables will always be written to ~/.aws/credentials). We will no longer write credential variables to ~/.aws/config.
• Anytime you get credential variables (either if you're rerunning aws configure where in the prompt we show you the current value or if you run the aws configure get command), we will first look in the ~/.aws/credentials for credentials before then falling back to the ~/.aws/config file.

Thanks everyone for the feedback, it was really helpful.