Amplify init fails to resolve credential from profile using source_profile and credential_process

[lorengordon]

Before opening, please confirm:

• I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
• I have searched for duplicate or closed issues.
• I have read the guide for submitting bug reports.
• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

v12.21.0

Amplify CLI Version

4.45.2

What operating system are you using?

Ubuntu

Amplify Categories

Not applicable

Amplify Commands

init

Describe the bug

A profile with a source_profile that itself uses credential_process fails with the message:

'Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1'

Expected behavior

Amplify should be able to authenticate with any method supported by the awscli, including assuming a role.

Reproduction steps

1. Configure profiles like so:

[profile another-profile]
region = us-east-1
credential_process = aws-okta-processor authenticate --organization xxxxxx --application xxxxxx --duration 43200 --role arn:aws:iam::xxxxxx:role/xxxxxx

[profile dev]
region = us-east-1
role_arn = arn:aws:iam::xxxxxx:role/xxxxxx
role_session_name = email@someplace.dev
source_profile = another-profile

2. Run amplify init

$ amplify init
Note: It is recommended to run this command from the root of your app directory
? Enter a name for the project amplifydemo
? Enter a name for the environment dev
? Choose your default editor: Visual Studio Code
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using react
? Source Directory Path:  src
? Distribution Directory Path: build
? Build Command:  npm run-script build
? Start Command: npm run-script start
Using default provider  awscloudformation
? Select the authentication method you want to use: AWS profile

For more information on AWS Profiles, see:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

? Please choose the profile you want to use dev
Error: connect ECONNREFUSED 169.254.169.254:80
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1144:16) {
  message: 'Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1',
  errno: 'ECONNREFUSED',
  code: 'CredentialsError',
  syscall: 'connect',
  address: '169.254.169.254',
  port: 80,
  time: 2021-03-15T21:05:16.123Z,
  originalError: {
    message: 'Could not load credentials from any providers',
    errno: 'ECONNREFUSED',
    code: 'CredentialsError',
    syscall: 'connect',
    address: '169.254.169.254',
    port: 80,
    time: 2021-03-15T21:05:16.123Z,
    originalError: {
      message: 'EC2 Metadata roleName request returned error',
      errno: 'ECONNREFUSED',
      code: 'ECONNREFUSED',
      syscall: 'connect',
      address: '169.254.169.254',
      port: 80,
      time: 2021-03-15T21:05:16.123Z,
      originalError: [Object]
    }
  }
}

Log output

# Put your logs below this line
2021-03-15T21:15:22.347Z|info : amplify init core
2021-03-15T21:16:19.882Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/home/loren/.aws/config"])
2021-03-15T21:16:19.893Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/home/loren/.aws/config"])
2021-03-15T21:16:19.902Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/home/loren/.aws/config"])
2021-03-15T21:16:25.214Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["dev"])
2021-03-15T21:16:25.214Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/home/loren/.aws/config"])
2021-03-15T21:16:25.219Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"us-east-1","role_arn":"[***]1129617:role/[***]3-[***]in","role_session_name":"[***]lus3it.com","source_profile":"another-profile"}])
2021-03-15T21:16:25.220Z|info : amplify-provider-awscloudformation.system-config-manager.getCacheFilePath(["/home/loren/.amplify/awscloudformation"])
2021-03-15T21:16:25.226Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileConfig(["another-profile"])
2021-03-15T21:16:25.227Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/home/loren/.aws/config"])
2021-03-15T21:16:25.229Z|info : amplify-provider-awscloudformation.system-config-manager.getProfiledAwsConfig.profileConfig([{"region":"us-east-1","credential_process":"[***]"}])
2021-03-15T21:16:25.229Z|info : amplify-provider-awscloudformation.system-config-manager.getProfileCredentials(["another-profile"])
2021-03-15T21:16:25.230Z|info : amplify-provider-awscloudformation.system-config-manager.makeFileOwnerReadWrite(["/home/loren/.aws/credentials"])
2021-03-15T21:16:25.232Z|info : amplify-provider-awscloudformation.system-config-manager.getRoleCredentials.aws.STS([{"region":"us-east-1","credential_process":"[***]"}])
2021-03-15T21:16:25.241Z|info : amplify-provider-awscloudformation.system-config-manager.getRoleCredentials.sts.assumeRole([{"RoleArn":"[***]1129617:role/[***]3-[***]in","RoleSessionName":"[***]lus3it.com"}])
2021-03-15T21:16:25.277Z|error : amplify-provider-awscloudformation.system-config-manager.getRoleCredentials.sts.assumeRole([{"RoleArn":"[***]1129617:role/[***]3-[***]in","RoleSessionName":"[***]lus3it.com"}])
CredentialsError: Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1
2021-03-15T21:16:25.277Z|info : amplify-provider-awscloudformation.system-config-manager.getCacheFilePath(["/home/loren/.amplify/awscloudformation"])
2021-03-15T21:16:25.286Z|info : amplify-provider-awscloudformation.amplify-service-permission-check.checkAmplifyServiceIAMPermission.amplifyClient.listApps([])
2021-03-15T21:16:25.303Z|error : amplify-provider-awscloudformation.amplify-service-permission-check.checkAmplifyServiceIAMPermission.amplifyClient.listApps([])
CredentialsError: Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1
2021-03-15T21:16:25.304Z|info : amplify-provider-awscloudformation.amplify-service-manager.init.amplifyClient.createApp([{"name":"[***]ydemo","environmentVariables":{"_LIVE_PACKAGE_UPDATES":"[{\"pkg\":\"@aws-amplify/cli\",\"type\":\"npm\",\"version\":\"latest\"}]"}}])
2021-03-15T21:16:25.322Z|error : amplify-provider-awscloudformation.amplify-service-manager.init.amplifyClient.createApp([{"name":"[***]ydemo","environmentVariables":{"_LIVE_PACKAGE_UPDATES":"[{\"pkg\":\"@aws-amplify/cli\",\"type\":\"npm\",\"version\":\"latest\"}]"}}])
CredentialsError: Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1

[kaustavghosh06]

@lorengordon We don't support AWS orgs yet and marking this as a feature request.

[lorengordon]

@kaustavghosh06 That's a bit of an odd response. credential_process has absolutely zero to do with AWS Organizations. Can you please take another look, or clarify what you mean?

[kaustavghosh06]

@lorengordon Sorry for te miscommunication earlier, but what's this statement out here docredential_process = aws-okta-processor authenticate --organization xxxxxx --application xxxxxx --duration 43200 --role arn:aws:iam::xxxxxx:role/xxxxxx?

[lorengordon]

The command itself is largely immaterial. It can be anything that returns the credential_process interface supported by every AWS SDK.

• https://docs.aws.amazon.com/credref/latest/refdocs/setting-global-credential_process.html

I'm away from a computer at the moment, but when I'm back I can contrive an example for you that uses echo and hardcoded access/secret keys.

[lorengordon]

@kaustavghosh06 Here is a contrived example you can use to test with a permanent access/secret key, just insert actual values for the AccessKeyId and SecretAccessKey:

[profile cred-process-test]
region = us-east-1
credential_process = echo '{"Version": 1, "AccessKeyId": "an AWS access key", "SecretAccessKey": "your AWS secret access key"}'

$ aws sts get-caller-identity --profile cred-process-test
{
    "UserId": "xxxx",
    "Account": "yyyyyyyyyyy",
    "Arn": "arn:aws:iam::yyyyyyyyyyy:user/aaaaaaa"
}

[DarkSector]

I downgraded to version 4.46.0 and was able to use profiles with credential_process.

[lorengordon]

@DarkSector That's interesting, since I was using 4.45.2 and it wasn't working... So it was maybe fixed in 4.46.0, and then broken again in a yet newer version, so it worked when you downgraded back to 4.46.0?

[mrserverless]

I'm still getting error using Amplify CLI 6.3.1. I've set AWS_SDK_LOAD_CONFIG=1 correctly and also tried to revert to the CLI version 4.46.0. The only difference is that instead of okta I'm using aws-sso-util as my credential_process.

[mrserverless]

Ok I think I figured out the issue. When you run amplify init you must set AWS_Profile environment variable and select the profile as part of the init process. So in the above scenario it needs to be:

• export AWS_Profile=dev
• Please choose the profile you want to use dev

[tjmcewan]

A ~/.aws/credentials file also needs to exist!! 😓 Can be empty.

Thanks to this comment.

[lorengordon]

@tjmcewan Not sure about that. I do have a credentials file also.

[tjmcewan]

Wasn't a question, but an anguished cry after two hours lost to debugging. Hoping to avoid anyone joining me.

On Mon, 25 Oct 2021 at 03:00, Loren Gordon ***@***.***> wrote: @tjmcewan <https://github.com/tjmcewan> Not sure about that. I do have a credentials file also. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6882 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2GLDPBTRJWI2WDVIIGM3UIQUQFANCNFSM4ZHJGS5A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- *I acknowledge that I live and work on the land of the Wurundjeri people of the Kulin Nation. Sovereignty has never been ceded. It always was and always will be, Aboriginal land.*

[josefaidt]

Hey @lorengordon and folks 👋 I've confirmed the source_profile and credential_process is now functioning as intended.

In our ~/.aws/config

[profile testing]
region=us-east-1
credential_process = echo '{"Version": 1, "AccessKeyId": "xxx", "SecretAccessKey": "xxx"}'

And for source_profile, in our ~/.aws/credentials:

[testing]
aws_access_key_id=xxx
aws_secret_access_key=xxx

and for our ~/.aws/config

[profile testing-sourced]
region=us-east-1
role_arn=arn:aws:iam::xxx:role/testing-role
source_profile=testing

I'll close this issue for now but please reply back to this thread or create a new bug report if you are still experiencing this issue 🙂

[lorengordon]

Hey @lorengordon and folks 👋 I've confirmed the source_profile and credential_process is now functioning as intended.

In our ~/.aws/config

[profile testing]
region=us-east-1
credential_process = echo '{"Version": 1, "AccessKeyId": "xxx", "SecretAccessKey": "xxx"}'

And for source_profile, in our ~/.aws/credentials:

[testing]
aws_access_key_id=xxx
aws_secret_access_key=xxx

and for our ~/.aws/config

[profile testing-sourced]
region=us-east-1
role_arn=arn:aws:iam::xxx:role/testing-role
source_profile=testing

I'll close this issue for now but please reply back to this thread or create a new bug report if you are still experiencing this issue 🙂

@josefaidt I'm not sure that setup replicates the problem, but I am also not using amplify at the moment on any active project, so I'm unable to confirm if this is actually fixed.

The problem with your setup is that you specified the testing profile in ~/.aws/credentials with hardcoded access/secret keys. When you do that, that config takes precedence over the credential_process config, so credential_process isn't even being invoked in your setup.

[nadetastic]

Also ran across this issue, and the problem was the file under /amplify/.config/local-aws-info.json also needed to have valid profile info. I had updated my aws profile names in ~/.aws/credentials and my existing amplify project also needed to be updated with this new profile name.