Support AWS SSO profiles

[iainelder]

I set up my environment variables and check that the session is valid using the AWS CLI.

$ export AWS_PROFILE=sandbox-logging
$ export AWS_DEFAULT_REGION=eu-west-1
$ aws sts get-caller-identity
{
    "UserId": "AROAXXXXXXXXXXXXXXXXX:iain",
    "Account": "111111111111",
    "Arn": "arn:aws:sts::111111111111:assumed-role/AWSReservedSSO_AdministratorAccess_aaaaaaaaaaaaaaaa/iain"
}

s5cmd seems to ignore my environment variables and instead tries to query the EC2 metadata service.

$ s5cmd --log debug ls
DEBUG retryable error: RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
DEBUG retryable error: RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
DEBUG retryable error: RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": dial tcp 169.254.169.254:80: connect: no route to host

I was expecting output like this:

$ aws s3 ls
2021-11-23 15:29:46 aws-sam-cli-managed-default-samclisourcebucket-xxxxxxxxxxxx
2021-11-16 17:47:47 cf-templates-xxxxxxxxxxxxx-eu-west-1
2021-11-30 20:11:58 org-trail-xxxxxxxx
2021-12-01 12:36:42 org-trail-yyyyyyyy
...

I'm using an SSO profile. Does that matter?

[profile sandbox-logging]
sso_start_url = https://d-1111111111.awsapps.com/start
sso_region = eu-west-1
sso_account_id = 111111111111
sso_role_name = AdministratorAccess
region = eu-west-1

[iainelder]

I'm using an SSO profile. Does that matter?

When I generate temporary credentials for the SSO profile using aws2-wrap, s5cmd works just fine.

$ s5cmd --log debug ls
2021/11/23 14:29:46  s3://aws-sam-cli-managed-default-samclisourcebucket-xxxxxxxxxxxx
2021/11/16 16:47:47  s3://cf-templates-xxxxxxxxxxxxx-eu-west-1
2021/11/30 19:11:58  s3://org-trail-xxxxxxxx
2021/12/01 14:25:45  s3://org-trail-yyyyyyyy
...

To support SSO profiles, s5cmd needs to upgrade its dependency on aws-sdk-go from v1.35.13.

s5cmd/go.mod

Line 6 in fd03f6e

github.com/aws/aws-sdk-go v1.35.13

The minimum version that supports SSO profiles is v.1.37.0.

The latest version to improve support for SSO profiles was v1.38.42.

Would you be able to upgrade the dependency to make this work automatically?

[igungor]

The minimum version that supports SSO profiles is v.1.37.0.

The latest version to improve support for SSO profiles was v1.38.42.

Would you be able to upgrade the dependency to make this work automatically?

Thanks for the pointer. We've updated the SDK version to v1.40.25. Could you please try it out and see if it works for you?

[iainelder]

Thanks @igungor . I won't have time to try it until next week, but I'll let you know how it works for me when I have a chance.

[iainelder]

@igungor which version contains your update? The latest release is still version 1.4.0 from September.

[igungor]

Hi @iainelder

We've released v2.0.0-beta which includes the change you requested along with new features and bugfixes.

[iainelder]

@igungor , thanks! Version v2.0.0-beta-df6b29f is working as expected.

[ollie-bell]

Edit: Never mind - I have found in the readme where the --profile flag is mentioned, but looks like that was only added recently and isn't in the latest release. In the meantime I have got it to work by setting the AWS_PROFILE env var.

Please can the readme be updated with instructions on using s5cmd with AWS SSO profiles? I'm unclear on how to get s5cmd to work whilst logged in with an AWS SSO profile (i.e. having followed instructions here https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html). What further steps are required?