object_store: Support assuming roles directly when using AWS S3

[alamb]

Originally from influxdata/object_store_rs#31 by @timvw

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Users have an expectation that a correctly configured AWS credentials or profile file will cause systems to assume configured roles. Unfortunately Rusoto does not do this out of the box (issue, pr).

One possible way to mitigate this is by migrating to aws-sdk-rust, but does not seem compatible with influxdata/object_store_rs#18 .

Describe the solution you'd like
Proposed solution is in influxdata/object_store_rs#37

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[timvw]

Existing implementation can be found in aws-sdk-rust sso.rs

This is typically used by devs, while awaiting this feature, one can run the following commands to acquire temporal credentials:

ACCOUNT_ID="XXX"
ROLE_NAME="XXX"
ACCOUNT_ROLE_NAME="${ACCOUNT_ID}-${ROLE_NAME}"
ACCESS_TOKEN="$(cat ~/.aws/sso/cache/*.json | jq -r ".accessToken" | grep -v null)"
ROLE_CREDENTIALS="$(aws sso get-role-credentials --account-id="${ACCOUNT_ID}" --role-name="${ACCOUNT_ROLE_NAME}" --access-token="${ACCESS_TOKEN}")"

export AWS_ACCESS_KEY_ID="$(echo "${ROLE_CREDENTIALS}" | jq -r ".roleCredentials.accessKeyId")"
export AWS_SECRET_ACCESS_KEY="$(echo "${ROLE_CREDENTIALS}" | jq -r ".roleCredentials.secretAccessKey")"
export AWS_SESSION_TOKEN="$(echo "${ROLE_CREDENTIALS}" | jq -r ".roleCredentials.sessionToken")"

[tustvold]

I don't think it is a practical goal to support the full suite of AWS auth options as supported by the CLI, we'd be opening ourselves up to supporting SSO, SAML, MFA, credential caching, etc... I think we should aim to support the authentication options used by workloads, which I think we have covered now, and delegate supporting the more esoteric interactive auth options to tools designed for the purpose.

I would personally recommend aws-vault as not only is it easier to use than the CLI config system imo, but it doesn't store credentials in plain text

[alamb]

Maybe there would be some way (feature flag perhaps) to use the official SDK (with its massive dependency chain) for users who want to be able to use some of the more sophisticated auth options

[tustvold]

Part of the motivation for moving away from the SDKs was we had serious difficulties providing consistent behaviour across stores, with missing functionality and inconsistent error handling to name a few of the problems. I think I would much rather keep the implementation here as lightweight as possible, primarily to reduce the maintenance burden on our limited set of maintainers, with the benefits this provides to users just an added bonus.

I think my preferred path forward would be for SDK-based implementations to be maintained out of tree, in the same vein as HDFS, if there are people who are willing to take on that effort and the proposed solution of using credential management tooling is not to their liking.

[alamb]

I think my preferred path forward would be for SDK-based implementations to be maintained out of tree, in the same vein as HDFS

This is a neat idea 👍

[timvw]

I am not willing to do a full implementation of an object store anymore but I have documented on how to acquire credentials from the aws-rust-sdk and build an AmazonS3 object store (Because using aws-vault felt tedious).

-> https://github.com/timvw/datafusion-examples/blob/main/s3-sdk-credentials/src/main.rs

[tustvold]

I bashed out a quick solution to this - #2891

It's pretty ugly, but it is slightly less ugly than working around this in downstreams