Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request to add Metadata API environment variables #1080

Closed
jhsmith opened this issue Feb 25, 2019 · 8 comments
Closed

Request to add Metadata API environment variables #1080

jhsmith opened this issue Feb 25, 2019 · 8 comments

Comments

@jhsmith
Copy link

jhsmith commented Feb 25, 2019

Would you consider adding support for explicit timeout & retry when accessing the AWS Metadata API, similar to AWS_METADATA_SERVICE_TIMEOUT & AWS_METADATA_SERVICE_NUM_ATTEMPTS in Boto3?

See:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html
https://github.com/boto/botocore/blob/21ad34d727a301dbfc3bb47159a41cbd64e3bff2/botocore/configprovider.py#L59-L68

Reasoning:
I'm indirectly using Minio (I'm an Argo user: https://github.com/argoproj/argo) in an Amazon EKS cluster. There's a bad interaction with another tool (kube2iam), that may introduce errors when accessing the Metadata API. We've been able to circumvent the issue in our Python projects using Boto3 using these two environment variables, but Argo (using Minio) has a high number of failures in our environment.

Thanks!

@kannappanr
Copy link
Collaborator

@jhsmith Just to clarify, do you need to use environment variables similar to the ones you have mentioned above but for S3 in minio-go?

Also, minio implements only S3 API. Will check internally if we can support such an environment variable.

@jhsmith
Copy link
Author

jhsmith commented Feb 26, 2019

Correct. We'd like environment variables that control performing retries (and controlling the timeout) when accessing the following urls upon encountering an error:

defaultIAMRoleEndpoint = "http://169.254.169.254"

We believe this will help alleviate what is may be a race-condition or a design issue with kube2iam: jtblin/kube2iam#136

Adding retries and increasing the timeout when accessing the AWS Metadata API in Boto3 allowed our Python projects to work as expected.

@agnewp
Copy link

agnewp commented Feb 26, 2019

i would really like to be able to control the behavior of this http call, as it seems precarious to have any http call that could end your session if it doesn't get success on the first and only try.

Additionally this call out to the metadata api is to a hard-coded endpoint which COULD also be configurable (or dynamically resolved) since AWS has regional endpoints to help make this call add less latency.
This is only is you want to go ++ on this particular problem

@harshavardhana
Copy link
Member

We would love to support feel free to send a PR

@kannappanr
Copy link
Collaborator

@jhsmith I spoke about this internally and we have decided not to add environment variables. Application should handle environment variables and not SDKs. If let's say there are more than one application using the same SDK and one wants to use the environment variable then another application will get it by mistake.

@kannappanr kannappanr removed the triage label Mar 19, 2019
@kannappanr
Copy link
Collaborator

Closing this issue as won't fix. Please feel free to reach out to us if you have further questions.

@stefansedich
Copy link

@jhsmith did you end up finding a workaround? just got bit by this one ourselves with KIAM.

@jhsmith
Copy link
Author

jhsmith commented May 15, 2020

@stefansedich we had to go around kiam/kube2iam and add explicit aws creds as env vars for our pods rather than relying on the metadata api. Rather disappointing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants