Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better error handling for AWS token fetching #884

Merged
merged 4 commits into from
Jul 9, 2020

Conversation

basepi
Copy link
Contributor

@basepi basepi commented Jul 7, 2020

What does this pull request do?

It turns out for docker containers on AWS elastic beanstalk, the PUT for the
token fails, and fails slowly. This is not the case for the ec2 host on which
the docker container is running. This was causing issues, compounded by the
fact that I didn't have retries=False on that call.

I also made the timeouts shorter -- all of these calls are local, so they're very fast.

Related issues

Fixes #880

It turns out for docker containers on AWS elastic beanstalk, the PUT for the
token fails, and fails slowly. This is not the case for the ec2 host on which
the docker container is running. This was causing issues, compounded by the
fact that I didn't have `retries=False` on that call.
@basepi basepi marked this pull request as ready for review July 7, 2020 21:53
@basepi basepi requested a review from beniwohli July 7, 2020 21:55
@apmmachine
Copy link
Contributor

apmmachine commented Jul 7, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Started by user Colton Myers, Replayed #6]

  • Start Time: 2020-07-09T22:06:37.395+0000

  • Duration: 29 min 58 sec

Test stats 🧪

Test Results
Failed 0
Passed 10672
Skipped 7953
Total 18625

Copy link
Contributor

@beniwohli beniwohli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a small nitpick, other than that LGTM

elasticapm/utils/cloud.py Show resolved Hide resolved
@basepi basepi merged commit d137d98 into elastic:master Jul 9, 2020
romulorosa pushed a commit to romulorosa/apm-agent-python that referenced this pull request Oct 15, 2020
* Better error handling for AWS token fetching

It turns out for docker containers on AWS elastic beanstalk, the PUT for the
token fails, and fails slowly. This is not the case for the ec2 host on which
the docker container is running. This was causing issues, compounded by the
fact that I didn't have `retries=False` on that call.

* Add changelog

* Add comments about IMDSv2
romulorosa pushed a commit to romulorosa/apm-agent-python that referenced this pull request Oct 15, 2020
* Better error handling for AWS token fetching

It turns out for docker containers on AWS elastic beanstalk, the PUT for the
token fails, and fails slowly. This is not the case for the ec2 host on which
the docker container is running. This was causing issues, compounded by the
fact that I didn't have `retries=False` on that call.

* Add changelog

* Add comments about IMDSv2
@eyalkoren
Copy link
Contributor

@elastic/apm-agent-devs one thing I missed when looking into this the first time is that not only we should expect the token query to fail, but that it still worth trying to retrieve the metadata even without the token

beniwohli pushed a commit to beniwohli/apm-agent-python that referenced this pull request Sep 14, 2021
* Better error handling for AWS token fetching

It turns out for docker containers on AWS elastic beanstalk, the PUT for the
token fails, and fails slowly. This is not the case for the ec2 host on which
the docker container is running. This was causing issues, compounded by the
fact that I didn't have `retries=False` on that call.

* Add changelog

* Add comments about IMDSv2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ReadTimeoutErrors on 169.254.169.254 /latest/api/token
4 participants