-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker "Client.Timeout" is considered fatal even tough the job policy is default "retry" #1858
Comments
I'm getting hit by this more and more often.....
Trying to add Could there be a retry or a longer timeout on starting containers? maybe configurable on the client As suggested initially, this kind of error should be retried on the client, or retried on a different box, as its a highly transient, non-permanent error condition |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Nomad Version
0.4.1
Driver
Docker
Problem
Retrying docker container doesn't work in case of docker timeout
A timeout to the docker process should not be fatal, and could be retried.. I'm not even sure why it would timeout, the server is more or less idle with no cpu, ram or io pressure.
Not sure what exact step is done during the executation, if its pulling the image from quay.io that is slow.. in that case, it should still be retried - maybe even on a different client?
Job File
https://gist.github.com/jippi/d2c60ae634f931ef379652481c57216f
Client log file
https://gist.github.com/jippi/2bef00eb2d3c335c8d9a98fd2ceeeb99
50 lines "grep context" around the allocation ID
Please don't worry about nomad version being "0.4.3", it's a 0.4.1 tag build with #1816 and and #1762 cherry-picked in as recommended by @dadgar
The text was updated successfully, but these errors were encountered: