-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed to start allocation due to binding to non-existent address #3487
Comments
@lovwal which interface did Nomad fingerprint after you restarted? |
@lovwal Could you share the client's logs when it started up. If you don't have them and you can reproduce would appreciate the logs in DEBUG level |
@lovwal Hmm, unfortunately all of that will be in the debug level logs. The INFO level logs doesn't provide much in this case. Would it be possible to collect some DEBUG logs? |
@dadgar I'll see if I can get something next week. |
@lovwal What is the networking like for this node. It looks like in those logs it picked up a IPv4 address and in your earlier dumps of the interfaces there were only link-local IPv6 address. Is it not possible that when Nomad started the first time you didn't have a routable address and that the veth pair was valid at the time? |
@dadgar I doubt it (although not impossible of course) since this machine has been online ~100 days and I had an active ssh session to it.
The docker interface will pass the check here https://github.com/hashicorp/nomad/blob/master/client/fingerprint/network.go#L204 In case it is of interest as well, I broke out the network fingerprint module and printed out the return values on the machine:
|
This PR introduces a better interface selection heuristic such that we select interfaces with globally routable unicast addresses over link local addresses. Fixes #3487
This PR introduces a better interface selection heuristic such that we select interfaces with globally routable unicast addresses over link local addresses. Fixes #3487
This PR introduces a better interface selection heuristic such that we select interfaces with globally routable unicast addresses over link local addresses. Fixes #3487
@dadgar Thanks for the quick fix! Cheers! |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Nomad version
Nomad v0.7.0
Operating system and Environment details
CentOS 7 3.10.0-514.el7.x86_64
Issue
Related to #3159
After upgrade to v0.7.0, nomad tries to bind the docker proxy address to a non-existent ipv6 link-local address (this only affected one node in our cluster):
Affected node is in drain mode, I can pull further information if required.
Reproduction steps
Restart nomad with running docker containers, unclear when/how often behavior appears.
The text was updated successfully, but these errors were encountered: