-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nomad client tries to connect to serf IP address, not to RPC #16211
Comments
@Kamilcuk can you share your |
Hi! It's missing. Default is [] empty array. |
Hi @Kamilcuk! This sounds an awful lot like the situation that was described in #11895 but that I was never able to replicate. The function on the client that handles this is
So Consul is used to initially discover an advertised list of servers, but then the client uses that list to get the peers (which I think is intended to exclude the non-voting servers). There are two serf tags of interest to us here. One is the
It looks like that behavior was introduced in 5976511 in v0.8.0 So the last thing to do is figure out where There's two diagnostics you can do to help us debug further:
|
Thank you. Your posts as always way too comprehensive for me. I fixed the issue by using: I see status/peers listens serf addresses, not RPC. As I understand, this is unexpected? outputs
(Relevant) server config:
|
Yeah, There's going to be some backwards compatibility concerns here that we'll need to discuss. I'm going to try to reproduce on my own in the meantime. |
Hi @Kamilcuk! I've just opened #16217 with what looks like a working fix. This isn't going to make it for Nomad 1.5.0 GA because we need to do some testing around the other auto-discovery mechanisms. But if you want to check out that patch and |
Nomad version
1.4.3
Operating system and Environment details
Fedora 29
Issue
Nomad client does not use RPC IP to connect to Nomad servers. He uses serf IP. He should be using RPC IP.
Reproduction steps
Configured the server with the following advertise block:
Nomad servers have registered services in consul:
However, nomad agents connect with 10.120 ip address to port 4647??
Expected Result
Nomad agents should connect to RPC IP port.
Actual Result
Nomad agents try to connect to serf IP and fail.
Client config
client config
The text was updated successfully, but these errors were encountered: