log flooding when using retry_join

[yikaus]

Nomd v.0.2.2

When applied below configuration to manually join a cluster , the log file get flooded until other nodes are available

Enable the server

server {
enabled = true
bootstrap_expect = 3
retry_join = ["192.169.0.20","192.168.0.21"]
}

However it showing retrying in 30s , but actually not having any delay

2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s
2015/12/17 22:11:27 [WARN] agent: Join failed: dial tcp 192.168.0.20:4648: getsockopt: connection refused, retrying in 30s

[cbednarski]

Thanks for the report! Can you show the command you used to invoke Nomad? The serf consensus / retry is much more aggressive than 30s delay so I suspect that one of your CLI flags was not passed in correctly.

[yikaus]

@charlieok I am not using special command , see below

nomad agent -config /etc/nomad.d/server.hcl

server.hcl

bind_addr = "192.168.0.22"
log_level = "DEBUG"
data_dir = "/tmp/nomad"
server {
enabled = true
bootstrap_expect = 3
retry_join = ["192.169.0.20","192.168.0.21"]
}

[c4milo]

I've seen this too myself before

[diptanu]

I am trying to reproduce this now.

[yikaus]

Cool , wait for next release .

[diptanu]

@yikaus We just released 0.2.3 which had this fix!

[yikaus]

Just tried new release , it is fixed . Thanks

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.