Nomad Server http health check fail timeout

[far-blue]

Nomad version

Nomad v0.4.1

Operating system and Environment details

Ubuntu 16.04 amd64

Issue

When nomad server is started and cannot talk to any peers or elect a leader it's '/v1/status/peers' http endpoint takes about 7 seconds to return and then returns with a 500 Internal Server Error and the response No cluster leader. The time it takes to do this is longer than the timeout on the consul healthcheck so all Consul reports is request canceled (Client.Timeout exceeded while awaiting headers)

I would recommend that the endpoint return with something other than a 500 error (maybe 503 would be better) and the timeout on the consul health check be increased to account for the time it takes for the peers endpoint takes to respond in this situation.

Reproduction steps

Spin up a Nomad server with consul but no peers. Curl to the peers endpoint to see how it responds and review the http healthcheck in consul to see the timeout issue.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.