consul.address config doesn't interpolate GetInterfaceIP, or use CONSUL_HTTP_ADDR

[patmaddox]

Nomad version

Nomad v1.1.3 (8c0c814)

Operating system and Environment details

Ubuntu 20.04.2 LTS running on GCE

Issue

The consul stanza doesn't interpolate values for the address field. It also doesn't use CONSUL_HTTP_ADDR if you don't set the stanza.

Reproduction steps

Add this consul stanza to /etc/nomad.d/nomad.hcl:

consul {
  address = "{{ GetInterfaceIP \"tailscale0\" }}:8500"
}

Alternatively, set CONSUL_HTTP_ADDR to the correct value and omit the stanza.

Expected Result

Nomad sets the consul address to the IP address bound to the tailscale0 interface, e.g. 100.123.45.67:8500, or uses CONSUL_HTTP_ADDR if no address is set.

Actual Result

Nomad doesn't register with consul. There is a warning in the output:

2021-08-18T05:01:03.025Z [WARN]  client.alloc_runner.runner_hook.alloc_health_watcher: error looking up Consul registrations for allocation: alloc_id=169e6b69-cebd-cf52-df77-a4482d0254fe error="failed to retrieve services from consul: Get "http://%7B%7B%20GetInterfaceIP%20\"tailscale0\"%20%7D%7D:8500/v1/agent/services": dial tcp: lookup {>
Aug 18 05:15:08 nomad-server systemd[1]:

The relevant part being the non-interpolated value Get "http://%7B%7B%20GetInterfaceIP%20\"tailscale0\"%20%7D%7D:8500/v1/agent/services"

[lgfa29]

Thanks @patmaddox. I changed this to an enhancement since we don't actually support this yet, but sounds like a nice thing to have.

[vjanssens]

We're facing the same issue that we need to interpolate the consul address value as our consul client runs on 10.0.0.0. Unfortunately, this means we now have to remote-exec through terraform to set it per instance, as we also can't get CONSUL_HTTP_ADDR to work inside the systemd context.

Questions

• How do others deal with this?
• Can anyone point me in the right direction on how I can fix this myself?

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.