network_interface parameter does not influence port allocations in 0.12.0

[snh]

Nomad version

Nomad v0.12.0 (8f7fbc8e7b5a4ed0d0209968faf41b238e6d5817)

Operating system and Environment details

• Release 10.4.0 of the debian/contrib-buster64 Vagrant box.
• Docker version 19.03.12, build 48a66213fe

$ lsb_release -d
Description:	Debian GNU/Linux 10 (buster)
$ uname -a
Linux node-1 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) x86_64 GNU/Linux

Issue

I've observed that the network_interface client parameter does not influence port allocations in 0.12.0.

My example system has two Ethernet interfaces, eth0 contains the default route, and eth1 is the interface that I wish to use for all port allocations:

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
       valid_lft 73981sec preferred_lft 73981sec
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 10.74.241.12/24 brd 10.74.241.255 scope global eth1
       valid_lft forever preferred_lft forever

This is configured using the network_interface client parameter in my Nomad configuration file:

client {
    enabled = true
    network_interface = "eth1"
}

With Nomad 0.11.3, as desired, the IP address associated with eth1 (10.74.241.12) is used for the port allocations:

$ nomad alloc status 8323d1eb-dd95-a7a8-4918-8167a3c5344d
...
Allocation Addresses (mode = "bridge")
Label                          Dynamic  Address
connect-proxy-count-dashboard  yes      10.74.241.12:21846 -> 21846
http                           yes      10.74.241.12:9002 -> 9002
...

With Nomad 1.12.0, the IP address associated with eth0 (10.0.2.15) is being used (the interface attached to the default route):

$ nomad alloc status 6820164f-7d5c-40a9-6a8a-a1a8d5b725de
...
Allocation Addresses (mode = "bridge")
Label                           Dynamic  Address
*http                           yes      10.0.2.15:9002 -> 9002
*connect-proxy-count-dashboard  yes      10.0.2.15:27517 -> 27517
...

This is resulting in the incorrect IP address being published for the service in Consul amongst other things.

Enabling DEBUG logging verbosity, I can see that Nomad 0.12.0 is at least still picking up the configuration option:

    2020-07-16T11:18:05.286Z [DEBUG] client.fingerprint_mgr.network: link speed detected: interface=eth1 mbits=1000
    2020-07-16T11:18:05.286Z [DEBUG] client.fingerprint_mgr.network: detected interface IP: interface=eth1 IP=10.74.241.12
    2020-07-16T11:18:05.286Z [DEBUG] client.fingerprint_mgr.network: unable to read link speed: path=/sys/class/net/lo/speed
    2020-07-16T11:18:05.286Z [DEBUG] client.fingerprint_mgr.network: link speed could not be detected, falling back to default speed: mbits=1000
    2020-07-16T11:18:05.297Z [DEBUG] client.fingerprint_mgr.network: unable to read link speed: path=/sys/class/net/docker0/speed
    2020-07-16T11:18:05.297Z [DEBUG] client.fingerprint_mgr.network: link speed could not be detected, falling back to default speed: mbits=1000

I note that Multi-Interface Networking was introduced in 0.12.0. Does this replace the use of the network_interface client parameter in this scenario, and if so, is there a way to set a default so that each group definition does not require this to be explicitly set?

Reproduction steps

Please let me know if you have any trouble reproducing this based on the below instructions (or if I am making a mistake somewhere!). I am happy to put together a completely automated Vagrant based environment that reproduces this if needed.

Vagrantfile

Vagrant.configure(2) do |config|
    config.vm.box = "debian/contrib-buster64"
    config.vm.network "private_network", ip: "10.74.241.12"
    config.vm.provision "shell", inline: $script, privileged: false
end

Additional steps

1. Install Docker CE.
2. Install Nomad 0.12.0.
3. Define basic Nomad configuration file including the network_interface client parameter set to 10.74.241.12.
4. Run a simple job containing a port allocations.

[Legogris]

Possibly related to #8432

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.