F sockaddr 0.7

[sean-]

Support late binding to IP addresses based on runtime information via the hashicorp/go-sockaddr library. This is a revised version of #2546 targeted specifically at the 0.7.X branch. Example syntax:

# Common use case:
$ consul agent -dev -bind="{{GetPrivateIP}}" -client='{{GetPrivateIP}}'
$ consul agent -dev -bind="{{GetPrivateIP  | exclude "name" "dummy0" }}" -client='{{GetPrivateIP}}'
$ consul agent -dev -bind='{{GetInterfaceIP "eth0"}}'

# A more complex and impractical example showing off how the template language can be used:
$ consul agent -dev -bind="{{GetAllInterfaces | include "type" "ip" | include "flags" "up|forwardable" | sort "type,size,address" | offset -2 | limit 2 | join "address" " " }}" -client='{{GetPrivateIP}}'

Fixes: #1478 #1620 #1570 #1110 #725 #1079

Possibly helps address: #2330 #2505 #473 #1914

And is the first step toward addressing: #2217 #1185

Known issues:

• Add Consul website docs/guide explaining this configuration syntax (this may be deferred until closer to 0.8 when we want to make this the default behavior). For now we're interested in testers.

[slackpad]

LGTM!

[rhyas]

I played a little bit with this, and I can state that it doesn't solve #473. (and possibly others) As an example, I tried this:

"addresses": {
"dns": "{{GetAllInterfaces | include "type" "ip" | include "flags" "up|forwardable" | sort "type,size,address" | offset -2 | limit 2 | join "address" " " }}",
"http": "{{GetAllInterfaces | include "type" "ip" | include "flags" "up|forwardable" | sort "type,size,address" | offset -2 | limit 2 | join "address" " " }}"
},

and the result is:

Error starting agent: DNS address resolution failed: Multiple addresses found ("127.0.0.1 10.98.164.146"), please configure one.

This is great for being able to find a single address using macros to hunt, but you still have to make sure you only find a single address for things to work. The underlying go networking layer does not support binding to a list of addresses/interfaces. It's all or nothing. (Even worse, it's only "all" tcp4 or tcp6, never both, so all acts unique for go vs. nearly every other lang!) This has apparently been an ongoing topic for quite a long time over there: golang/go#9334

Also, looked at some of the other referenced issue numbers and there seem to be a lot of examples that would return multiple addresses being thrown around that won't work. Not sure if you want to go back and clarify on those.

[sean-]

@rhyas Thank you for giving this a twirl, and yes, that's correct. Internally we don't iterate over multiple addresses... yet. That's next. For now you do have to apply a limit 1 and get to a single address, but the exact use case you describe is planned so that you can do exactly what you suggested above, which is to listen to 127.0.0.1 and the first IP address of an interface (e.g. GetDefaultIP).

[rhyas]

Is someone actually working on the iteration over multiples internally? I ask, because I'm taking a stab at that. I have it working for HTTP/S and passing all the automated tests. I'm working on DNS now. Not sure it will be as straightforward for RPC, because there doesn't appear to be a handy mux class in use. But HTTP/DNS are the two most common needs, so at least those would be a big step forward.

[sean-]

As of today, no, no one has taken this on. Please feel free to give this a twirl if you've got cycles available (multiple listeners would be very welcome work by everyone). I would not bother with RPC (it's about to be deprecated in favor of exclusively relying on the HTTP interfaces).

[RRAlex]

"bind_addr": "{{ GetAllInterfaces | include \"network\" \"10.0.0.0/16\" | limit 1 | attr \"address\" }}",

I have this line in consul 0.7.4 config.json, but it seems to be ignored and instead ends up on eth0 (default) with 10.2.x.x.

It is possible that -bind and the "bind_addr" in the json config file don't have the same support?

edit: It's running in docker with the following initial arguments (because of a setup.sh):

ENTRYPOINT []
CMD ["docker-entrypoint.sh","agent","-server","-datacenter","mydc1","-bootstrap-expect","1","-client=0.0.0.0"]

[sean-]

@RRAlex I spun up and merged #2786 a few min ago in response to this. Use -bind={{GetPrivateIP}} instead of -client=0.0.0.0 and let me know if that works. Thanks!

[sean-]

@RRAlex Until I get around to fixing the tests that I broke in #2786, a few questions:

1. If you run the above template through the sockaddr command line utility, does it return the right value?

2. Why is the -client argument set to 0.0.0.0? Can you set that to {{GetPrivateIP}} and does that work for you? The above patch removes the need to double-specify -client and -bind, but it's harmless in your case to do that as a temporary workaround until that PR gets revisited.

[RRAlex]

For context, I'm running the consul docker container as a single node cluster, as back-end to a single vault (also with hashicorp's container). Right now I have to give it a static IP or wipe consul db on every boot as I think the UUID identification has only recently been added (?).

As for -client, I was using 0.0.0.0, hoping it would listen on any interface. For restriction, I use docker networks to create n-tier access to those containers, but it can have many interfaces.

To test this, as I can't build it right now, is there a container I can pull from somewhere with your version?
If you have a docker hub repo that isn't /_/consul/, maybe I can pull from it and I'll gladly test it again!

[sean-]

@RRAlex Please be careful with your Vault data! The UUID-based NodeID lookups were added in 0.7.4 (Feb 6th). I don't personally use Consul in any Docker containers for anything that I do or work on so I'm not sure what the situation is there (we run Consul on the host itself and not in a container). Because Consul can only advertise one address, I'm a little concerned about the notion that binding to IN_ADDR_ANY will reliably work in your environment. Once you do have a chance to try -client with a non-0.0.0.0 value, I'd be curious to know how things work.

[RRAlex]

It's really just to make it available to docker swarm, I can recreate the whole content of the vault+consul anytime, so no worries there.
I found this much easier and fitting for my case than having a network filesystem just for secrets. I'll keep you posted as soon as this hits a version I can pull, but getting rid of the static IP I have in the docker-compose right now would already be a good thing.
Cheers!