Feature request: add runtime/env var of host ip #2429

bsphere · 2017-03-11T16:20:45Z

It is needed to add the host ip as a dns server to a the docker driver (for querying consul).
I know about NOMAD_IP_<port_label> but not all running jobs have open ports...

lnguyen · 2017-03-11T23:40:22Z

👍

dadgar · 2017-03-13T18:14:19Z

What exactly do you think the host ip would be given that there can be many network devices, multiple IPs per NIC, etc?

bsphere · 2017-03-14T07:23:34Z

good question, for a "standard" setup I'd like to get the private ip,
the use case I had in mind is setting dns_servers in a docker task to the local consul/dnsmasq client or passing url arguments for connecting tasks to other local running services (such as nsq), which I'm currently doing with 172.17.0.1 (docker)..

dadgar · 2017-03-14T17:30:14Z

If you are on a bridge network, that private IP wouldn't be routable? I do not want to add an environment variable that doesn't have clear behavior or benefit. It may be better to add an option to the client config to allow the operator to set the --dns flag for docker when using the bridge network.

bsphere · 2017-03-14T20:42:56Z

@dadgar AFAIK it is routable on a bridge network

dadgar · 2017-03-14T21:17:01Z

@bsphere Sorry you are right about the routing. Did my point about the DNS make sense though?

bsphere · 2017-03-15T05:54:34Z

It'll definitely improve the integration with consul and seems like a good feature. But it won't solve the issue of passing arguments or env vars of locally running services such as nsq. On Mar 14, 2017 23:17, "Alex Dadgar" <notifications@github.com> wrote: @bsphere <https://github.com/bsphere> Sorry you are right about the routing. Did my point about the DNS make sense though? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2429 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB2LTC07MX-HW-LDTXYiz4f6qEbmpQGlks5rlwPRgaJpZM4MaP5c> .

dadgar · 2017-03-15T17:27:04Z

Can you not just register those with consul and do a lookup of their IP using DNS? Thanks, Alex Dadgar

…

On Mar 14, 2017, 10:54 PM -0700, Gal Ben-Haim ***@***.***>, wrote: It'll definitely improve the integration with consul and seems like a good feature. But it won't solve the issue of passing arguments or env vars of locally running services such as nsq. On Mar 14, 2017 23:17, "Alex Dadgar" ***@***.***> wrote: @bsphere <https://github.com/bsphere> Sorry you are right about the routing. Did my point about the DNS make sense though? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2429 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB2LTC07MX-HW-LDTXYiz4f6qEbmpQGlks5rlwPRgaJpZM4MaP5c> . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

skyrocknroll · 2017-03-18T03:28:08Z

We run consul-template in each container. We want all the consul-template in each container to point to the consul installed in the host machine. So that loads on the consul are minimized. To get the host ip of the containers right now we are exposing an portmap. It would be great if the host ip (Nomad client_addr) is exposed inside the containers.

LinusU · 2017-03-20T13:33:07Z

I'm experiencing this problem as well, currently we are pointing all of our containers in our cluster to our master consul nodes. But we really want them to just query the consul node local to the host that they are running on. Is anyone aware of a workaround that could work in the meantime?

hmalphettes · 2017-03-23T22:58:11Z

[ Edit: dont use this.
Use @dadgar HOST_IP = "${attr.unique.network.ip-address}"
https://github.com//issues/2429#issuecomment-289320884 ]

@LinusU Here is my workaround.

The script that starts nomad on my machines generates the config.
I use that to set the IP address in the node_class of the nomad config:

cat >${PREFIX}/etc/nomad/config.hcl <<EOL
(...)
client {
  enabled = true
  node_class = "role=$ROLE,address=$PRIVATE_IPV4"
  (...)
EOL

I then pass the node_class to the environment (https://www.nomadproject.io/docs/job-specification/env.html#interpolation):

env {
  NODE_CLASS = "${nomad.class}"
}

Is this applicable to your situation?

LinusU · 2017-03-24T08:37:38Z

Hehe, that's quite clever 😄 I think I could use something similar, thanks!

dadgar · 2017-03-26T22:16:47Z

@hmalphettes I would not do that. The node class is used to optimize the scheduler by detecting feasibility of placements at a class level. This allows skipping nodes that we know wont work when picking a new placement. So by making the node class unique you will loose that and it goes against the purpose of the node class.

Instead use this which already provides the IP:

env {
  HOST_IP = "${attr.unique.network.ip-address}"
}

I had actually forgotten about this but I think have the above lets us close this issue as there is a direct work around. Thoughts?

jippi · 2017-03-27T07:12:07Z

that seem like a solid hack @dadgar :)

bsphere · 2017-03-27T17:33:17Z

yeah you close the issue. can you add it to the documentation?

…

On 27 March 2017 at 01:16, Alex Dadgar ***@***.***> wrote: @hmalphettes <https://github.com/hmalphettes> I would not do that. The node class is used to optimize the scheduler by detecting feasibility of placements at a class level. This allows skipping nodes that we know wont work when picking a new placement. So by making the node class unique you will loose that and it goes against the purpose of the node class. Instead use this which already provides the IP: env { HOST_IP = "${attr.unique.network.ip-address}" } I have actually forgotten about this but I think have the above lets us close this issue as there is a direct work around. Thoughts? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2429 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB2LTFY8CmfyEp2OO6LX81Vp3wT0fRyCks5rpuPSgaJpZM4MaP5c> .

dadgar · 2017-03-27T17:45:44Z

Thanks all!

Docs cc7cdc1

bsphere · 2017-03-27T17:49:46Z

dns_servers = ["${attr.unique.network.ip-address}", "8.8.8.8"]
this doesn't work for me with Nomad 0.5.5, results in a strange /etc/resolv.conf

dadgar · 2017-03-27T18:09:28Z

@bsphere It worked for me:

Starting nomad:

$ ifconfig
ens32     Link encap:Ethernet  HWaddr 00:0c:29:b1:7c:df
          inet addr:192.168.74.136  Bcast:192.168.74.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:feb1:7cdf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:683723 errors:0 dropped:0 overruns:0 frame:0
          TX packets:311208 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:615142211 (615.1 MB)  TX bytes:35101803 (35.1 MB)
...

$ sudo nomad agent -dev -network-interface=ens32

Running this job:

job "example" {
  type = "batch"
  datacenters = ["dc1"]
  group "foo" {
    task "foo" {
      driver = "docker"
      config {
	image = "redis:3.2"
	dns_servers = ["${attr.unique.network.ip-address}", "8.8.8.8"]
        command = "sleep"
        args = ["1000"]
      }
    }
  }
}

Resulted in this resolve.conf

docker exec -it foo-92ae988f-b374-e0a1-e726-b448349cc515 cat /etc/resolv.conf
search localdomain
nameserver 192.168.74.136
nameserver 8.8.8.8

bsphere · 2017-03-27T18:53:18Z

hmm. maybe because I use bind_addr = "0.0.0.0" ?!

advertise addresses are a private AWS ip

dadgar · 2017-03-27T19:17:48Z

@bsphere If you run nomad node-status -verbose <node-id> what is the value of attr.unique.network.ip-address.

This value is actually separate from the bind_addr.

bsphere · 2017-03-27T19:23:57Z

@dadgar its the private ip address.. i'll have to try it again.
I can also see there's unique.platform.aws.local-ipv4.

is there a way to use those runtime values in a template?

lnguyen · 2017-03-27T19:39:27Z

is ${attr.unique.network.ip-address} new?

dadgar · 2017-03-27T22:17:10Z

@bsphere Will be for 0.5.6: #2488

@lnguyen Nope it has been around for many releases!

github-actions · 2022-12-14T02:17:42Z

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

schmichael added theme/client type/enhancement labels Mar 13, 2017

dadgar closed this as completed Mar 27, 2017

github-actions bot locked as resolved and limited conversation to collaborators Dec 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature request: add runtime/env var of host ip #2429

Feature request: add runtime/env var of host ip #2429

bsphere commented Mar 11, 2017

lnguyen commented Mar 11, 2017

dadgar commented Mar 13, 2017

bsphere commented Mar 14, 2017

dadgar commented Mar 14, 2017

bsphere commented Mar 14, 2017 •

edited

Loading

dadgar commented Mar 14, 2017

bsphere commented Mar 15, 2017 via email

dadgar commented Mar 15, 2017 via email

skyrocknroll commented Mar 18, 2017

LinusU commented Mar 20, 2017

hmalphettes commented Mar 23, 2017 •

edited

Loading

LinusU commented Mar 24, 2017

dadgar commented Mar 26, 2017 •

edited

Loading

jippi commented Mar 27, 2017

bsphere commented Mar 27, 2017 via email

dadgar commented Mar 27, 2017

bsphere commented Mar 27, 2017 •

edited

Loading

dadgar commented Mar 27, 2017

bsphere commented Mar 27, 2017 •

edited

Loading

dadgar commented Mar 27, 2017

bsphere commented Mar 27, 2017

lnguyen commented Mar 27, 2017

dadgar commented Mar 27, 2017

github-actions bot commented Dec 14, 2022

Feature request: add runtime/env var of host ip #2429

Feature request: add runtime/env var of host ip #2429

Comments

bsphere commented Mar 11, 2017

lnguyen commented Mar 11, 2017

dadgar commented Mar 13, 2017

bsphere commented Mar 14, 2017

dadgar commented Mar 14, 2017

bsphere commented Mar 14, 2017 • edited Loading

dadgar commented Mar 14, 2017

bsphere commented Mar 15, 2017 via email

dadgar commented Mar 15, 2017 via email

skyrocknroll commented Mar 18, 2017

LinusU commented Mar 20, 2017

hmalphettes commented Mar 23, 2017 • edited Loading

LinusU commented Mar 24, 2017

dadgar commented Mar 26, 2017 • edited Loading

jippi commented Mar 27, 2017

bsphere commented Mar 27, 2017 via email

dadgar commented Mar 27, 2017

bsphere commented Mar 27, 2017 • edited Loading

dadgar commented Mar 27, 2017

bsphere commented Mar 27, 2017 • edited Loading

dadgar commented Mar 27, 2017

bsphere commented Mar 27, 2017

lnguyen commented Mar 27, 2017

dadgar commented Mar 27, 2017

github-actions bot commented Dec 14, 2022

bsphere commented Mar 14, 2017 •

edited

Loading

hmalphettes commented Mar 23, 2017 •

edited

Loading

dadgar commented Mar 26, 2017 •

edited

Loading

bsphere commented Mar 27, 2017 •

edited

Loading

bsphere commented Mar 27, 2017 •

edited

Loading