Port selection for Connect envoy when network.port.to is used

[passcod]

Nomad version

Nomad v1.0.2 (4c1d4fc)

Operating system and Environment details

• Ubuntu 20.04.1 LTS
• Running with docker driver, docker version: 19.03.8

Issue

Given a job that looks like this:

job "takeout" {
  type = "service"
  datacenters = ["dc1"]

  group "pizzeria" {
    network {
      mode = "bridge"

      port "http" {
        to = 5000
      }
    }

    service {
      name = "pizza"
      port = "http"

      connect {
        sidecar_service {}
      }
    }

    task "server" { ... }
  }
}

The envoy sidecar will get configured to connect to 127.0.0.1:12345 where 12345 is the dynamic port allocated for http. That may not connect! (In my setup that port is on the host's interface instead.) It should be the to port.

Current workaround is to specify:

sidecar_service {
  proxy {
    local_service_port = 5000
  }
}

This may possibly be related to #9730?

[idrennanvmware]

So we do what you're looking for in the following manner (essentially)

We make sure the port we want to receive inbound traffic on is registered with the service (9090 in this case). Then we listen for it on 0.0.0.0:9090

service {
        name = "fake-service-api"
        port = 9090

        connect {
            sidecar_service {
                proxy {
                    upstreams {
                        destination_name = "fake-service-backend"
                        local_bind_port = 9091
                    }

                    upstreams {
                        destination_name = "fake-service-cache"
                        local_bind_port = 9092
                    }

                    config {
                        protocol = "http"
                    }
                }
            }
        }
    }

and

    task "fake-service-api" {
        driver = "docker"

        config {
            image = "nicholasjackson/fake-service:v0.12.0"
        }

        env {
            UPSTREAM_URIS = "http://localhost:9091, http://localhost:9092"
            LISTEN_ADDR = "0.0.0.0:9090"
            NAME = "${NOMAD_TASK_NAME}"
            MESSAGE = "Hello from ${NOMAD_TASK_NAME}:${NOMAD_ALLOC_ID} HostIp: ${attr.unique.network.ip-address}"
            HTTP_CLIENT_KEEP_ALIVES = "false"
        }
    }

I'm not sure what the "right" way of doing it is, but this might get you rolling :)

[passcod]

I do want to use named ports, though. At the very least, it seems silly to have named ports and then not being able to use them for anything useful.

But fundamentally, it doesn't matter what I'm trying to do. A definition that looks like my OP produces a proxy setup that is critically misconfigured and surely that's a bug.

[passcod]

I should add, I've already got my setup working. Not the issue.