Taking advantage of server RPC streaming for requesting log info from clients in another cluster

[margueritepd]

Feel free to close this if this has already been done. I didn't see it mentioned in #404 though so I'm opening this issue up.

I have two federated nomad clusters. The clients in both only advertise private IPs for HTTP and RPC, since in theory only the servers in their region would need to communicate with them. The servers advertise public addresses for RPC, since in federated clusters, the clusters must communicate with each other over RPC.

Unfortunately, pre Nomad 0.8, that would mean that if Hashi UI was running in one cluster, it would be unable to stream the logs and files of jobs running in the other cluster, because pre Nomad 0.8, the only way to get that information was to make an HTTP call directly to the client on which the job was running, which only advertised its private IP and so would be inaccessible to the Hashi UI instance.

Nomad 0.8 fixed this via streaming RPCs through the server: hashicorp/nomad#3892 and they updated their UI to take advantage of this: hashicorp/nomad#3908

It would be great if Hashi UI would take advantage of being able to query the server in the neighbouring cluster for file & log information (and apparently also allocation utilization stats).

[jippi]

We use the official Nomad SDK, I need to see if our usage automatically brought in the RPC streaming or not - should be trivial to migrate to RPC streaming if it isn't done so automatically

[margueritepd]

Looking at the Nomad UI PR, it looks like there needs to be a change in the address used to query this info. Instead of querying the client, it needs to query the server. (But only if useServer is set. Not sure where that would be configured.)