Secure tracing data to Jaeger backend

[fmhwong]

Jim Mulvey, Ajay Reddy, Chunlong Liang and Felix Wong had a meeting on 9/09/2019 to discuss how to secure the tracing data from Jaeger Client to Jaeger backend.

Since the data transfer between Jaeger Client and Agent is using unsecure UDP, we agreed this is acceptable to go out as is as a beta.

For GA, we need to find a more secure solution.

Jim found a git issue on Jaeger security:
jaegertracing/jaeger#1718

And it has a link to this article.
https://medium.com/@larsmilland01/secure-architecture-for-jaeger-with-apache-httpd-reverse-proxy-on-openshift-f31983fad400

[fmhwong]

Copy and paste Jim's comment from email:

To summarize, we agreed to going ahead with UDP for the beta but before we approve this for GA, we need to look at securing this better. Two options we discussed:

1. using the option to configure it so the Jaeger client calls over HTTPS out to the Jaeger Collector (skipping the Agent). We could also implement an oauth proxy in front for the Jaeger Collector. Someone needs to investigate whether the Jaeger client can be configured to call out over HTTPS- if not we need to make this happen.
   or
2. implementing encryption of trace data/Jaeger propagated credentials inside Liberty on the call to the Jaeger client -for this we'd need a way to provide a secret to the Jaeger agent so that it can decrypt the payload before sending it over gRPC or TLS to the Jaeger Collector. I expect this means adding this support to the Jaeger agent.

Jim

[yurishkuro]

(1) yes, but varies by language
(2) agent->collector over gRPC supports TLS with client cert authentication

[fmhwong]

@yurishkuro We are having concerns on the unsecured connection between client and agent. Does the Java client support https calls to the collector?

[yurishkuro]

(1) yes, but varies by language

[fmhwong]

@yurishkuro Does it work for Java? If yes, do you have pointers to the documentation?

[yurishkuro]

I assume you can specify https as JAEGER_URL. But collector itself doesn't run https server iirc so you'd need to put a proxy in front of it.

[fmhwong]

jaegertracing/jaeger-client-java#602 hasn't been merged yet so I guess the client can't send https directly to collector yet.

[yurishkuro]

That PR adds client cert validation, if you need it for client auth. If you're only concerned with transport security, then https should already be supported.

[fmhwong]

Successfully setup a nginx https reverse proxy in front of jaeger-controller and I was able to use a https endpoint in jaeger-client.