Support CORS on Zipkin endpoint

[torhovland]

Hi!

I've instrumented a simple Javascript frontend and a web API backend with Zipkin, and this works. I wanted to use Jaeger as a drop-in replacement for Zipkin on port 9411 by stopping Zipkin and starting Jaeger according to the docs:

docker run -d -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 -p5775:5775/udp -p6831:6831/udp -p6832:6832/udp -p5778:5778 -p16686:16686 -p14268:14268 -p9411:9411 jaegertracing/all-in-one:latest

However, now I get the following error in my Javascript app:

Failed to load http://localhost:9411/api/v2/spans: 
Response to preflight request doesn't pass access control check: 
No 'Access-Control-Allow-Origin' header is present on the requested resource. 
Origin 'http://localhost:8080' is therefore not allowed access. 
The response had HTTP status code 404. 
If an opaque response serves your needs, set the request's mode to 'no-cors' 
to fetch the resource with CORS disabled.

Here's some more details about the failing request:

Request URL:http://localhost:9411/api/v2/spans
Request Method:OPTIONS
Status Code:404 Not Found
Remote Address:[::1]:9411
Referrer Policy:no-referrer-when-downgrade
Content-Length:19
Content-Type:text/plain; charset=utf-8
Date:Tue, 20 Feb 2018 17:25:30 GMT
X-Content-Type-Options:nosniff
Accept:*/*
Accept-Encoding:gzip, deflate, br
Accept-Language:en-US,en;q=0.9,nb;q=0.8,da;q=0.7
Access-Control-Request-Headers:content-type
Access-Control-Request-Method:POST
Connection:keep-alive
Host:localhost:9411
Origin:http://localhost:8080
Referer:http://localhost:8080/
User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36

I have verified that Jaeger does indeed listen on POST http://localhost:9411/api/v2/spans using Postman.

[yurishkuro]

I would guess that the error happens in the web app itself because it's trying to talk to a different address vs where it was loaded from. When you said "this works" did you mean you are able to submit spans to Zipkin backend but not to Jaeger backend? Any chance you can share the code with which we can reproduce?

cc @tiffon

[torhovland]

Yes, all of the above is correct. Attached is a small frontend app that demonstrates the issue.

cd client
npm install
npm start

Then browse to http://localhost:8080. The "app" simply says Hello, but if you open the browser dev tools, you will see that it also immediately tries to do a traced call to a backend. I've not included the backend code, but it shouldn't matter. You should still see the CORS issue when trying to call Jaeger.

client.zip

[bedorlan]

Hello. Have you found a workaround for this issue? Thanks

[yurishkuro]

If an opaque response serves your needs, set the request's mode to 'no-cors' 
to fetch the resource with CORS disabled.

this seems to indicate that you can change the client submitting spans to not require CORS from the API server.

I'll check what we did internally for CORS.

[wyfrel]

Any news concerning this issue ?

[jpkrohling]

I'd say that you should have a reverse proxy in front of the collector for this kind of use case. This reverse proxy should handle authentication, authorization, TLS, CORS and related options.

Most OAuth proxy implementations that I know of handle all (or most) of this for you.

[ianblenke]

Unfortunately, even with a reverse proxy attempting to set the CORS response headers for Access-Control-Allow-Origin: * and Vary: Origin, the CORS pre-flight check OPTIONS method is getting back a 405 from the jaeger collector zipkin service.

I'm deploying to a kubernetes cluster with a traefik reverse proxy, and I have this Ingress defined:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jaeger-zipkin-ingress
  namespace: kube-system
  annotations:
    ingress.kubernetes.io/custom-request-headers: "Origin:https://zipkin.example.com"
    ingress.kubernetes.io/custom-response-headers: "Access-Control-Allow-Origin:*||Access-Control-Allow-Methods:POST,GET,OPTIONS||Vary:Origin"
    ingress.kubernetes.io/enable-cors: "true"
    ingress.kubernetes.io/cors-allow-methods: "POST,GET,OPTIONS"
    ingress.kubernetes.io/cors-allow-origin: "*"
    ingress.kubernetes.io/cors-allow-credentials: "true"
spec:
  rules:
  - host: zipkin.example.com
    http:
      paths:
      - backend:
          serviceName: jaeger-collector
          servicePort: 9411

Note: While some of that may appear duplicitous, traefik v1.7.9 is only honoring the custom headers annotations, the others are mostly there as an example of annotations that may accomplish the same task with nginx or other ingress controllers.

This traefik does have ACME SSL certs, and is merrily proxying over 50 other services. The dashboard is showing the custom headers are there, and I see them in a developer mode in my browser, so that's all working correctly.

The challenge is that I'm getting back:

Access to fetch at 'https://zipkin.example.com/api/v1/spans' from origin 'https://my-web-service.example.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.

It appears that the jaeger collector's zipkin implementation doesn't support the OPTIONS method at all, and is returning a 405 Invalid Method response.

I've tried all manner of setting request headers and response headers for CORS, with the same result every time.

I am actively looking at how I can redirect just the OPTIONS method to another backend service that can simply reply with a 200 OK so that I can get past this silly pre-flight status check for CORS.

The primary reason I've ended up trying to use a simple zipkin collector here is that the node.js jaeger-client doesn't webpack well (a mess of native node.js dependencies), and does not appear to be another "easy" way to instrument opentracing in javascript on the browser side.

Any suggestions on how to either instrument jaeger tracing on the browser side from a Vue.js app that is talking to a .NET core backend (not node.js), or get the jaeger collector's zipkin service to respond to an OPTIONS request with a 200 OK to get past this CORS pre-flight check are greatly welcomed.

[jpkrohling]

It appears that the jaeger collector's zipkin implementation doesn't support the OPTIONS method at all, and is returning a 405 Invalid Method response.

It's been a while since I last tried this out, but proxies should handle this case. Typically, if OPTIONS is implemented in the backend for an endpoint, proxies would just use it, otherwise, they'd respond with the proxy's configuration.

In any case, I wouldn't have anything against accepting a PR that implements a generic OPTIONS handler, if that would solve the problem for you. It would probably just be a matter of adding a handler around here:

jaeger/cmd/query/app/http_handler.go

Lines 110 to 120 in 6bbe8dc

	func (aH *APIHandler) RegisterRoutes(router *mux.Router) {
	aH.handleFunc(router, aH.getTrace, "/traces/{%s}", traceIDParam).Methods(http.MethodGet)
	aH.handleFunc(router, aH.archiveTrace, "/archive/{%s}", traceIDParam).Methods(http.MethodPost)
	aH.handleFunc(router, aH.search, "/traces").Methods(http.MethodGet)
	aH.handleFunc(router, aH.getServices, "/services").Methods(http.MethodGet)
	// TODO change the UI to use this endpoint. Requires ?service= parameter.
	aH.handleFunc(router, aH.getOperations, "/operations").Methods(http.MethodGet)
	// TODO - remove this when UI catches up
	aH.handleFunc(router, aH.getOperationsLegacy, "/services/{%s}/operations", serviceParam).Methods(http.MethodGet)
	aH.handleFunc(router, aH.dependencies, "/dependencies").Methods(http.MethodGet)
	}

[yurishkuro]

@mrbeann only Zipkin APi for submitting spans i supported, not for retrieval.