Move zipkin reporting logic to jaeger agent/collector

[black-adder]

Requirement - what kind of business use case are you trying to solve?

There's a lot of system owners who use jaeger clients with zipkin backends so that they can take advantage of the OpenTracing API (this is an assumption). However, for us to support these uses cases, we needed to add new zipkin reporters to every client python, java, go. In most cases, the bloat added can be hidden but there's is a lot of repeated work (converting client model to zipkin model, adding a new zipkin http reporter, etc.).

Perhaps, we can move the zipkin reporting to a backend component in the jaeger stack.

Problem - what in Jaeger blocks you from solving the requirement?

There's currently no one central place where we can report spans to zipkin

Proposal - what do you suggest to solve the problem or improve the existing situation?

We can update the jaeger-agent so that it can be configured to convert jaeger spans to zipkin spans and report them to zipkin directly. This way, all the user will require is to roll out a new jaeger agent throughout the stack rather than updating every service.

An alternative for those who don't want to run jaeger agent daemons, we can configure the collector to just be a proxy to zipkin collectors.

Any open questions to address

In most cases, users are only using the jaeger clients so that they don't have to run the rest of the jaeger stack. Forcing them to run jaeger collector/agent in their stack just to report to zipkin might be overkill.

[black-adder]

@jaegertracing/jaeger-maintainers thoughts?
@rmfitzpatrick, @keitwb would something like this work for you guys?

[yurishkuro]

I like this because it reduces the bloat (and the respective maintenance burden) in the clients.

[jpkrohling]

Just as an idea: how about having something similar to the span-router scenario that was discussed in the mailing list on a recent thread about multi-tenancy? This new component would offload Zipkin logic from pretty much all components (agent/collector/clients). Bonus points: where it goes could be tenant dependent: one tenant goes to Zipkin, another goes to Jaeger.

[rmfitzpatrick]

@black-adder, while the language-specific repetitive conversion to the zipkin json format for each client is potential bloat, providing users with the option for an http reporter and tunable, versioned protocol seems like a nice feature. Would this be a potential aim for jaeger clients?

[yurishkuro]

@jpkrohling don't think that'd work, since the point here is that we want Jaeger clients to only report data in Jaeger format, to minimize client's footprint and maintenance across languages. Then conversion to Zipkin format can be implemented only once in this new component, which can look like Jaeger backend component to the clients, even though it reports spans to Zipkin.

@rmfitzpatrick designing clients to have pluggable transport and encoder is still valuable, we just don't want to support Zipkin-specific code in Jaeger clients.

[jpkrohling]

The goal would be the same: the router would respond to the same endpoints as the collector, acting as one as far as the caller is concerned. Like:

Client -> Agent -> Router -> Zipkin
Client -> Agent -> Router -> Jaeger Collector
Client -> Router -> Jaeger Collector 1
Client -> Router -> Jaeger Collector 2
Client -> Router -> Zipkin

Single tenant scenarios and scenarios where Zipkin isn't being used would just skip the deployment of the router.

Another possible solution is to use the new Kafka storage and build an ingester that would send data to Zipkin, but that would require consumers to deploy an agent, a collector, Kafka and Zipkin, as opposed to just the agent and the router (from the scenario above) or collector (from @black-adder's scenario)

[pavolloffay]

I think this module/component/service would have to ideally support all kinds of zipkin codecs and transports.

The reporters can be community contributions outside of the main clients.

[yurishkuro]

I'm not sure about "all kinds". There's only one integration point between router and Zipkin backend, so we only need to maintain one codec. For transports, I would only have http, because that's what we are replacing in the clients.

[pavolloffay]

For example in Java client the transport and codecs are pluggable, one can use it with kafka, amqp or zipkin.proto encoding.

[yurishkuro]

I think we'd want the bridge/router to be written in Go, reusing a lot of existing backend functionality. In Go it's not easy to make components pluggable at runtime the way you can add jars in Java. So my goal is to minimize the scope of the components - it's not a goal to maintain all the things for Zipkin ecosystem. For people who want more variety, they can easily take Zipkin collector code to accept spans (in a single format) from jaeger-zipkin-bridge and forward them to some other transport like Kafka/amqp/etc.

[yurishkuro]

Not really something we're planning to implement any more, partially solved by OTEL Collector.