-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Streaming ingestion support in the collector #212
Comments
I mean as a transport similar to Zipkin (https://github.com/openzipkin/zipkin-reporter-java). I understand jaeger uses UDP based agent but we already use Kafka for centralized logging and it would be nice to use the same infrastructure to report spans. Hope this clarifies :-) |
@hzariv there are a few reasons we decided against a streaming ingestion model, Kafka or others:
As I mentioned, we do have the ability to steam spans out of collectors. We are building a post-collection data pipeline that will provide various aggregates on top of standalone trace data. Internally we do ingest some Kafka streams that contain certain enrichment data generated by tools other than Jaeger clients, such as haproxy/routing logs, or tracing data from mobile apps. At some point we can implement (or better, accept a PR) ingestion of spans from Kafka, mostly for compatibility with existing Zipkin instrumentation / installation. |
Looking forward to support this. |
This is supported by jaeger-ingestor. |
@hzariv you mentioned a lack of streaming support as an issue, could you please elaborate? Do you mean streaming ingestion of spans (e.g. from Kafka), or a stream coming out of collectors?
Note that we actually have support for both (subject to the exact format) that we can open source if there's a demand. The streaming ingestion somewhat goes against the design of Jaeger, since we want an active bi-directional communication between clients-agent-collector, but it's certainly possible to add.
The text was updated successfully, but these errors were encountered: