any recommendation for kafka monitoring tools ? #129
-
|
hi there, this is a general question about kafka, in case you might have some cool insight. Not a issue w/ avro_turf (thanks again for this cool gem). Do you have any particular recommendation for a neat kafka dashboard/monitoring tool? one that we can script to install as part of our amazon ec2 deploy, runs by itself, has some form of authentication/security, and is a great view into the kafka/schema registry eco system? |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments
-
|
We use Datadog at Zendesk, which has built-in integration with Racecar, the Kafka consumer framework we use, as well as DeliveryBoy, the producer framework we use. They also support Burrow, which is a language-agnostic tool for tracking Kafka consumers, although it gives less specific insights. |
Beta Was this translation helpful? Give feedback.
-
|
@dasch is there a (efficient) way thru avro_turf to found out any information on the topic size? (number of messages, offset between consumed/written, anything? i'm creating a admin page in our rails app where i show the kafka topics, but would really like some. data to show for the topic that can validate that we're adding data to the topic or if it's being consumed. |
Beta Was this translation helpful? Give feedback.
-
|
AvroTurf isn't meant as a Kafka specific tool, even if it integrates nicely with the ecosystem. I'd use e.g. ruby-kafka for that stuff. |
Beta Was this translation helpful? Give feedback.
We use Datadog at Zendesk, which has built-in integration with Racecar, the Kafka consumer framework we use, as well as DeliveryBoy, the producer framework we use.
They also support Burrow, which is a language-agnostic tool for tracking Kafka consumers, although it gives less specific insights.