Does this still work with 0.7.3?

[dkador]

I wasn't able to get it to work with kafka 0.7.3 running on top of zookeeper. Barfed when instantiating the KafkaClient object. Said it failed to connect.

My server is definitely running - I was able to get another python kafka client (https://github.com/dsully/pykafka) to work. But it seems like this is the official client and I want to be a good citizen here.

[mumrah]

Kafka 0.8 is protocol incompatible with 0.7.x. I have a 0.7 branch branch of this library which is tested against 0.7.2 (should work fine with 0.7.3 as well). The client for 0.8 was a total rewrite, so there's not much in common actually.

FWIW, 0.8 should be out in the next few weeks. I'd definitely recommend checking it out as it has many improvements over 0.7.

Neither the 0.7 branch nor the current stuff in trunk talk to ZK. That's a big mess I want to avoid. In 0.8.1, Kafka will support centralized offset management so consumers don't need to talk to ZK directly (this is already implemented on the 0.8 branch). For this client, you cannot do multi-process consumer groups (like you can with Java/Scala, and possibly some 3rd party clients).

Let me know if you have other questions.

Thanks!

[dkador]

Awesome, that's a huge help. I'm new to the Kafka world, but learning
quickly.

Can I get clarification on what you meant with "For this client, you cannot
do multi-process consumer groups"? To me that means the Java/Scala clients
handle running consumers across many processors/machines that will share
the load of consuming messages from a topic/partition in such a way that
each message is processed once and this client does not.

Yes?

[mumrah]

Yes the Scala/Java clients support consumer groups which load balance a
topic across multiple consumers which can be on different machines. It
requires a rebalancing algorithm that uses ZK and is pretty complex.