Kafka client panic: assignment to entry in nil map

[chenziliang]

In rare, I run into the following panic. I don't know how to preproduce this issue. It seems like a race condition. I use the latest build of sarama and go1.4.2. I will continue to reproduce the issue on my side.

panic: assignment to entry in nil map

goroutine 1738 [running]:
github.com/Shopify/sarama.(*client).registerBroker(0xc2094c2180, 0xc208defa40)
    /Users/Ken/gocode/descartes/src/github.com/Shopify/sarama/client.go:367 +0xac
github.com/Shopify/sarama.(*client).updateMetadata(0xc2094c2180, 0xc208fabb60, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/Ken/gocode/descartes/src/github.com/Shopify/sarama/client.go:611 +0x127
github.com/Shopify/sarama.(*client).tryRefreshMetadata(0xc2094c2180, 0x0, 0x0, 0x0, 0x3, 0x0, 0x0)
    /Users/Ken/gocode/descartes/src/github.com/Shopify/sarama/client.go:563 +0x2d2
github.com/Shopify/sarama.(*client).RefreshMetadata(0xc2094c2180, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/Ken/gocode/descartes/src/github.com/Shopify/sarama/client.go:301 +0x161
github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc2094c2180)
    /Users/Ken/gocode/descartes/src/github.com/Shopify/sarama/client.go:541 +0x10a
github.com/Shopify/sarama.*client.(github.com/Shopify/sarama.backgroundMetadataUpdater)·fm()
    /Users/Ken/gocode/descartes/src/github.com/Shopify/sarama/client.go:135 +0x27
github.com/Shopify/sarama.withRecover(0xc209668640)
    /Users/Ken/gocode/descartes/src/github.com/Shopify/sarama/utils.go:42 +0x3a
created by github.com/Shopify/sarama.NewClient
    /Users/Ken/gocode/descartes/src/github.com/Shopify/sarama/client.go:135 +0x794

[eapache]

This looks like a duplicate of #422, which was fixed in Sarama 1.3.0. Which SHA of Sarama are you using specifically?

[chenziliang]

Hi eapache,
I am using SHA d779673. Look like the latest one is b61c4fb in master branch, probably this issue has been fixed.

BTW, I don't see the consumer group is supported for now, do we ? What I woud like to do is creating multiple goroutines in one consumer group to consume a bunch of partitions.

Thanks a lot !

[eapache]

Yes, this was fixed in 8f3630c which is a handful of commits ahead of d779673. Upgrading to v1.3.0 or to the latest master will be sufficient.

Sarama does not support consumer groups at the moment (though it is planned eventually). For now, https://github.com/wvanbergen/kafka provides a consumergroup library built on top of Sarama that you can use.

[chenziliang]

Thank you a lot !