Sarama Producer hangs when broker dies no errors are thrown. #765
Comments
|
Try looking at |
|
I did try by adding Read, Write and Dial timeout to 30 seconds , even then once the cluster is gone its still hung |
If nothing else there will be logs on producer startup, please provide whatever logs there are.
Please provide an actual version number or git SHA, "Latest" changes all the time. |
|
The One I took belongs to Git Sha "02452b3987ac0e7b93702149e96d4f9b2e6521ec" I dont see any logs either while using Async producer or sycn producers . Is there any way to produce debug logs so that I can provide the same to you |
|
You can set |
|
# //// START OF THE ASYNC PRODUCER [sarama] 2016/10/17 19:09:47 Initializing new client # WHEN KAFKA CLUSTER GOES DOWN[sarama] 2016/10/17 19:10:17 client/metadata fetching metadata for [my_topic] from broker 10.203.31.230:9092 |
|
Please find the start of producer and issue logs for the scenario |
|
Seems like metadata fetching is going on for ever I did set "Metadata Retry to 2 and backoff time to 30sec" |
|
I tried closing the producer after receiving message "Brokers are not available" but Below logs continues to come.. even though I invoke Close() any config setting which is required for this. [sarama] 2016/10/17 20:07:50 client/metadata got error from broker while fetching metadata: dial tcp 10.203.29.89:9092: getsockopt: no route to host |
With the above configuration able to work around broker failure scenarios. The Debug logs really helped thank you. |
|
So in conclusion, not a bug, we just default to a lot of retries and long timeouts. |
|
after reading the codes, i I have summarized a formula： if all brokers can't be connected, the total timeout is: the default |
|
The timeout is also related to the number of messages buffered in the producer. |
Versions
Sarama Version: Latest
Kafka Version:2.11-0
Go Version:1.5.3
Configuration
config.Producer.Retry.Max = 2
config.Producer.Return.Successes = true
config.Producer.Return.Errors = true
config.Producer.RequiredAcks = 1
config.Producer.Timeout = 10
Logs
No logs are present. When broker dies prodcuer.SendMessage just hangs does not return.
Problem Description
We are using sync producer to produce stats to Kafka. It works fine if the kafka cluster is up and running.
If in between kafka cluster dies then "Producer.SendMessage"/"Prodcuer.Close" Just hangs and never returns.
I have to restart the producer instance in order to come out if this issue. Is there any setting which is missing?
Regards
Chaitra
The text was updated successfully, but these errors were encountered: