-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Lower Performance Compared to Java Client #2103
Comments
@dnwe I have tested the PR, the overall throughput of our application increased from about 500 qps to 2500 qps. It's awesome. Hope the PR can be merged soon. |
@HY1310, glad to hear you are getting better performance out of Sarama. If you have a constant flow of records and 20ms of latency (including the time it takes to persist records), you can send up to With But if you generate records at a higher rate you can actually get better throughput by accumulating more than 10 records in a
By fitting/batching more records in a |
Versions
Please specify real version numbers or git SHAs, not just "Latest" since that changes fairly regularly.
Configuration
What configuration values are you using for Sarama and Kafka?
Problem Description
test scenario,
message size - 1KB
network latency - 20ms
If the network latency is lower than 10ms Sarama will have similar qps as java client. But Sarama's performance decreases a lot when the network latency is high.
Looking into "async_producer.go" and "broker.go" we found that it's actually a sync invocation,
This explains the qps decreasing. So is it possible to optimize the throughput of Sarama in high latency scenario? Thanks a lot.
The text was updated successfully, but these errors were encountered: