Skip to content
This repository has been archived by the owner on Aug 23, 2023. It is now read-only.

kafka: make sure we can do 50kmetrics/s without issues in devstack #234

Closed
Dieterbe opened this issue Jul 10, 2016 · 1 comment
Closed

kafka: make sure we can do 50kmetrics/s without issues in devstack #234

Dieterbe opened this issue Jul 10, 2016 · 1 comment
Assignees
@Dieterbe

Comments

@Dieterbe
Copy link
Contributor

Dieterbe commented Jul 10, 2016 •
edited
Loading

currently at this rate some fake-metrics request take >4s to ingest, which is not acceptable, on my laptop with SSD. (without compression and at 20kHz)
every 30s there seems to be a sync to disk which causes lots of iowait, we should smooth this out over time I think. and test the different compression options

metric-tank-kafka-troubleshooting and collectd-disk-io dashboards in docker stack help with this.
@Dieterbe Dieterbe self-assigned this Jul 18, 2016
@Dieterbe
Copy link
Contributor Author

since i've been having issues with getting the compression to work (IBM/sarama#698 and https://issues.apache.org/jira/browse/KAFKA-3961) this goal has been postponed and will be picked up again later in proactive qa testing.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@Dieterbe Dieterbe

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Dieterbe