Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding guidance of configuring compaction related parameters #8018

Closed
mitake opened this issue Jun 1, 2017 · 5 comments
Closed

Adding guidance of configuring compaction related parameters #8018

mitake opened this issue Jun 1, 2017 · 5 comments
Assignees
Milestone

Comments

@mitake
Copy link
Contributor

mitake commented Jun 1, 2017

I did some performance evaluation and found that too short compaction is harmful for throughput but too long compaction also degrades throughput. The result can be found in page 49 to 51 of https://www.slideshare.net/mitakeh/understanding-performance-aspects-of-etcd-and-raft

Based on profiling with pprof, I found that some functions related to memory allocation and garbage collection (e.g. runtime.mallocgc and runtime.scanobject) are consuming much more cycles when the compaction interval becomes larger. It can be considered that the mark phase of GC becomes heavy because of the increased number of live objects (in this case, raft.MemoryStorage.ents would be a dominating object).

Although the difference is not so significant, adding guideline for performance tuning would be helpful because the long interval is not good for memory consumption and CPU cycles.

/cc @xiang90 @heyitsanthony this is what I talked today

@mitake mitake self-assigned this Jun 1, 2017
@gyuho
Copy link
Contributor

gyuho commented Jun 1, 2017

@mitake Link is broken?

EDIT: seems like slideshare issue. Works with different links.

@mitake
Copy link
Contributor Author

mitake commented Jun 1, 2017

@gyuho really? I could see the link even without login to slideshare. Did you see 404?

@fanminshi
Copy link
Member

@gyuho link works for me

@heyitsanthony
Copy link
Contributor

Related: #7607

@mitake
Copy link
Contributor Author

mitake commented Jun 1, 2017

@fanminshi thanks for checking!
@gyuho if you still see the problem, please let me know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

5 participants