Improve auto-sharding documentation #1559
Conversation
k8s-ci-robot
added
cncf-cla: yes
| @@ -43,7 +43,7 @@ are deleted they are no longer visible on the `/metrics` endpoint. | |||
| - [kube-state-metrics vs. metrics-server](#kube-state-metrics-vs-metrics-server) | |||
| - [Scaling kube-state-metrics](#scaling-kube-state-metrics) | |||
| - [Resource recommendation](#resource-recommendation) | |||
| - [Horizontal scaling (sharding)](#horizontal-scaling-sharding) | |||
There was a problem hiding this comment.
I think horizontal scaling can be a bit confusing since it might indicate that all KSM replicas are the same.
README.md
Outdated
|
|
||
| There is also an experimental feature, that allows kube-state-metrics to auto discover its nominal position if it is deployed in a StatefulSet, in order to automatically configure sharding. This is an experimental feature and may be broken or removed without notice. | ||
| KSM supports a feature which allows each shard to discover its nominal position when deployed in a StatefulSet. which is useful for automatically configuring sharding. This is an experimental feature and may be broken or removed without notice. |
There was a problem hiding this comment.
@brancz do you have any background information on why this feature was marked as experimental? Is it simply because it never got promoted to stable?
There was a problem hiding this comment.
@kubernetes/kube-state-metrics-maintainers should we start marking this as non-experimental with the next release to get more users test it? Or do we have plans to remove it at some point?
|
/assign @mrueg |
|
Thanks for working on this @fpetkovski! I added some comments @SuperQ as you asked about more information, is this doc change helpful for you? |
README.md
Outdated
|
|
||
| There is also an experimental feature, that allows kube-state-metrics to auto discover its nominal position if it is deployed in a StatefulSet, in order to automatically configure sharding. This is an experimental feature and may be broken or removed without notice. | ||
| KSM supports a feature which allows each shard to discover its nominal position when deployed in a StatefulSet. which is useful for automatically configuring sharding. This is an experimental feature and may be broken or removed without notice. |
There was a problem hiding this comment.
Do we want to introduce KSM as an abbreviation for the project or stick with "kube-state-metrics"?
There was a problem hiding this comment.
Good point, I used it as second nature 😄. I added the abbreviation next to the first reference of kube-state-metrics at the beginning of the readme. I think that it is used commonly used in the community so IMO it makes sense to have it in the docs. Wdyt?
There was a problem hiding this comment.
I you change the sentence structure a bit, you could write it like this.
Automatic sharding allows each shard each shard to discover its nominal position
This avoids needing to be overly self-referential about the fact that this is a kube-state-metrics feature.
There was a problem hiding this comment.
Thanks for the suggestion, applied
README.md
Outdated
|
|
||
| * `--shard` (zero indexed) | ||
| * `--total-shards` | ||
|
|
||
| Sharding is done by taking an md5 sum of the Kubernetes Object's UID and performing a modulo operation on it, with the total number of shards. The configured shard decides whether the object is handled by the respective instance of kube-state-metrics or not. Note that this means all instances of kube-state-metrics even if sharded will have the network traffic and the resource consumption for unmarshaling objects for all objects, not just the ones it is responsible for. To optimize this further, the Kubernetes API would need to support sharded list/watch capabilities. Overall memory consumption should be 1/n th of each shard compared to an unsharded setup. Typically, kube-state-metrics needs to be memory and latency optimized in order for it to return its metrics rather quickly to Prometheus. | ||
| Sharding is done by taking an md5 sum of the Kubernetes Object's UID and performing a modulo operation on it with the total number of shards. Each shard decides whether the object is handled by the respective instance of kube-state-metrics or not. Note that this means all instances of kube-state-metrics, even if sharded, will have the network traffic and the resource consumption for unmarshaling objects for all objects, not just the ones they are responsible for. To optimize this further, the Kubernetes API would need to support sharded list/watch capabilities. In the optimal case, memory consumption for each shard will be 1/n compared to an unsharded setup. Typically, kube-state-metrics needs to be memory and latency optimized in order for it to return its metrics rather quickly to Prometheus. |
There was a problem hiding this comment.
Should we mention --use-apiserver-cache as a way to reduce load on etcd here?
There was a problem hiding this comment.
Good idea, added the suggestion
fpetkovski
force-pushed
the
document-autosharding
branch
2 times, most recently
from
c7184e0
to
99dcf27
Compare
Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
fpetkovski
force-pushed
the
document-autosharding
branch
from
99dcf27
to
8eecd40
Compare
|
/approve |
k8s-ci-robot
added
the
approved
|
/lgtm Thanks for improving the doc @fpetkovski ! |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: fpetkovski, mrueg, SuperQ The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
What this PR does / why we need it:
This PR improves the documentation on auto-sharding. It explains the pros and cons of using auto-sharding vs manual sharding.
How does this change affect the cardinality of KSM: (increases, decreases or does not change cardinality)
Does not affect it
Which issue(s) this PR fixes:
Fixes #1546