Skip to content

Commit

Permalink
feat: fix Markdown format
Browse files Browse the repository at this point in the history
  • Loading branch information
opeco17 committed Sep 2, 2023
1 parent 3030d4e commit efc5dce
Show file tree
Hide file tree
Showing 11 changed files with 408 additions and 388 deletions.
3 changes: 2 additions & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ If the matter is security related, please disclose it privately see https://gith
**Anything else we need to know?**:

**Environment**:
- kube-state-metrics version:

- kube-state-metrics version:
- Kubernetes version (use `kubectl version`):
- Cloud provider or hardware configuration:
- Other info:
710 changes: 357 additions & 353 deletions CHANGELOG.md

Large diffs are not rendered by default.

3 changes: 1 addition & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ Here's a list of types that we use:
| chore | Other changes that don't modify src or test files |
| revert | Reverts a previous commit |


### Further Information

- [Contributor License Agreement](https://git.k8s.io/community/CLA.md) Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests
Expand All @@ -52,4 +51,4 @@ Here's a list of types that we use:
## Contact Information

- [Join Slack](http://slack.k8s.io) to sign up and join the Kubernetes Slack. Please make sure to read our [Slack Guidelines](https://github.com/kubernetes/community/blob/master/communication/slack-guidelines.md) before participating.
- The [kube-state-metrics slack channel](https://kubernetes.slack.com/messages/CJJ529RUY) provides an effective communication platform to reach out to members and other users of the project. It offers an alternative to submitting a GitHub issue for when you have questions and issues.
- The [kube-state-metrics slack channel](https://kubernetes.slack.com/messages/CJJ529RUY) provides an effective communication platform to reach out to members and other users of the project. It offers an alternative to submitting a GitHub issue for when you have questions and issues.
3 changes: 2 additions & 1 deletion MAINTAINER.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,13 @@ kube-state-metrics is welcoming contributions from the community. If you are int
As part of the Kubernetes project, we use the community membership process as described [here](https://github.com/kubernetes/community/blob/master/community-membership.md). We do not adhere strictly to the numbers of contributions and reviews. Still as becoming a maintainer is a trust-based process and we desire positive outcomes for the project, we look for a long-term interest and engagement.

## Adding a new reviewer

* Ensure the new reviewer is a member of the [kubernetes organization](https://github.com/kubernetes/org/blob/main/config/kubernetes/org.yaml).
* Add the new reviewer to the [OWNERS](OWNERS) file to be able to review pull requests.
* Add the new reviewer to the [kube-state-metrics-maintainers group](https://github.com/kubernetes/org/blob/main/config/kubernetes/sig-instrumentation/teams.yaml), to gain write access to the kube-state-metrics repository (e.g. for creating new releases).


## Adding a new approver

* Ensure the new approver is already a reviewer in the [OWNERS](OWNERS) file.
* Add the new approver to the [OWNERS](OWNERS) file to be able to approve pull requests.
* Add the new approver to the [SECURITY_CONTACTS](SECURITY_CONTACTS) file to be able to get notified on security related incidents.
Expand Down
34 changes: 23 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,6 @@ Generally, it is recommended to use the latest release of kube-state-metrics. If
| **v2.10.0** | v1.27 |
| **main** | v1.27 |


#### Resource group version compatibility

Resources in Kubernetes can evolve, i.e., the group version for a resource may change from alpha to beta and finally GA
Expand All @@ -90,8 +89,14 @@ release.
#### Container Image

The latest container image can be found at:
<<<<<<< HEAD
* `registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.10.0` (arch: `amd64`, `arm`, `arm64`, `ppc64le` and `s390x`)
* View all multi-architecture images at [here](https://explore.ggcr.dev/?image=registry.k8s.io%2Fkube-state-metrics%2Fkube-state-metrics:v2.10.0)
=======

- `registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.9.2` (arch: `amd64`, `arm`, `arm64`, `ppc64le` and `s390x`)
- View all multi-architecture images at [here](https://explore.ggcr.dev/?image=registry.k8s.io%2Fkube-state-metrics%2Fkube-state-metrics:v2.9.2)
>>>>>>> 700766b2 (feat: fix Markdown format)
### Metrics Documentation

Expand Down Expand Up @@ -132,13 +137,15 @@ If you encounter those errors in the metrics, it is most likely a configuration
at the logs of kube-state-metrics.

Example of the above mentioned metrics:

```
kube_state_metrics_list_total{resource="*v1.Node",result="success"} 1
kube_state_metrics_list_total{resource="*v1.Node",result="error"} 52
kube_state_metrics_watch_total{resource="*v1beta1.Ingress",result="success"} 1
```

kube-state-metrics also exposes some http request metrics, examples of those are:

```
http_request_duration_seconds_bucket{handler="metrics",method="get",le="2.5"} 30
http_request_duration_seconds_bucket{handler="metrics",method="get",le="5"} 30
Expand All @@ -149,6 +156,7 @@ http_request_duration_seconds_count{handler="metrics",method="get"} 30
```

kube-state-metrics also exposes build and configuration metrics:

```
kube_state_metrics_build_info{branch="main",goversion="go1.15.3",revision="6c9d775d",version="v2.0.0-beta"} 1
kube_state_metrics_shard_ordinal{shard_ordinal="0"} 0
Expand Down Expand Up @@ -180,8 +188,8 @@ To some extent, the Kubernetes objects in a cluster are in direct proportion to

As a general rule, you should allocate:

* 250MiB memory
* 0.1 cores
- 250MiB memory
- 0.1 cores

Note that if CPU limits are set too low, kube-state-metrics' internal queues will not be able to be worked off quickly enough, resulting in increased memory consumption as the queue length grows. If you experience problems resulting from high memory allocation or CPU throttling, try increasing the CPU limits.

Expand Down Expand Up @@ -226,8 +234,8 @@ metrics from monitoring systems such as Prometheus.

In order to shard kube-state-metrics horizontally, some automated sharding capabilities have been implemented. It is configured with the following flags:

* `--shard` (zero indexed)
* `--total-shards`
- `--shard` (zero indexed)
- `--total-shards`

Sharding is done by taking an md5 sum of the Kubernetes Object's UID and performing a modulo operation on it with the total number of shards. Each shard decides whether the object is handled by the respective instance of kube-state-metrics or not. Note that this means all instances of kube-state-metrics, even if sharded, will have the network traffic and the resource consumption for unmarshaling objects for all objects, not just the ones they are responsible for. To optimize this further, the Kubernetes API would need to support sharded list/watch capabilities. In the optimal case, memory consumption for each shard will be 1/n compared to an unsharded setup. Typically, kube-state-metrics needs to be memory and latency optimized in order for it to return its metrics rather quickly to Prometheus. One way to reduce the latency between kube-state-metrics and the kube-apiserver is to run KSM with the `--use-apiserver-cache` flag. In addition to reducing the latency, this option will also lead to a reduction in the load on etcd.

Expand All @@ -246,11 +254,13 @@ The downside of using an auto-sharded setup comes from the rollout strategy supp
### Daemonset sharding for pod metrics

For pod metrics, they can be sharded per node with the following flag:
* `--node`

- `--node`

Each kube-state-metrics pod uses FieldSelector (spec.nodeName) to watch/list pod metrics only on the same node.

A daemonset kube-state-metrics example:

```
apiVersion: apps/v1
kind: DaemonSet
Expand Down Expand Up @@ -285,6 +295,7 @@ go get k8s.io/kube-state-metrics

Simply run the following command in this root folder, which will create a
self-contained, statically-linked binary and build a Docker image:

```
make container
```
Expand Down Expand Up @@ -312,7 +323,7 @@ To have Prometheus discover kube-state-metrics instances it is advised to create
kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=$(gcloud info --format='value(config.account)')
```

Note that your GCP identity is case sensitive but `gcloud info` as of Google Cloud SDK 221.0.0 is not. This means that if your IAM member contains capital letters, the above one-liner may not work for you. If you have 403 forbidden responses after running the above command and `kubectl apply -f examples/standard`, check the IAM member associated with your account at https://console.cloud.google.com/iam-admin/iam?project=PROJECT_ID. If it contains capital letters, you may need to set the --user flag in the command above to the case-sensitive role listed at https://console.cloud.google.com/iam-admin/iam?project=PROJECT_ID.
Note that your GCP identity is case sensitive but `gcloud info` as of Google Cloud SDK 221.0.0 is not. This means that if your IAM member contains capital letters, the above one-liner may not work for you. If you have 403 forbidden responses after running the above command and `kubectl apply -f examples/standard`, check the IAM member associated with your account at <https://console.cloud.google.com/iam-admin/iam?project=PROJECT_ID>. If it contains capital letters, you may need to set the --user flag in the command above to the case-sensitive role listed at <https://console.cloud.google.com/iam-admin/iam?project=PROJECT_ID>.

After running the above, if you see `Clusterrolebinding "cluster-admin-binding" created`, then you are able to continue with the setup of this service.

Expand All @@ -321,6 +332,7 @@ After running the above, if you see `Clusterrolebinding "cluster-admin-binding"
If you want to run kube-state-metrics in an environment where you don't have cluster-reader role, you can:

- create a serviceaccount

```yaml
apiVersion: v1
kind: ServiceAccount
Expand All @@ -330,6 +342,7 @@ metadata:
```

- give it `view` privileges on specific namespaces (using roleBinding) (*note: you can add this roleBinding to all the NS you want your serviceaccount to access*)

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
Expand Down Expand Up @@ -361,7 +374,6 @@ spec:

For the full list of arguments available, see the documentation in [docs/cli-arguments.md](./docs/cli-arguments.md)


#### Helm Chart

Starting from the kube-state-metrics chart `v2.13.3` (kube-state-metrics image `v1.9.8`), the official [Helm chart](https://artifacthub.io/packages/helm/prometheus-community/kube-state-metrics/) is maintained in [prometheus-community/helm-charts](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics). Starting from kube-state-metrics chart `v3.0.0` only kube-state-metrics images of `v2.0.0 +` are supported.
Expand All @@ -373,12 +385,12 @@ running:

> Users can override the apiserver address in KUBE-CONFIG file with `--apiserver` command line.
go install
kube-state-metrics --port=8080 --telemetry-port=8081 --kubeconfig=<KUBE-CONFIG> --apiserver=<APISERVER>
go install
kube-state-metrics --port=8080 --telemetry-port=8081 --kubeconfig=<KUBE-CONFIG> --apiserver=<APISERVER>

Then curl the metrics endpoint

curl localhost:8080/metrics
curl localhost:8080/metrics

To run the e2e tests locally see the documentation in [tests/README.md](./tests/README.md).

Expand Down
2 changes: 2 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,14 @@ Maintaining the release branches for older minor releases happens on a best effo
* Changelog entry
* Only include user relevant changes
* Entries in the [`CHANGELOG.md`](CHANGELOG.md) are meant to be in this order:

```
[CHANGE]
[FEATURE]
[ENHANCEMENT]
[BUGFIX]
```

* All lines should be full sentences
* kube-state-metrics image tag used in Kubernetes deployment yaml config.
* Cut the new release branch, e.g. `release-1.2`, or merge/cherry-pick changes onto the minor release branch you intend to tag the release on
Expand Down
3 changes: 1 addition & 2 deletions docs/cli-arguments.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,8 @@ spec:
- '--apiserver=<APISERVER>'
```

## Available options:
## Available options

[embedmd]:# (../help.txt)
```txt
$ kube-state-metrics -h
kube-state-metrics is a simple service that listens to the Kubernetes API server and generates metrics about the state of the objects.
Expand Down
16 changes: 8 additions & 8 deletions docs/customresourcestate-metrics.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Custom Resource State Metrics

This section describes how to add metrics based on the state of a custom resource without writing a custom resource
This section describes how to add metrics based on the state of a custom resource without writing a custom resource
registry and running your own build of KSM.

## Configuration
Expand Down Expand Up @@ -86,7 +86,6 @@ NOTE: The `customresource_group`, `customresource_version`, and `customresource_

Please be aware that kube-state-metrics needs list and watch permissions granted to `customresourcedefinitions.apiextensions.k8s.io` as well as to the resources you want to gather metrics from.


### Examples

The examples in this section will use the following custom resource:
Expand Down Expand Up @@ -287,7 +286,6 @@ spec:

The above configuration was tested on [this](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/examples/hamster.yaml) VPA configuration, with an added annotation (`foo: 123`).


### Metric types

The configuration supports three kind of metrics from the [OpenMetrics specification](https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md).
Expand Down Expand Up @@ -335,9 +333,9 @@ Supported types are:
* for string the following logic applies
* `"true"` and `"yes"` are mapped to `1.0` and `"false"` and `"no"` are mapped to `0.0` (all case-insensitive)
* RFC3339 times are parsed to float timestamp
* Quantities like "250m" or "512Gi" are parsed to float using https://github.com/kubernetes/apimachinery/blob/master/pkg/api/resource/quantity.go
* Quantities like "250m" or "512Gi" are parsed to float using <https://github.com/kubernetes/apimachinery/blob/master/pkg/api/resource/quantity.go>
* Percentages ending with a "%" are parsed to float
* finally the string is parsed to float using https://pkg.go.dev/strconv#ParseFloat which should support all common number formats. If that fails an error is yielded
* finally the string is parsed to float using <https://pkg.go.dev/strconv#ParseFloat> which should support all common number formats. If that fails an error is yielded

##### Example for status conditions on Kubernetes Controllers

Expand Down Expand Up @@ -368,7 +366,7 @@ spec:
valueFrom: ["status"]
```

This will work for kubernetes controller CRs which expose status conditions according to the kubernetes api (https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Condition):
This will work for kubernetes controller CRs which expose status conditions according to the kubernetes api (<https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1#Condition>):

```yaml
status:
Expand Down Expand Up @@ -462,6 +460,7 @@ spec:
```

Produces:

```prometheus
myteam_foos_uptime{customresource_group="myteam.io", customresource_kind="Foo", customresource_version="v1"} 43.21
```
Expand All @@ -480,6 +479,7 @@ spec:
```

Produces:

```prometheus
uptime{customresource_group="myteam.io", customresource_kind="Foo", customresource_version="v1"} 43.21
```
Expand Down Expand Up @@ -562,5 +562,5 @@ kube_customresource_myobject_info{customresource_group="myteam.io",customresourc

#### Note

- For cases where the GVKs defined in a CRD have multiple versions under a single group for the same kind, as expected, the wildcard value will resolve to *all* versions, but a query for any specific version will return all resources under all versions, in that versions' representation. This basically means that for two such versions `A` and `B`, if a resource exists under `B`, it will reflect in the metrics generated for `A` as well, in addition to any resources of itself, and vice-versa. This logic is based on the [current `list`ing behavior](https://github.com/kubernetes/client-go/issues/1251#issuecomment-1544083071) of the client-go library.
- The introduction of this feature further discourages (and discontinues) the use of native objects in the CRS featureset, since these do not have an explicit CRD associated with them, and conflict with internal stores defined specifically for such native resources. Please consider opening an issue or raising a PR if you'd like to expand on the current metric labelsets for them. Also, any such configuration will be ignored, and no metrics will be generated for the same.
* For cases where the GVKs defined in a CRD have multiple versions under a single group for the same kind, as expected, the wildcard value will resolve to *all* versions, but a query for any specific version will return all resources under all versions, in that versions' representation. This basically means that for two such versions `A` and `B`, if a resource exists under `B`, it will reflect in the metrics generated for `A` as well, in addition to any resources of itself, and vice-versa. This logic is based on the [current `list`ing behavior](https://github.com/kubernetes/client-go/issues/1251#issuecomment-1544083071) of the client-go library.
* The introduction of this feature further discourages (and discontinues) the use of native objects in the CRS featureset, since these do not have an explicit CRD associated with them, and conflict with internal stores defined specifically for such native resources. Please consider opening an issue or raising a PR if you'd like to expand on the current metric labelsets for them. Also, any such configuration will be ignored, and no metrics will be generated for the same.
9 changes: 1 addition & 8 deletions docs/design/metrics-store-performance-optimization.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,15 @@
# Kube-State-Metrics - Performance Optimization Proposal


---

Author: Max Inden (IndenML@gmail.com)
Author: Max Inden (<IndenML@gmail.com>)

Date: 23. July 2018

Target release: v1.5.0

---


## Glossary

- kube-state-metrics: “Simple service that listens to the Kubernetes API server
Expand All @@ -20,7 +18,6 @@ Target release: v1.5.0
- Time series: A single line in a /metrics response e.g.
“metric_name{label="value"} 1”


## Problem Statement

There has been repeated reports of two issues running kube-state-metrics on
Expand All @@ -30,7 +27,6 @@ instances dropping the scrape interval request and marking the given time series
as stale. Second kube-state-metrics uses a lot of memory and thereby being
out-of-memory killed due to low set Kubernetes resource limits.


## Goal

The goal of this proposal can be split into the following sub-goals ordered by
Expand All @@ -40,7 +36,6 @@ their priority:

2. Decrease overall runtime memory usage


## Status Quo

Instead of requesting the needed information from the Kubernetes API-Server on
Expand All @@ -56,7 +51,6 @@ On a scrape request by e.g. Prometheus on the /metrics endpoint
kube-state-metrics calculates the configured time series on demand based on the
objects in its cache and converts them to the Prometheus string representation.


## Proposal

Instead of a full representation of all Kubernetes objects with all its
Expand Down Expand Up @@ -139,7 +133,6 @@ pod_collector -> metrics_endpoint: concat(metrics)

</details>


## FAQ / Follow up improvements

- If kube-state-metrics only listens on add, update and delete events, how is it
Expand Down
3 changes: 1 addition & 2 deletions docs/developer/guide.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Developer Guide
# Developer Guide

This developer guide documentation is intended to assist all contributors in various code contributions.
Any contribution to improving this documentation will be appreciated.
Expand Down Expand Up @@ -30,4 +30,3 @@ The following steps are needed to introduce a new resource and its respective re
|------------------------|--------------------|
| EXPERIMENTAL | basemetrics.ALPHA |
| STABLE | basemetrics.STABLE |

Loading

0 comments on commit efc5dce

Please sign in to comment.