Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Only keep certain K8s API server metrics by default #384

Merged
merged 5 commits into from
Feb 4, 2021
Merged

Only keep certain K8s API server metrics by default #384

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3051939
Reduce default set of kept K8s API server metrics
hjet Feb 4, 2021
6b76e1c
Update changelog
hjet Feb 4, 2021
f5b437d
regenerate manifests
rfratto Feb 4, 2021
bc60b9b
Update CHANGELOG
hjet Feb 4, 2021
a956117
Merge PR changes
hjet Feb 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ files to the new format.
that updating a config is guaranteed to always hash to the same Agent,
reducing the number of metrics gaps. (@rfratto)

- [ENHANCEMENT] Only keep a handful of K8s API server metrics by default to reduce
default active series usage. (@hjet)

- [BUGFIX] `agentctl config-check` will now work correctly when the supplied
config file contains integrations. (@hoenn)

Expand Down
8 changes: 2 additions & 6 deletions production/kubernetes/agent-sigv4.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,12 +207,8 @@ data:
kubernetes_sd_configs:
- role: endpoints
metric_relabel_configs:
- action: drop
regex: apiserver_admission_controller_admission_latencies_seconds_.*
source_labels:
- __name__
- action: drop
regex: apiserver_admission_step_admission_latencies_seconds_.*
- action: keep
regex: workqueue_queue_duration_seconds_bucket|process_cpu_seconds_total|process_resident_memory_bytes|workqueue_depth|rest_client_request_duration_seconds_bucket|workqueue_adds_total|up|rest_client_requests_total|apiserver_request_total|go_goroutines
source_labels:
- __name__
relabel_configs:
Expand Down
8 changes: 2 additions & 6 deletions production/kubernetes/agent.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,12 +205,8 @@ data:
kubernetes_sd_configs:
- role: endpoints
metric_relabel_configs:
- action: drop
regex: apiserver_admission_controller_admission_latencies_seconds_.*
source_labels:
- __name__
- action: drop
regex: apiserver_admission_step_admission_latencies_seconds_.*
- action: keep
regex: workqueue_queue_duration_seconds_bucket|process_cpu_seconds_total|process_resident_memory_bytes|workqueue_depth|rest_client_request_duration_seconds_bucket|workqueue_adds_total|up|rest_client_requests_total|apiserver_request_total|go_goroutines
source_labels:
- __name__
relabel_configs:
Expand Down
11 changes: 3 additions & 8 deletions production/tanka/grafana-agent/config.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,17 +121,12 @@
action: 'keep',
}],

// Drop some high cardinality metrics.
// Keep limited set of metrics to reduce default usage, drop all others
metric_relabel_configs: [
{
source_labels: ['__name__'],
regex: 'apiserver_admission_controller_admission_latencies_seconds_.*',
action: 'drop',
},
{
source_labels: ['__name__'],
regex: 'apiserver_admission_step_admission_latencies_seconds_.*',
action: 'drop',
regex: 'workqueue_queue_duration_seconds_bucket|process_cpu_seconds_total|process_resident_memory_bytes|workqueue_depth|rest_client_request_duration_seconds_bucket|workqueue_adds_total|up|rest_client_requests_total|apiserver_request_total|go_goroutines',
action: 'keep',
},
],

Expand Down
11 changes: 3 additions & 8 deletions production/tanka/grafana-agent/v1/internal/kubernetes_instance.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,12 @@ local k8s_tls_config(config) = {
action: 'keep',
}],

// Drop some high cardinality metrics.
// Keep limited set of metrics to reduce default usage, drop all others
metric_relabel_configs: [
{
source_labels: ['__name__'],
regex: 'apiserver_admission_controller_admission_latencies_seconds_.*',
action: 'drop',
},
{
source_labels: ['__name__'],
regex: 'apiserver_admission_step_admission_latencies_seconds_.*',
action: 'drop',
regex: 'workqueue_queue_duration_seconds_bucket|process_cpu_seconds_total|process_resident_memory_bytes|workqueue_depth|rest_client_request_duration_seconds_bucket|workqueue_adds_total|up|rest_client_requests_total|apiserver_request_total|go_goroutines',
action: 'keep',
},
],
},
Expand Down