-
Notifications
You must be signed in to change notification settings - Fork 40
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add example otel-collector configurations
- Loading branch information
1 parent
64e8f7e
commit 5b2808b
Showing
3 changed files
with
225 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
# Purpose of this directory | ||
|
||
This folder provides sample [custom monitoring configurations](http://cloud/anthos-config-management/docs/how-to/monitor-config-sync-custom) | ||
for Config Sync. These examples are intended for your convenience. While Config | ||
Sync strives to keep them updated, always consult the [latest configuration](https://github.com/GoogleContainerTools/kpt-config-sync/blob/main/pkg/metrics/otel.go). | ||
|
||
# Available Configurations | ||
|
||
_otel-collector-monarch.yaml_: Serves as a template for exporting metrics | ||
exclusively to Google Cloud Monarch and Prometheus. | ||
|
||
_otel-collector-prometheus.yaml_: Serves as a template for exporting metrics | ||
exclusively to Prometheus. | ||
|
||
# Instructions | ||
|
||
* **Apply ConfigMap**: Apply the desired ConfigMap to your cluster. | ||
* **Restart otel-collector**: The otel-collector deployment should restart automatically. If it doesn't, execute: | ||
``` | ||
kubectl rollout restart deployment otel-collector -n config-management-monitoring | ||
``` | ||
# Removal | ||
|
||
* **Delete ConfigMap**: Remove the ConfigMap from your cluster. | ||
* **Restart otel-collector**: The otel-collector deployment should restart automatically. If it doesn't, use the same kubectl rollout restart command as above. |
152 changes: 152 additions & 0 deletions
152
examples/otel-collector-sample-configurations/otel-collector-monarch.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,152 @@ | ||
# Copyright 2024 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
labels: | ||
app: opentelemetry | ||
component: otel-collector | ||
configmanagement.gke.io/arch: csmr | ||
configmanagement.gke.io/system: "true" | ||
name: otel-collector-custom | ||
namespace: config-management-monitoring | ||
data: | ||
otel-collector-config.yaml: |- | ||
receivers: | ||
opencensus: | ||
exporters: | ||
prometheus: | ||
endpoint: :8675 | ||
namespace: config_sync | ||
resource_to_telemetry_conversion: | ||
enabled: true | ||
googlecloud/kubernetes: | ||
metric: | ||
prefix: "kubernetes.io/internal/addons/config_sync/" | ||
# skip_create_descriptor: Metrics start with 'kubernetes.io/' have already | ||
# got descriptors defined internally. Skip sending dupeicated metric | ||
# descriptors here to prevent errors or conflicts. | ||
skip_create_descriptor: true | ||
# instrumentation_library_labels: Otel Collector by default attaches | ||
# 'instrumentation_version' and 'instrumentation_source' labels that are | ||
# not specified in our Cloud Monarch definitions, thus skipping them here | ||
instrumentation_library_labels: false | ||
# create_service_timeseries: This is a recommended configuration for | ||
# 'service metrics' starts with 'kubernetes.io/' prefix. It uses | ||
# CreateTimeSeries API and has its own quotas, so that custom metric write | ||
# will not break this ingestion pipeline | ||
create_service_timeseries: true | ||
service_resource_labels: false | ||
retry_on_failure: | ||
enabled: false | ||
sending_queue: | ||
enabled: false | ||
processors: | ||
batch: | ||
# resourcedetection: This processor is needed to correctly mirror resource | ||
# labels from OpenCensus to OpenTelemetry. We also want to keep this same | ||
# processor in Otel Agent configuration as the resource labels are added from | ||
# there | ||
resourcedetection: | ||
detectors: [env, gcp] | ||
filter/kubernetes: | ||
metrics: | ||
include: | ||
match_type: regexp | ||
metric_names: | ||
- kustomize.* | ||
- api_duration_seconds | ||
- reconciler_errors | ||
- pipeline_error_observed | ||
- reconcile_duration_seconds | ||
- rg_reconcile_duration_seconds | ||
- parser_duration_seconds | ||
- declared_resources | ||
- apply_operations_total | ||
- apply_duration_seconds | ||
- resource_fights_total | ||
- remediate_duration_seconds | ||
- resource_conflicts_total | ||
- internal_errors_total | ||
- rendering_count_total | ||
- skip_rendering_count_total | ||
- resource_override_count_total | ||
- git_sync_depth_override_count_total | ||
- no_ssl_verify_count_total | ||
- kcc_resource_count | ||
- last_sync_timestamp | ||
# Remove custom configsync metric labels that are not registered with Monarch | ||
# This action applies to all metrics that are sent through the pipeline that | ||
# is using this processor | ||
attributes/kubernetes: | ||
actions: | ||
- key: configsync.sync.kind | ||
action: delete | ||
- key: configsync.sync.name | ||
action: delete | ||
- key: configsync.sync.namespace | ||
action: delete | ||
- key: commit | ||
action: delete | ||
metricstransform/kubernetes: | ||
transforms: | ||
- include: declared_resources | ||
action: update | ||
new_name: current_declared_resources | ||
- include: reconciler_errors | ||
action: update | ||
new_name: last_reconciler_errors | ||
- include: pipeline_error_observed | ||
action: update | ||
new_name: last_pipeline_error_observed | ||
- include: apply_operations_total | ||
action: update | ||
new_name: apply_operations_count | ||
- include: resource_fights_total | ||
action: update | ||
new_name: resource_fights_count | ||
- include: resource_conflicts_total | ||
action: update | ||
new_name: resource_conflicts_count | ||
- include: internal_errors_total | ||
action: update | ||
new_name: internal_errors_count | ||
- include: rendering_count_total | ||
action: update | ||
new_name: rendering_count | ||
- include: skip_rendering_count_total | ||
action: update | ||
new_name: skip_rendering_count | ||
- include: resource_override_count_total | ||
action: update | ||
new_name: resource_override_count | ||
- include: git_sync_depth_override_count_total | ||
action: update | ||
new_name: git_sync_depth_override_count | ||
- include: no_ssl_verify_count_total | ||
action: update | ||
new_name: no_ssl_verify_count | ||
extensions: | ||
health_check: | ||
service: | ||
extensions: [health_check] | ||
pipelines: | ||
metrics/prometheus: | ||
receivers: [opencensus] | ||
processors: [batch] | ||
exporters: [prometheus] | ||
metrics/kubernetes: | ||
receivers: [opencensus] | ||
processors: [batch, filter/kubernetes, attributes/kubernetes, metricstransform/kubernetes, resourcedetection] | ||
exporters: [googlecloud/kubernetes] |
48 changes: 48 additions & 0 deletions
48
examples/otel-collector-sample-configurations/otel-collector-prometheus.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
# Copyright 2024 Google LLC | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
labels: | ||
app: opentelemetry | ||
component: otel-collector | ||
configmanagement.gke.io/arch: csmr | ||
configmanagement.gke.io/system: "true" | ||
name: otel-collector-custom | ||
namespace: config-management-monitoring | ||
data: | ||
otel-collector-config.yaml: |- | ||
receivers: | ||
opencensus: | ||
exporters: | ||
prometheus: | ||
endpoint: :8675 | ||
namespace: config_sync | ||
resource_to_telemetry_conversion: | ||
enabled: true | ||
processors: | ||
batch: | ||
# resourcedetection: This processor is needed to correctly mirror resource | ||
# labels from OpenCensus to OpenTelemetry. We also want to keep this same | ||
# processor in Otel Agent configuration as the resource labels are added from | ||
# there | ||
extensions: | ||
health_check: | ||
service: | ||
extensions: [health_check] | ||
pipelines: | ||
metrics/prometheus: | ||
receivers: [opencensus] | ||
processors: [batch] | ||
exporters: [prometheus] |