secrets dynatrace-operator-otel-config not found

[whigley]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:
Follow the installation steps provided at the Quick-Start

Expected behavior
Install Dynatrace operator and webhook

Error
{"level":"info","logLevel":"info","msg":"logging level","ts":"2024-05-13T20:24:42.156Z"}
{"buildDate":"2024-05-07T07:46:47+00:00","gitCommit":"7d0913a83292aaff2728ff23c874fea19f6aea15","goVersion":"go1.22.2","level":"info","logger":"version","msg":"dynatrace-operator","platform":"linux/amd64","ts":"2024-05-13T20:24:42.159Z","version":"v1.1.0"}
{"err":"secrets "dynatrace-operator-otel-config" not found","level":"info","logger":"open-telemetry","msg":"failed to read OpenTelementry config secret","ts":"2024-05-13T20:24:42.189Z"}
{"level":"info","logger":"open-telemetry","msg":"use Noop providers for OpenTelemetry","ts":"2024-05-13T20:24:42.189Z"}
{"level":"info","logger":"main.controller-runtime.webhook","msg":"Registering webhook","path":"/label-ns","ts":"2024-05-13T20:24:42.198Z"}
Error: unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: stale GroupVersion discovery: metrics.k8s.io/v1beta1
Usage:
dynatrace-operator webhook-server [flags]

Flags:
--cert string File name for the public certificate. (default "tls.crt")
--cert-key string File name for the private key. (default "tls.key")
--certs-dir string Directory to look certificates for. (default "/tmp/webhook/certs")
-h, --help help for webhook-server

{"level":"info","logger":"main","msg":"unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: stale GroupVersion discovery: metrics.k8s.io/v1beta1","ts":"2024-05-13T20:24:42.211Z"}

Additional context
AWS EKS 1.29
"dynatrace-operator pods" are online, but "dynatrace-webhook" pods keep crashing with the logs described above.

NAME READY STATUS RESTARTS AGE
dynatrace-oneagent-csi-driver-429r7 4/4 Running 0 3h27m
dynatrace-oneagent-csi-driver-swndr 4/4 Running 0 3h27m
dynatrace-operator-58596f664f-pfwrs 1/1 Running 0 3h28m
dynatrace-webhook-86bdd79d8d-snqdg 0/1 CrashLoopBackOff 1 (8s ago) 12s
dynatrace-webhook-86bdd79d8d-zmrwx 0/1 CrashLoopBackOff 1 (8s ago) 12s

[chrismuellner]

The following error log is not blocking and just points out missing, optional configuration.

{"err":"secrets "dynatrace-operator-otel-config" not found","level":"info","logger":"open-telemetry","msg":"failed to read OpenTelementry config secret","ts":"2024-05-13T20:24:42.189Z"}

Thanks for pointing that out! We'll update the log to make that more clear.

The underlying issue seems to be a setup issue, so please reach out to support for that!

[github-actions]

Thank you for opening a Dynatrace Operator Issue. We've identified and tagged the issue as a "Support request".

Dynatrace responds to requests like these via Dynatrace ONE support rather than Github. This helps our team respond as quickly as possible using the support team's tools and procedures.

Thanks for your help!

[whigley]

Hello,
Does this optional configuration causes pod to crash on startup?

dynatrace-webhook-86bdd79d8d-snqdg 0/1 CrashLoopBackOff 1 (8s ago) 12s
dynatrace-webhook-86bdd79d8d-zmrwx 0/1 CrashLoopBackOff 1 (8s ago) 12s

[wangdong2023]

We met the same error on Huawei cloud, and our pod also starts to crash