Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics not generated #847

Closed
jvanz opened this issue Jul 26, 2024 · 3 comments · Fixed by #848
Closed

Metrics not generated #847

jvanz opened this issue Jul 26, 2024 · 3 comments · Fixed by #848
Assignees
Labels
area/observability Monitoring, logging, tracing kind/bug
Milestone

Comments

@jvanz
Copy link
Member

jvanz commented Jul 26, 2024

During the testing of the Kubewarden v1.15.0 release candidates we noticed that policy server is not generating metrics. When the metrics are fetched from the OpenTelemetry collector endpoint the returned file is empty. We need to investigate if this is related to the recent libraries updates or is something else.

The metrics can be fetched using a command like this:

kubectl run curlpod -t -i --rm --wait --image curlimages/curl:8.00.1 --restart=Never -- --silent policy-server-default.kubewarden.svc.cluster.local:8080/metrics

Acceptance criteria

  • Investigate why the metrics are not being generated
  • Fix policy server metrics to generate and exposed in the OpTel collector endpoint
@jvanz jvanz added kind/bug area/observability Monitoring, logging, tracing labels Jul 26, 2024
@jvanz jvanz moved this to Todo in Kubewarden Jul 26, 2024
@jvanz jvanz added this to the 1.15 milestone Jul 26, 2024
@jvanz jvanz self-assigned this Jul 26, 2024
@jvanz jvanz moved this from Todo to In Progress in Kubewarden Jul 26, 2024
@viccuad
Copy link
Member

viccuad commented Jul 26, 2024

Adding info:
policy-server-1.15.0-rc2 tracing work, but metrics don't work by following the steps in docs.kubewarden.io. Neither with the deps versions of docs.kubewarden.io (opentelemetry-operator-0.56.0 , jaeger-operator-2.49.0 , kube-prometheus-stack-51.5.3), nor with latest at the time of writing opentelemetry-operator-0.64.4, jaeger-operator-2.54.0, kube-prometheus-stack-61.3.2.

policy-server-1.14.0 tracing and metrics work with the steps and versions of docs.kubewarden.io.

@viccuad
Copy link
Member

viccuad commented Jul 26, 2024

Another data point.
Having a completely working Kubewarden 1.14 with the deps from docs, I started upgrading to 1.15.0-rc2. Using chart kubewarden-controller-1.15.0-rc2 with kubewarden-defaults-v1.14.0 works. Upgrading to kubewarden-defaults-v1.15.0-rc2 breaks metrics.
Which corroborates the theory that policy-server metrics are broken in rc2 and the controller is fine.

@jvanz
Copy link
Member Author

jvanz commented Jul 26, 2024

@fabriziosestito @viccuad thanks for the fix! Furthermore, sorry for missing this issue when I've updated the libraries. I've testing the tracing only and did not read properly the changelog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/observability Monitoring, logging, tracing kind/bug
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants