prometheus missing labels in 0.32.0

[skadefro]

Describe the bug
Up til 0.31.0 prometheusremotewrite and prometheus have worked with labels.
But when using the docker image for 0.32.0 ( or latest ) the labels are gone

Steps to reproduce
Upgrade to 0.32.0 and with

  prometheus:
   endpoint: "0.0.0.0:9464"
  prometheusremotewrite:
    endpoint: "http://victoriametrics:8428/api/v1/write"

in the config, all metrics are missing labels

What did you expect to see?
the labels
What did you see instead?
no labels

What version did you use?
Version: (e.g., v0.4.0, 1eb551b, etc)

What config did you use?

receivers:
  otlp:
    protocols:
      grpc:
      http:
        cors_allowed_origins:
        - http://*
        - https://*
processors:
  batch:
extensions:
  health_check: {}
  zpages: {}
exporters:
  jaeger:
    endpoint: jaeger:14250
    insecure: true
  logging:
   loglevel: info
  prometheus:
   endpoint: "0.0.0.0:9464"
  prometheusremotewrite:
    endpoint: "http://victoriametrics:8428/api/v1/write"
service:
  extensions: [health_check, zpages]
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [prometheusremotewrite, logging, prometheus]
      processors: []
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger, logging]
     

Environment
GKE

Additional context

[bogdandrutu]

@codeboten this is because #3770

[skadefro]

Not sure what 3770 means. So is this a bug, or a decision to no longer support prometheusremotewrite and prometheus ?

[bogdandrutu]

That is a PR number, that was merged and released without updating the promotheus components.

This is a bug on our side.

[skadefro]

Perfect, thank you for all the work you guys are doing, and fast feedback. 😊

[codeboten]

I just tested this w/ 0.32.0 and compared the results w/ main. I believe this is now fixed in main with e084857 for the OTLP receiver.

The problem is that the prometheus exporter and prometheus remote write exporters were released including the PR that @bogdandrutu mentioned earlier in this thread, which causes them to use OTLP Metric Attributes instead of Labels, but the OTLP wrapper code to translate Labels into Attributes wasn't part of the 0.32.0 release.

[bogdandrutu]

@skadefro can you help us, and run the latest build (from head) to check if this got fixed

[skadefro]

I would love too, but i only see 0.32.0 in docker hub, and I don't know how to create an image my self. If you could push an :edge or :0.33.0 that would help

[bogdandrutu]

Here is the image (as -dev) https://hub.docker.com/r/otel/opentelemetry-collector-dev/tags

[bogdandrutu]

More exactly try this one https://hub.docker.com/layers/otel/opentelemetry-collector-dev/7d0c66d793200ac0c206521edaf025d065cbda50/images/sha256-ae0eca0cb32623f3b5f91ecdad1bc4756ddd42fd90feac4577f4f07536a4a807?context=explore

[skadefro]

I just tested the above image. And everything is working again for me :-)

[bogdandrutu]

v0.33.0 releasing now