Create blog post about Prometheus and OpenTelemetry

[reese-lee]

This is a blog post written by myself and @avillela. It is based on a talk we are delivering at KubeCon EU 2024. It is about how OpenTelemetry and Prometheus can work together, and talks about some of the different components available in OTel for your Prometheus metrics.

---

Preview: https://deploy-preview-4119--opentelemetry.netlify.app/blog/2024/prom-and-otel/

[theletterf]

Just some nits in a first pass. I really like the topic and the content! My only concern is that, from the title, I expected a more conceptual post, but then the post delves into a rather detailed tutorial. Could there be a way of better setting expectations from the title and intro?

[aknuds1]

That's a lot of text :) Please see comments.

[reese-lee]

Also requesting a review from @open-telemetry/operator-ta-maintainers

[reese-lee]

@theletterf I incorporated your comment about setting clearer expectations in the intro. Is this more effective?

While both Prometheus and OTel emit metrics, there is a lot to cover on the
differences and similarities, and is out of scope for this article. Rather, we
want to show you how OTel supports Prometheus, specifically in a Kubernetes
environment. You'll learn:

• How the OTel Collector's Prometheus Receiver can be used to ingest Prometheus
  metrics.
• Alternative methods for Prometheus metric collection through OTel-native
  options such as the K8s cluster receiver and Kubelet stats receiver.

We will also do a technical dive into the OTel Operator's Target Allocator (TA)
and learn:

• How it can be used for Prometheus service discovery.
• How it ensures even distribution of Prometheus targets.

[theletterf]

Love it! Thanks!

[gouthamve]

Thanks for writing this! Looking forward to your Kubecon talk!

I left a few notes that I think better represent the Prometheus capabilities.

[jaronoff97]

Hype!! Thanks for writing this up!

[gouthamve]

Also, this just got published: https://prometheus.io/blog/2024/03/14/commitment-to-opentelemetry/

Maybe at the end we can add reference saying something along the line of "Prometheus is also making it easier to integrate natively with OpenTelemetry and here are their plans."?

[reese-lee]

@gouthamve Gotchu! I added this:

Prometheus maintainers have also been further developing the interoperability
between the two projects from the Prometheus side to make it easier for it to be
the backend for OTLP metrics. For instance, Prometheus can now accept OTLP, and
soon, you’ll be able to use Prometheus exporters to export OTLP. So if you have a
service instrumented with a Prometheus SDK, you’ll be able to push OTLP and
take advantage of the rich Prometheus exporter ecosystem for OTel users. The
maintainers are also working on adding support for delta temporality. This
component will aggregate delta samples to their
respective cumulative counterparts. Read more about Prometheus' support for OTel
here!

[avillela]

@reese-lee added the mermaid diagrams. The closing back-ticks for the mermaid diagrams weren't rendering properly. You may have to manually add them.

[avillela]

Add link to OTel Operator

[svrnm]

@chalin we can point to this PR for the "rebase vs not-rebase" discussion, I am convinced now ;-)

[svrnm]

@open-telemetry/wg-prometheus PTAL

[jaronoff97]

a few minor things :) otherwise this looks great!

[dashpole]

great post!

[svrnm]

@reese-lee please take a look at the outstanding review comments

[svrnm]

@reese-lee @avillela any updates?

[reese-lee]

@svrnm Thank you for your patience! I believe we are good to go on this (finally). I'm not sure what to do about this failing check: "Component owners / Assign component owners (pull_request_target)"?

[svrnm]

@svrnm Thank you for your patience! I believe we are good to go on this (finally). I'm not sure what to do about this failing check: "Component owners / Assign component owners (pull_request_target)"?

let me fix the failing CI checks and then I can merge it:-)