Clarification about histogram metric and buckets

[tbolon]

Hello,

I have a question about current histogram metric and buckets. I was trying to use histogram metrics in a rendering solution (grafana or influxdb) and have hard time trying to obtain percentiles based on current histogram bucket format.

(Note: I have already posted this question on the community discussion, but based on lack of feedback and because it seems the community repo is more about the community around opentelemetry than the opentelemetry project internals, I have decided to cross-post my question here).

It seems the payload is quite different from openmetrics and prometheus:

In OpenMetrics, histogram buckets counts are cumulative: buckets contains count value from interval AND previous buckets.

[...] Buckets MUST be cumulative. As an example for a metric representing request latency in seconds its values for buckets with thresholds 1, 2, 3, and +Inf MUST follow value_1 <= value_2 <= value_3 <= value_+Inf. If ten requests took 1 second each, the values of the 1, 2, 3, and +Inf buckets MUST equal 10.

It's the same in prometheus:

A histogram with a base metric name of <basename> exposes multiple time series during a scrape:

• cumulative counters for the observation buckets, exposed as <basename>_bucket{le="<upper inclusive bound"}
• the total sum of all observed values, exposed as <basename>_sum
• the count of events that have been observed, exposed as <basename>_count (identical to <basename>_bucket{le="+Inf"} above)

But in OpenTelemetry, buckets only contains count inside the bucket interval (see also the spec):

  // bucket_counts is an optional field contains the count values of histogram
  // for each bucket.
  //
  // The sum of the bucket_counts must equal the value in the count field.
  //
  // The number of elements in bucket_counts array must be by one greater than
  // the number of elements in explicit_bounds array.
  repeated fixed64 bucket_counts = 6;

  // explicit_bounds specifies buckets with explicitly defined bounds for values.
  //
  // The boundaries for bucket at index i are:
  //
  // (-infinity, explicit_bounds[i]] for i == 0
  // (explicit_bounds[i-1], explicit_bounds[i]] for 0 < i < size(explicit_bounds)
  // (explicit_bounds[i-1], +infinity) for i == size(explicit_bounds)
  //
  // The values in the explicit_bounds array must be strictly increasing.
  //
  // Histogram buckets are inclusive of their upper boundary, except the last
  // bucket where the boundary is at infinity. This format is intentionally
  // compatible with the OpenMetrics histogram definition.
  repeated double explicit_bounds = 7;

The important part is: The sum of the bucket_counts must equal the value in the count field.

In prometheus there is already some function to handle histogram buckets, where the bucket bound is defined as a tag, and extract percentile information expecting cumulative buckets.

But with this difference, I don't see an easy way to do something useful with common metrics rendering systems, as the bucket system seems to be totally different from other OpenMetrics and prometheus.

Do you have any comment or explanation about this? Is there any reason for having a different format? Are there any examples of end-to-end exploitation of histogram data as actually produced by open-telemetry (for example in grafana)?

Thanks,