Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify optional Exponential Histogram Aggregation, add example code in the data model #2252

Merged
merged 44 commits into from
May 13, 2022
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
8d31b22
Add more sample code for exponential histogram boundaries; mention su…
Jan 7, 2022
3e01b83
Specify an optional exponential histogram Aggregation with two config…
Jan 7, 2022
aefaed5
changelog entry
Jan 7, 2022
698c2e8
update PR number
Jan 7, 2022
27af2a7
Lint
Jan 7, 2022
f8ae27d
clarify & note
Jan 7, 2022
639d565
markdownlint
Jan 7, 2022
8ed6d94
revisions from code review
Jan 25, 2022
590d483
remove range limits; separate normative text a bit more
Feb 1, 2022
5770ebf
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Feb 1, 2022
a653529
missing add
Feb 1, 2022
6d9baa1
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Feb 11, 2022
513e987
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Feb 28, 2022
854c9a9
lint
Feb 28, 2022
ef363b5
lintier
Feb 28, 2022
8547158
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 1, 2022
499831b
edits
Apr 1, 2022
c49d268
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 1, 2022
dd0e2d6
lint
Apr 1, 2022
bf5c7ab
Merge branch 'main' into jmacd/sdkexpohisto
jmacd Apr 4, 2022
ee22edc
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 4, 2022
7c462f7
Merge branch 'jmacd/sdkexpohisto' of github.com:jmacd/opentelemetry-s…
Apr 4, 2022
77fbca0
use MAY
Apr 4, 2022
59f3a00
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 15, 2022
7952ff4
Merge branch 'main' into jmacd/sdkexpohisto
reyang Apr 18, 2022
3600988
Merge branch 'main' into jmacd/sdkexpohisto
reyang Apr 18, 2022
c5104a4
Merge branch 'main' into jmacd/sdkexpohisto
reyang Apr 19, 2022
f6cf11a
Merge branch 'main' into jmacd/sdkexpohisto
reyang Apr 19, 2022
36e0947
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 21, 2022
34f6166
Apply suggestions from code review
jmacd Apr 21, 2022
559a6cb
lint
Apr 21, 2022
7f46b7c
Merge branch 'jmacd/sdkexpohisto' of github.com:jmacd/opentelemetry-s…
Apr 21, 2022
a89aba9
Update specification/metrics/sdk.md
jmacd Apr 21, 2022
be7d875
yzhuge suggestion
Apr 21, 2022
4f62a4c
Merge branch 'main' into jmacd/sdkexpohisto
reyang Apr 29, 2022
d15ea33
note on inclusivity
Apr 29, 2022
1a473d0
require consistency; do not histogram non-normal values
Apr 29, 2022
a26fc6e
Merge branch 'jmacd/sdkexpohisto' of github.com:jmacd/opentelemetry-s…
Apr 29, 2022
e0cf769
Merge branch 'main' into jmacd/sdkexpohisto
reyang May 3, 2022
bdc6593
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
May 5, 2022
0b32697
Merge branch 'jmacd/sdkexpohisto' of github.com:jmacd/opentelemetry-s…
May 5, 2022
982e015
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
May 10, 2022
6d2edef
Changelog.
May 10, 2022
4b14fed
Merge branch 'main' into jmacd/sdkexpohisto
reyang May 13, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ release.
([#2210](https://github.com/open-telemetry/opentelemetry-specification/pull/2210))
- Use UCUM units in Metrics Semantic Conventions.
([#2199](https://github.com/open-telemetry/opentelemetry-specification/pull/2199))
- Specify optional support for an Exponential Histogram Aggregation.
([#2252](https://github.com/open-telemetry/opentelemetry-specification/pull/2252))

### Logs

Expand Down
65 changes: 59 additions & 6 deletions specification/metrics/datamodel.md
Original file line number Diff line number Diff line change
Expand Up @@ -622,6 +622,21 @@ func GetExponent(value float64) int32 {
}
```

Implementations are permitted to round subnormal values up to the
smallest normal value, which may permit the use of a built-in function:

```golang

func GetExponent(value float64) int {
// Note: Frexp() rounds submnormal values to the smallest normal
// value and returns an exponent corresponding to fractions in the
// range [0.5, 1), whereas we want [1, 2), so subtract 1 from the
// exponent.
_, exp := math.Frexp(value)
return exp - 1
}
```

##### Negative Scale: Extract and Shift the Exponent

For negative scales, the index of a value equals the normalized
Expand All @@ -633,19 +648,57 @@ correct rounding for the negative indices. This may be written as:
return GetExponent(value) >> -scale
```

The reverse mapping function is:

```golang
return math.Ldexp(1, index << -scale)
```

Note that the reverse mapping function is expected to produce
subnormal values even when the mapping function rounds them into
normal values, since the lower boundary of the bucket containing the
smallest normal value may be subnormal. For example, at scale -4 the
smallest normal value `0x1p-1022` falls into a bucket with lower
boundary `0x1p-1024`.

##### All Scales: Use the Logarithm Function

For any scale, use of the built-in natural logarithm
function. A multiplicative factor equal to `2**scale / ln(2)`
proves useful (where `ln()` is the natural logarithm), for example:
For any scale, the built-in natural logarithm function can be used to
compute the bucket index. A multiplicative factor equal to `2**scale
/ ln(2)` proves useful (where `ln()` is the natural logarithm), for
example:

```golang
scaleFactor := math.Log2E * math.Exp2(scale)
return int64(math.Floor(math.Log(value) * scaleFactor))
scaleFactor := math.Ldexp(math.Log2E, scale)
return math.Floor(math.Log(value) * scaleFactor)
```

Note that in the example Golang code above, the built-in `math.Log2E`
is defined as `1 / ln(2)`.
is defined as the inverse natural logarithm of 2.
jmacd marked this conversation as resolved.
Show resolved Hide resolved

The reverse mapping function is:

```golang
inverseFactor := math.Ldexp(math.Ln2, -scale)
return math.Exp(index * inverseFactor), nil
```

Implementations should verify that their mapping function and inverse
mapping function are correct near the lowest and highest IEEE floating
point value. In the Golang reference implementation, for example, the
jmacd marked this conversation as resolved.
Show resolved Hide resolved
above formula computes `+Inf` for the maximum-index bucket. In this
case, it is appropriate to subtract `1<<scale` from the index and
multiply the result by `2`.

jmacd marked this conversation as resolved.
Show resolved Hide resolved
```golang
// Use this form in case the equation above computes +Inf
// as the lower boundary of a valid bucket.
inverseFactor := math.Ldexp(math.Ln2, -scale)
return 2.0 * math.Exp((index - (1 << scale)) * inverseFactor), nil
```

_Note that floating-point to integer type conversions have been
omitted from the code fragments above, to improve readability._

##### Positive Scale: Use a Lookup Table

Expand Down
63 changes: 56 additions & 7 deletions specification/metrics/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,11 @@
+ [Sum Aggregation](#sum-aggregation)
+ [Last Value Aggregation](#last-value-aggregation)
+ [Histogram Aggregation](#histogram-aggregation)
- [Histogram Aggregation common behavior](#histogram-aggregation-common-behavior)
+ [Explicit Bucket Histogram Aggregation](#explicit-bucket-histogram-aggregation)
+ [Exponential Histogram Aggregation](#exponential-histogram-aggregation)
- [Exponential Histogram Aggregation: Handle all normal values](#exponential-histogram-aggregation-handle-all-normal-values)
- [Exponential Histogram Aggregation: Maintain the ideal scale](#exponential-histogram-aggregation-maintain-the-ideal-scale)
- [Attribute limits](#attribute-limits)
- [Exemplar](#exemplar)
* [ExemplarFilter](#exemplarfilter)
Expand Down Expand Up @@ -327,6 +331,10 @@ The SDK MUST provide the following `Aggregation` to support the
- [Histogram](./sdk.md#histogram-aggregation)
- [Explicit Bucket Histogram](./sdk.md#explicit-bucket-histogram-aggregation)

The SDK MAY provide the following `Aggregation`:

- [Exponential Histogram Aggregation](./sdk.md#exponential-histogram-aggregation)

#### Drop Aggregation

The Drop Aggregation informs the SDK to ignore/drop all Instrument Measurements
Expand Down Expand Up @@ -393,6 +401,19 @@ Aggregation](./sdk.md#explicit-bucket-histogram-aggregation).

This Aggregation does not have any configuration parameters.

##### Histogram Aggregation common behavior

Histogram aggregations should not fill out `sum` when used with
jmacd marked this conversation as resolved.
Show resolved Hide resolved
instruments that record negative measurements, e.g. `UpDownCounter` or
`ObservableGauge`.

All histogram Aggregations inform the SDK to collect:

- Count of `Measurement` values in population.
- Arithmetic sum of `Measurement` values in population.
- Min (optional) `Measurement` value in population.
- Max (optional) `Measurement` value in population.

#### Explicit Bucket Histogram Aggregation

The Explicit Bucket Histogram Aggregation informs the SDK to collect data for
Expand All @@ -406,15 +427,43 @@ This Aggregation honors the following configuration parameters:
| Boundaries | double\[\] | [ 0, 5, 10, 25, 50, 75, 100, 250, 500, 1000 ] | Array of increasing values representing explicit bucket boundary values.<br><br>The Default Value represents the following buckets:<br>(-&infin;, 0], (0, 5.0], (5.0, 10.0], (10.0, 25.0], (25.0, 50.0], (50.0, 75.0], (75.0, 100.0], (100.0, 250.0], (250.0, 500.0], (500.0, 1000.0], (1000.0, +&infin;) |
| RecordMinMax | true, false | true | Whether to record min and max. |

Note: This aggregation should not fill out `sum` when used with instruments that
record negative measurements, e.g. `UpDownCounter` or `ObservableGauge`.
Explicit buckets are stated in terms of their upper boundary. Buckets
are exclusive of their lower boundary and inclusive of their upper
bound (except at positive infinity). A measurement is defined to fall
into the least-numbered bucket with boundary that is greater than or
equal to the measurement.

This Aggregation informs the SDK to collect:
#### Exponential Histogram Aggregation

- Count of `Measurement` values falling within explicit bucket boundaries.
- Arithmetic sum of `Measurement` values in population.
- Min (optional) `Measurement` value in population.
- Max (optional) `Measurement` value in population.
The Exponential Histogram Aggregation informs the SDK to collect data
for the [Exponential Histogram Metric
Point](./datamodel.md#exponentialhistogram), which uses an exponential
formula to determine bucket boundaries and an integer `scale`
parameter to control resolution.

This Aggregation honors the following configuration parameters:

| Key | Value | Default Value | Description |
| -- | -- | -- | -- |
| MaxSize | integer | 320 | Maximum number of buckets in each of the positive and negative ranges, not counting the special zero bucket. |
jmacd marked this conversation as resolved.
Show resolved Hide resolved
| RangeLimits (optional) | min, max double | _not set_ | When set, limit the range of positive measurements to the inclusive range [min, max] and limit the range of negative measurements to [-max, -min]. Paired with maximum size, this determines a fixed exponential scale during the constructor. |

##### Exponential Histogram Aggregation: Handle all normal values

When RangeLimits are not set, implementations are REQUIRED to handle
the entire normal range of IEEE floating point values (i.e., all
values except for +Inf, -Inf and NaN values). Implementations are
permitted to round subnormal values away from zero to the nearest
normal value where it simplifies the implementation.

##### Exponential Histogram Aggregation: Maintain the ideal scale

Implementations SHOULD adjust the histogram scale as necessary to
jmacd marked this conversation as resolved.
Show resolved Hide resolved
maintain the best resolution possible, given the maximum size. This
results in histogram Aggregations where at least one of the positive
or negative ranges of buckets is more than half full, such that
increasing scale by one would not be possible given the size
constraint.

## Attribute limits

Expand Down