From 574887f19d997c3fa180776eae52f5b32c8191ad Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Thu, 16 May 2024 14:56:23 -0700 Subject: [PATCH 1/8] Add exemplar reservoir docs. --- docs/metrics/customizing-the-sdk/README.md | 39 ++++++++-- docs/metrics/extending-the-sdk/README.md | 85 ++++++++++++++++++++-- 2 files changed, 113 insertions(+), 11 deletions(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 7ed1a73855e..d07db067dca 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -331,6 +331,28 @@ var meterProvider = Sdk.CreateMeterProviderBuilder() .Build(); ``` +### Changing the ExemplarReservoir for a Metric + +To set the [ExemplarReservoir](#exemplarreservoir) for an individual metric, use +the `MetricStreamConfiguration.ExemplarReservoirFactory` property on the View +API: + +> [!NOTE] +> `MetricStreamConfiguration.ExemplarReservoirFactory` is an experimental API only + available in pre-release builds. For details see: + [OTEL1004](../../diagnostics/experimental-apis/OTEL1004.md). + +```csharp +var meterProvider = Sdk.CreateMeterProviderBuilder() + .AddMeter("MyCompany.MyProduct.MyLibrary") + // Use MyCustomExemplarReservoir for "MyFruitCounter" + .AddView( + instrumentName: "MyFruitCounter", + new MetricStreamConfiguration { ExemplarReservoirFactory = () => new MyCustomExemplarReservoir() }) + .AddConsoleExporter() + .Build(); +``` + ### Exemplars Exemplars are example data points for aggregated data. They provide access to @@ -421,20 +443,25 @@ and is responsible for recording `Exemplar`s. The following are the default reservoirs: * `AlignedHistogramBucketExemplarReservoir` is the default reservoir used for -Histograms with buckets, and it stores at most one exemplar per histogram -bucket. The exemplar stored is the last measurement recorded - i.e. any new +Histograms with buckets, and it stores at most one `Exemplar` per histogram +bucket. The `Exemplar` stored is the last measurement recorded - i.e. any new measurement overwrites the previous one in that bucket. * `SimpleFixedSizeExemplarReservoir` is the default reservoir used for all -metrics except Histograms with buckets. It has a fixed reservoir pool, and +metrics except histograms with buckets. It has a fixed reservoir pool, and implements the equivalent of [naive reservoir](https://en.wikipedia.org/wiki/Reservoir_sampling). The reservoir pool -size (currently defaulting to 1) determines the maximum number of exemplars +size (currently defaulting to 1) determines the maximum number of `Exemplar`s stored. Exponential histograms use a `SimpleFixedSizeExemplarReservoir` with a pool size equal to the number of buckets up to a max of `20`. -> [!NOTE] -> Currently there is no ability to change or configure `ExemplarReservoir`. +See [Changing the ExemplarReservoir for a +Metric](#changing-the-exemplarreservoir-for-a-metric) for details on how to use +the View API to change `ExemplarReservoir`s for a Metric. + +See [Building your own exemplar +reservoir](../extending-the-sdk/README.md#exemplarreservoir) for details on how +to implement custom `ExemplarReservoir`s. ### Instrumentation diff --git a/docs/metrics/extending-the-sdk/README.md b/docs/metrics/extending-the-sdk/README.md index c7293ac418a..761009736e3 100644 --- a/docs/metrics/extending-the-sdk/README.md +++ b/docs/metrics/extending-the-sdk/README.md @@ -2,7 +2,6 @@ * [Building your own exporter](#exporter) * [Building your own reader](#reader) -* [Building your own exemplar filter](#exemplarfilter) * [Building your own exemplar reservoir](#exemplarreservoir) * [Building your own resource detector](../../resources/README.md#resource-detector) * [References](#references) @@ -72,12 +71,88 @@ to the `MeterProvider` as shown in the example [here](./Program.cs). Not supported. -## ExemplarFilter +## ExemplarReservoir -Not supported. +> [!NOTE] +> `ExemplarReservoir` is an experimental API only available in pre-release + builds. For details see: + [OTEL1004](../../diagnostics/experimental-apis/OTEL1004.md). -## ExemplarReservoir +Custom [ExemplarReservoir](../customizing-the-sdk/README.md#exemplarreservoir)s +can be implemented to control how `Exemplar`s are recorded for a metric: -Not supported. +* `ExemplarReservoir`s should derive from `FixedSizeExemplarReservoir` (which + belongs to the [OpenTelemetry](../../../src/OpenTelemetry/README.md) package) + and implement the `Offer` methods. +* The `FixedSizeExemplarReservoir` constructor accepts a `capacity` parameter to + control the number of `Exemplar`s which may be recorded by the + `ExemplarReservoir`. +* The `virtual` `OnCollected` method is called after the `ExemplarReservoir` + collection operation has completed and may be used to implement cleanup or + reset logic. +* The `bool` `ResetOnCollect` property on `ExemplarReservoir` is set to `true` + when delta aggregation temporality is used for the metric using the + `ExemplarReservoir`. + +```csharp +class HighestValueExemplarReservoir : FixedSizeExemplarReservoir +{ + private readonly object lockObject = new(); + private long? previousValueLong; + private double? previousValueDouble; + + public HighestValueExemplarReservoir() + : base(capacity: 1) + { + } + + public override void Offer(in ExemplarMeasurement measurement) + { + if (!this.previousValueLong.HasValue || measurement.Value > this.previousValueLong.Value) + { + lock (this.lockObject) + { + if (!this.previousValueLong.HasValue || measurement.Value > this.previousValueLong.Value) + { + this.UpdateExemplar(0, in measurement); + this.previousValueLong = measurement.Value; + } + } + } + } + + public override void Offer(in ExemplarMeasurement measurement) + { + if (!this.previousValueDouble.HasValue || measurement.Value > this.previousValueDouble.Value) + { + lock (this.lockObject) + { + if (!this.previousValueDouble.HasValue || measurement.Value > this.previousValueDouble.Value) + { + this.UpdateExemplar(0, in measurement); + this.previousValueDouble = measurement.Value; + } + } + } + } + + protected override void OnCollected() + { + if (this.ResetOnCollect) + { + lock (this.lockObject) + { + this.previousValueLong = null; + this.previousValueDouble = null; + } + } + } +} +``` + +Custom [ExemplarReservoir](../customizing-the-sdk/README.md#exemplarreservoir)s +can be configured using the View API. For details see: [Changing the +ExemplarReservoir for a +Metric](../customizing-the-sdk/README.md#changing-the-exemplarreservoir-for-a-metric). ## References From f60e07b495a3fcdd621052bcac5fb7d15e78e5f2 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 17 May 2024 12:37:40 -0700 Subject: [PATCH 2/8] Add feedback link. --- docs/diagnostics/experimental-apis/OTEL1004.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/diagnostics/experimental-apis/OTEL1004.md b/docs/diagnostics/experimental-apis/OTEL1004.md index 543d9437bac..a3876df74b9 100644 --- a/docs/diagnostics/experimental-apis/OTEL1004.md +++ b/docs/diagnostics/experimental-apis/OTEL1004.md @@ -54,9 +54,9 @@ We are exposing these APIs experimentally for the following reasons: **TL;DR** We want to gather feedback on the usability of the API and for the need(s) in general for custom reservoirs before exposing a stable API. - +Please provide feedback on [this +issue](https://github.com/open-telemetry/opentelemetry-dotnet/issues/5629) if +you need stable support for custom `ExemplarReservoir`s. The feedback will help +inform decisions about what to expose stable and when. From 572906a111f5859a2fa6192558399b50dd20f132 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 17 May 2024 12:56:56 -0700 Subject: [PATCH 3/8] Code review. --- docs/metrics/extending-the-sdk/README.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/docs/metrics/extending-the-sdk/README.md b/docs/metrics/extending-the-sdk/README.md index 761009736e3..cd77bc1e30f 100644 --- a/docs/metrics/extending-the-sdk/README.md +++ b/docs/metrics/extending-the-sdk/README.md @@ -76,7 +76,9 @@ Not supported. > [!NOTE] > `ExemplarReservoir` is an experimental API only available in pre-release builds. For details see: - [OTEL1004](../../diagnostics/experimental-apis/OTEL1004.md). + [OTEL1004](../../diagnostics/experimental-apis/OTEL1004.md). Please [provide + feedback](https://github.com/open-telemetry/opentelemetry-dotnet/issues/5629) + to help inform decisions about what should be exposed stable and when. Custom [ExemplarReservoir](../customizing-the-sdk/README.md#exemplarreservoir)s can be implemented to control how `Exemplar`s are recorded for a metric: @@ -93,6 +95,20 @@ can be implemented to control how `Exemplar`s are recorded for a metric: * The `bool` `ResetOnCollect` property on `ExemplarReservoir` is set to `true` when delta aggregation temporality is used for the metric using the `ExemplarReservoir`. +* The `Offer` and `Collect` `ExemplarReservoir` methods are called concurrently + by the OpenTelemetry SDK. As such any state required by custom + `ExemplarReservoir` implementations needs to be managed using appropriate + thread-safety/concurrency mechanisms (`lock`, `Interlocked`, etc.). +* Custom `ExemplarReservoir` implementations MUST NOT throw exceptions. + Exceptions thrown in custom implementations MAY lead to unreleased locks and + undefined behaviors. + +The following example demonstrates a custom `ExemplarReservoir` implementation +which records `Exemplar`s for measurements which have the highest value. When +delta aggregation temporality is used the recorded `Exemplar` will be the +highest value for a given collection cycle. When cumulative aggregation +temporality is used the recorded `Exemplar` will be the highest value for the +lifetime of the process. ```csharp class HighestValueExemplarReservoir : FixedSizeExemplarReservoir From f8ed6d33abf79e1708d4240be8c39b413298dcc0 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 17 May 2024 13:17:51 -0700 Subject: [PATCH 4/8] Code review. --- docs/metrics/customizing-the-sdk/README.md | 54 +++++++++++----------- 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index d07db067dca..5ab6d440a08 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -251,6 +251,27 @@ by using Views. See [Program.cs](./Program.cs) for a complete example. +#### Change the ExemplarReservoir + +To set the [ExemplarReservoir](#exemplarreservoir) for an instrument, use the +`MetricStreamConfiguration.ExemplarReservoirFactory` property on the View API: + +> [!NOTE] +> `MetricStreamConfiguration.ExemplarReservoirFactory` is an experimental API only + available in pre-release builds. For details see: + [OTEL1004](../../diagnostics/experimental-apis/OTEL1004.md). + +```csharp +var meterProvider = Sdk.CreateMeterProviderBuilder() + .AddMeter("MyCompany.MyProduct.MyLibrary") + // Use MyCustomExemplarReservoir for "MyFruitCounter" + .AddView( + instrumentName: "MyFruitCounter", + new MetricStreamConfiguration { ExemplarReservoirFactory = () => new MyCustomExemplarReservoir() }) + .AddConsoleExporter() + .Build(); +``` + ### Changing maximum Metric Streams Every instrument results in the creation of a single Metric stream. With Views, @@ -331,28 +352,6 @@ var meterProvider = Sdk.CreateMeterProviderBuilder() .Build(); ``` -### Changing the ExemplarReservoir for a Metric - -To set the [ExemplarReservoir](#exemplarreservoir) for an individual metric, use -the `MetricStreamConfiguration.ExemplarReservoirFactory` property on the View -API: - -> [!NOTE] -> `MetricStreamConfiguration.ExemplarReservoirFactory` is an experimental API only - available in pre-release builds. For details see: - [OTEL1004](../../diagnostics/experimental-apis/OTEL1004.md). - -```csharp -var meterProvider = Sdk.CreateMeterProviderBuilder() - .AddMeter("MyCompany.MyProduct.MyLibrary") - // Use MyCustomExemplarReservoir for "MyFruitCounter" - .AddView( - instrumentName: "MyFruitCounter", - new MetricStreamConfiguration { ExemplarReservoirFactory = () => new MyCustomExemplarReservoir() }) - .AddConsoleExporter() - .Build(); -``` - ### Exemplars Exemplars are example data points for aggregated data. They provide access to @@ -455,13 +454,12 @@ size (currently defaulting to 1) determines the maximum number of `Exemplar`s stored. Exponential histograms use a `SimpleFixedSizeExemplarReservoir` with a pool size equal to the number of buckets up to a max of `20`. -See [Changing the ExemplarReservoir for a -Metric](#changing-the-exemplarreservoir-for-a-metric) for details on how to use -the View API to change `ExemplarReservoir`s for a Metric. +See [Change the ExemplarReservoir](#change-the-exemplarreservoir) for details on +how to use the View API to change `ExemplarReservoir`s for an instrument. -See [Building your own exemplar -reservoir](../extending-the-sdk/README.md#exemplarreservoir) for details on how -to implement custom `ExemplarReservoir`s. +See [Building your own +ExemplarReservoir](../extending-the-sdk/README.md#exemplarreservoir) for details +on how to implement custom `ExemplarReservoir`s. ### Instrumentation From 9dcb5be15ca29b6ec27c1eb8a5cd5a4d46f0918f Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 17 May 2024 13:23:15 -0700 Subject: [PATCH 5/8] Tweak. --- docs/metrics/customizing-the-sdk/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 5ab6d440a08..af4ae372e15 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -261,6 +261,11 @@ To set the [ExemplarReservoir](#exemplarreservoir) for an instrument, use the available in pre-release builds. For details see: [OTEL1004](../../diagnostics/experimental-apis/OTEL1004.md). +> [!IMPORTANT] +> Setting `MetricStreamConfiguration.ExemplarReservoirFactory` alone will NOT + enable `Exemplar`s for an instrument. An [ExemplarFilter](#exemplarfilter) + MUST also be used. + ```csharp var meterProvider = Sdk.CreateMeterProviderBuilder() .AddMeter("MyCompany.MyProduct.MyLibrary") From c1017d54b7103f81b6aa3100ec1146591f8c45f0 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 17 May 2024 13:28:46 -0700 Subject: [PATCH 6/8] Lint. --- docs/metrics/customizing-the-sdk/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index af4ae372e15..4503dd12046 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -253,14 +253,14 @@ See [Program.cs](./Program.cs) for a complete example. #### Change the ExemplarReservoir -To set the [ExemplarReservoir](#exemplarreservoir) for an instrument, use the -`MetricStreamConfiguration.ExemplarReservoirFactory` property on the View API: - > [!NOTE] > `MetricStreamConfiguration.ExemplarReservoirFactory` is an experimental API only available in pre-release builds. For details see: [OTEL1004](../../diagnostics/experimental-apis/OTEL1004.md). +To set the [ExemplarReservoir](#exemplarreservoir) for an instrument, use the +`MetricStreamConfiguration.ExemplarReservoirFactory` property on the View API: + > [!IMPORTANT] > Setting `MetricStreamConfiguration.ExemplarReservoirFactory` alone will NOT enable `Exemplar`s for an instrument. An [ExemplarFilter](#exemplarfilter) From 15e16abb9e88d88395c5301ed80550ab0631f750 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 17 May 2024 13:37:14 -0700 Subject: [PATCH 7/8] Link fix. --- docs/metrics/extending-the-sdk/README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/metrics/extending-the-sdk/README.md b/docs/metrics/extending-the-sdk/README.md index cd77bc1e30f..bf32c3368ff 100644 --- a/docs/metrics/extending-the-sdk/README.md +++ b/docs/metrics/extending-the-sdk/README.md @@ -167,8 +167,7 @@ class HighestValueExemplarReservoir : FixedSizeExemplarReservoir ``` Custom [ExemplarReservoir](../customizing-the-sdk/README.md#exemplarreservoir)s -can be configured using the View API. For details see: [Changing the -ExemplarReservoir for a -Metric](../customizing-the-sdk/README.md#changing-the-exemplarreservoir-for-a-metric). +can be configured using the View API. For details see: [Change the +ExemplarReservoir](../customizing-the-sdk/README.md#change-the-exemplarreservoir). ## References From 5431913b10fed14fd196a8936379e5216ca2acf4 Mon Sep 17 00:00:00 2001 From: Mikel Blanchard Date: Fri, 17 May 2024 13:51:41 -0700 Subject: [PATCH 8/8] Code review. --- docs/metrics/customizing-the-sdk/README.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/docs/metrics/customizing-the-sdk/README.md b/docs/metrics/customizing-the-sdk/README.md index 4503dd12046..830ad66957c 100644 --- a/docs/metrics/customizing-the-sdk/README.md +++ b/docs/metrics/customizing-the-sdk/README.md @@ -239,7 +239,7 @@ within the maximum number of buckets defined by `MaxSize`. The default `MaxSize` is 160 buckets and the default `MaxScale` is 20. ```csharp - // Change the maximum number of buckets + // Change the maximum number of buckets for "MyHistogram" .AddView( instrumentName: "MyHistogram", new Base2ExponentialBucketHistogramConfiguration { MaxSize = 40 }) @@ -267,14 +267,10 @@ To set the [ExemplarReservoir](#exemplarreservoir) for an instrument, use the MUST also be used. ```csharp -var meterProvider = Sdk.CreateMeterProviderBuilder() - .AddMeter("MyCompany.MyProduct.MyLibrary") // Use MyCustomExemplarReservoir for "MyFruitCounter" .AddView( instrumentName: "MyFruitCounter", new MetricStreamConfiguration { ExemplarReservoirFactory = () => new MyCustomExemplarReservoir() }) - .AddConsoleExporter() - .Build(); ``` ### Changing maximum Metric Streams