Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sdk-traces] Add support for OTEL_TRACES_SAMPLER and OTEL_TRACES_SAMPLER_ARG #5448

Merged
merged 29 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
844b103
config driven sampler config with default fallback
matt-hensley Mar 13, 2024
746d139
basic test cases
matt-hensley Mar 13, 2024
e3af439
additional log
matt-hensley Mar 14, 2024
bbc4435
CHANGELOG entry
matt-hensley Mar 14, 2024
de6e89a
nullabe not needed
matt-hensley Mar 14, 2024
0e1d3a7
restructure for logging when sampler was previously set
matt-hensley Mar 14, 2024
cc2578f
log message consistency
matt-hensley Mar 14, 2024
91e2155
make nullable explicit
matt-hensley Mar 14, 2024
9888beb
Update src/OpenTelemetry/Trace/TracerProviderSdk.cs
matt-hensley Mar 14, 2024
f9f9ffa
trailing spaces from GitHub editor
matt-hensley Mar 14, 2024
36d6c99
misc formatting
matt-hensley Mar 14, 2024
236c14a
adjust CHANGELOG entry
matt-hensley Mar 14, 2024
bf84a97
add test to ensure programmatic usage takes precedence
matt-hensley Mar 14, 2024
4f4f7b1
default value for debugging
matt-hensley Mar 15, 2024
4ccfd1d
log type info per feedback
matt-hensley Mar 15, 2024
1b3e472
rename var
matt-hensley Mar 15, 2024
4ac637e
reference env var
matt-hensley Mar 15, 2024
098eff6
rename method
matt-hensley Mar 15, 2024
779467b
if/else -> switch per feedback
matt-hensley Mar 19, 2024
a253144
document supported env vars
matt-hensley Mar 19, 2024
f12142f
document supported values. jaeger_remote and xray are not supported
matt-hensley Mar 19, 2024
96b568f
add dedicated warning logging for invalid values
matt-hensley Mar 19, 2024
8035ca6
copy/paste error
matt-hensley Mar 19, 2024
282ee74
language suggestion
matt-hensley Mar 19, 2024
6aea8ee
nullable warning
matt-hensley Mar 19, 2024
1ea40e8
addtional test
matt-hensley Mar 19, 2024
29bc01e
adjust to always have config keys present
matt-hensley Mar 19, 2024
db9093d
markdown linter error
matt-hensley Mar 19, 2024
bb35bce
Merge branch 'main' into sampler-config
matt-hensley Mar 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/trace/customizing-the-sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,26 @@ var tracerProvider = Sdk.CreateTracerProviderBuilder()
.Build();
```

It is also possible to configure the sampler by using the following
environmental variables:

| Environment variable | Description |
| -------------------------- | -------------------------------------------------- |
| `OTEL_TRACES_SAMPLER` | Sampler to be used for traces. See the [General SDK Configuration specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/configuration/sdk-environment-variables.md#general-sdk-configuration) for more details. |
| `OTEL_TRACES_SAMPLER_ARG` | String value to be used as the sampler argument. |

The supported values for `OTEL_TRACES_SAMPLER` are:

* `always_off`
* `always_on`
* `traceidratio`
* `parentbased_always_on`,
* `parentbased_always_off`
* `parentbased_traceidratio`

The options `traceidratio` and `parentbased_traceidratio` may have the sampler
probability configured via the `OTEL_TRACES_SAMPLER_ARG` environment variable.

Follow [this](../extending-the-sdk/README.md#sampler) document
to learn about writing custom samplers.

Expand Down
10 changes: 10 additions & 0 deletions src/OpenTelemetry/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,16 @@ Released 2024-Mar-14
Specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/configuration/sdk-environment-variables.md#exemplar).
([#5412](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5412))

* `TracerProvider`s can now have a sampler configured via the
matt-hensley marked this conversation as resolved.
Show resolved Hide resolved
`OTEL_TRACES_SAMPLER` environment variable. The supported values are:
`always_off`, `always_on`, `traceidratio`, `parentbased_always_on`,
`parentbased_always_off`, and `parentbased_traceidratio`. The options
`traceidratio` and `parentbased_traceidratio` may have the sampler probability
configured via the `OTEL_TRACES_SAMPLER_ARG` environment variable.
For details see: [OpenTelemetry Environment Variable
Specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/configuration/sdk-environment-variables.md#general-sdk-configuration).
([#5448](https://github.com/open-telemetry/opentelemetry-dotnet/pull/5448))

## 1.7.0

Released 2023-Dec-08
Expand Down
12 changes: 12 additions & 0 deletions src/OpenTelemetry/Internal/OpenTelemetrySdkEventSource.cs
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,18 @@ public void MetricInstrumentRemoved(string instrumentName, string meterName)
this.WriteEvent(53, instrumentName, meterName);
}

[Event(54, Message = "OTEL_TRACES_SAMPLER configuration was found but the value '{0}' is invalid and will be ignored.", Level = EventLevel.Warning)]
public void TracesSamplerConfigInvalid(string configValue)
{
this.WriteEvent(54, configValue);
}

[Event(55, Message = "OTEL_TRACES_SAMPLER_ARG configuration was found but the value '{0}' is invalid and will be ignored, default of value of '1.0' will be used.", Level = EventLevel.Warning)]
public void TracesSamplerArgConfigInvalid(string configValue)
{
this.WriteEvent(55, configValue);
}

#if DEBUG
public class OpenTelemetryEventListener : EventListener
{
Expand Down
81 changes: 80 additions & 1 deletion src/OpenTelemetry/Trace/TracerProviderSdk.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
using System.Runtime.CompilerServices;
using System.Text;
using System.Text.RegularExpressions;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using OpenTelemetry.Internal;
using OpenTelemetry.Resources;
Expand All @@ -13,6 +14,9 @@ namespace OpenTelemetry.Trace;

internal sealed class TracerProviderSdk : TracerProvider
{
internal const string TracesSamplerConfigKey = "OTEL_TRACES_SAMPLER";
internal const string TracesSamplerArgConfigKey = "OTEL_TRACES_SAMPLER_ARG";

internal readonly IServiceProvider ServiceProvider;
internal readonly IDisposable? OwnedServiceProvider;
internal int ShutdownCount;
Expand Down Expand Up @@ -57,7 +61,7 @@ internal TracerProviderSdk(
resourceBuilder.ServiceProvider = serviceProvider;
this.Resource = resourceBuilder.Build();

this.sampler = state.Sampler ?? new ParentBasedSampler(new AlwaysOnSampler());
this.sampler = GetSampler(serviceProvider!.GetRequiredService<IConfiguration>(), state.Sampler);
OpenTelemetrySdkEventSource.Log.TracerProviderSdkEvent($"Sampler added = \"{this.sampler.GetType()}\".");

this.supportLegacyActivity = state.LegacyActivityOperationNames.Count > 0;
Expand Down Expand Up @@ -401,6 +405,81 @@ protected override void Dispose(bool disposing)
base.Dispose(disposing);
}

private static Sampler GetSampler(IConfiguration configuration, Sampler? stateSampler)
{
Sampler? sampler = null;

if (stateSampler != null)
{
sampler = stateSampler;
}

if (configuration.TryGetStringValue(TracesSamplerConfigKey, out var configValue))
{
if (sampler != null)
{
OpenTelemetrySdkEventSource.Log.TracerProviderSdkEvent(
$"Trace sampler configuration value '{configValue}' has been ignored because a value '{sampler.GetType().FullName}' was set programmatically.");
return sampler;
}

switch (configValue)
{
case var _ when string.Equals(configValue, "always_on", StringComparison.OrdinalIgnoreCase):
sampler = new AlwaysOnSampler();
break;
case var _ when string.Equals(configValue, "always_off", StringComparison.OrdinalIgnoreCase):
sampler = new AlwaysOffSampler();
break;
case var _ when string.Equals(configValue, "traceidratio", StringComparison.OrdinalIgnoreCase):
{
var traceIdRatio = ReadTraceIdRatio(configuration);
sampler = new TraceIdRatioBasedSampler(traceIdRatio);
break;
}

case var _ when string.Equals(configValue, "parentbased_always_on", StringComparison.OrdinalIgnoreCase):
sampler = new ParentBasedSampler(new AlwaysOnSampler());
break;
case var _ when string.Equals(configValue, "parentbased_always_off", StringComparison.OrdinalIgnoreCase):
sampler = new ParentBasedSampler(new AlwaysOffSampler());
break;
case var _ when string.Equals(configValue, "parentbased_traceidratio", StringComparison.OrdinalIgnoreCase):
{
var traceIdRatio = ReadTraceIdRatio(configuration);
sampler = new ParentBasedSampler(new TraceIdRatioBasedSampler(traceIdRatio));
break;
}

default:
OpenTelemetrySdkEventSource.Log.TracesSamplerConfigInvalid(configValue ?? string.Empty);
break;
}

if (sampler != null)
utpilla marked this conversation as resolved.
Show resolved Hide resolved
{
OpenTelemetrySdkEventSource.Log.TracerProviderSdkEvent($"Trace sampler set to '{sampler.GetType().FullName}' from configuration.");
}
}

return sampler ?? new ParentBasedSampler(new AlwaysOnSampler());
}

private static double ReadTraceIdRatio(IConfiguration configuration)
matt-hensley marked this conversation as resolved.
Show resolved Hide resolved
{
if (configuration.TryGetStringValue(TracesSamplerArgConfigKey, out var configValue) &&
double.TryParse(configValue, out var traceIdRatio))
{
return traceIdRatio;
}
else
{
OpenTelemetrySdkEventSource.Log.TracesSamplerArgConfigInvalid(configValue ?? string.Empty);
}

return 1.0;
}

private static ActivitySamplingResult ComputeActivitySamplingResult(
ref ActivityCreationOptions<ActivityContext> options,
Sampler sampler)
Expand Down
55 changes: 55 additions & 0 deletions test/OpenTelemetry.Tests/Trace/TracerProviderSdkTest.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
// SPDX-License-Identifier: Apache-2.0

using System.Diagnostics;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using OpenTelemetry.Instrumentation;
using OpenTelemetry.Resources;
using OpenTelemetry.Resources.Tests;
Expand Down Expand Up @@ -1039,6 +1041,59 @@ public void SdkPopulatesSamplingParamsCorrectlyForLegacyActivityWithInProcParent
activity.Stop();
}

[Theory]
[InlineData(null, null, "ParentBased{AlwaysOnSampler}")]
[InlineData("always_on", null, "AlwaysOnSampler")]
[InlineData("always_off", null, "AlwaysOffSampler")]
[InlineData("always_OFF", null, "AlwaysOffSampler")]
[InlineData("traceidratio", "0.5", "TraceIdRatioBasedSampler{0.500000}")]
[InlineData("traceidratio", "not_a_double", "TraceIdRatioBasedSampler{1.000000}")]
[InlineData("parentbased_always_on", null, "ParentBased{AlwaysOnSampler}")]
[InlineData("parentbased_always_off", null, "ParentBased{AlwaysOffSampler}")]
[InlineData("parentbased_traceidratio", "0.111", "ParentBased{TraceIdRatioBasedSampler{0.111000}}")]
matt-hensley marked this conversation as resolved.
Show resolved Hide resolved
[InlineData("parentbased_traceidratio", "not_a_double", "ParentBased{TraceIdRatioBasedSampler{1.000000}}")]
[InlineData("ParentBased_TraceIdRatio", "0.000001", "ParentBased{TraceIdRatioBasedSampler{0.000001}}")]
public void TestSamplerSetFromConfiguration(string configValue, string argValue, string samplerDescription)
{
var configBuilder = new ConfigurationBuilder();

configBuilder.AddInMemoryCollection(new Dictionary<string, string>
{
[TracerProviderSdk.TracesSamplerConfigKey] = configValue,
[TracerProviderSdk.TracesSamplerArgConfigKey] = argValue,
});

var builder = Sdk.CreateTracerProviderBuilder();
builder.ConfigureServices(s => s.AddSingleton<IConfiguration>(configBuilder.Build()));
using var tracerProvider = builder.Build();
var tracerProviderSdk = tracerProvider as TracerProviderSdk;

Assert.NotNull(tracerProviderSdk);
Assert.NotNull(tracerProviderSdk.Sampler);
Assert.Equal(samplerDescription, tracerProviderSdk.Sampler.Description);
}

[Fact]
public void TestSamplerConfigurationIgnoredWhenSetProgrammatically()
{
var configBuilder = new ConfigurationBuilder();
configBuilder.AddInMemoryCollection(new Dictionary<string, string>
{
[TracerProviderSdk.TracesSamplerConfigKey] = "always_off",
});

var builder = Sdk.CreateTracerProviderBuilder();
builder.ConfigureServices(s => s.AddSingleton<IConfiguration>(configBuilder.Build()));
matt-hensley marked this conversation as resolved.
Show resolved Hide resolved
builder.SetSampler(new AlwaysOnSampler());

using var tracerProvider = builder.Build();
var tracerProviderSdk = tracerProvider as TracerProviderSdk;

Assert.NotNull(tracerProviderSdk);
Assert.NotNull(tracerProviderSdk.Sampler);
Assert.Equal("AlwaysOnSampler", tracerProviderSdk.Sampler.Description);
}

[Fact]
public void TracerProvideSdkCreatesAndDiposesInstrumentation()
{
Expand Down