Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[otlp] UseOtlpExporter cross-cutting extension #5400

Merged
merged 61 commits into from
Mar 14, 2024
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
b060f51
OtlpExporter cross-cutting registration extension rough draft.
CodeBlanch Feb 24, 2024
c729fd7
Tweaks and evolving design.
CodeBlanch Feb 26, 2024
fbd97d9
Merge remote-tracking branch 'upstream/main' into otlp-crosscutting-e…
CodeBlanch Feb 28, 2024
ab87eb0
First compiling version.
CodeBlanch Feb 28, 2024
0e9709e
Tweaks.
CodeBlanch Feb 28, 2024
a19a81f
Add concept of weight to BaseProcessor.
CodeBlanch Feb 28, 2024
7e1cffb
Merge remote-tracking branch 'upstream/main' into otlp-crosscutting-e…
CodeBlanch Mar 6, 2024
77e550c
Updates based on discussion.
CodeBlanch Mar 6, 2024
ef6652b
Tweak processor pipeline weight design.
CodeBlanch Mar 7, 2024
ea8d6a1
Revert OtlpExporterOptionsBase refactor to shrink the diff of changes.
CodeBlanch Mar 7, 2024
5512bc7
Merge branch 'main' into otlp-crosscutting-extension
CodeBlanch Mar 7, 2024
b3d9db4
Reduce changes shown on diff.
CodeBlanch Mar 7, 2024
75111d2
Tweaks.
CodeBlanch Mar 7, 2024
c2db100
Code review.
CodeBlanch Mar 7, 2024
cb83c0e
Merge remote-tracking branch 'upstream/main' into otlp-crosscutting-e…
CodeBlanch Mar 7, 2024
1506a34
Merge remote-tracking branch 'upstream/main' into otlp-crosscutting-e…
CodeBlanch Mar 7, 2024
8c896a3
Merge from main.
CodeBlanch Mar 8, 2024
d55e488
Merge from main.
CodeBlanch Mar 8, 2024
2a1046a
Merge from main.
CodeBlanch Mar 8, 2024
15e0272
Merge from main.
CodeBlanch Mar 11, 2024
24cbc81
Merge fixes.
CodeBlanch Mar 11, 2024
2e7811d
Tweaks and fixes.
CodeBlanch Mar 11, 2024
7bdf936
Merge from main.
CodeBlanch Mar 11, 2024
71c56e6
Revert order changes.
CodeBlanch Mar 11, 2024
aa84a40
Tweak.
CodeBlanch Mar 11, 2024
4ba37bb
Tweak.
CodeBlanch Mar 11, 2024
7f3c3cb
API files.
CodeBlanch Mar 11, 2024
ded0687
Bug fix.
CodeBlanch Mar 11, 2024
bc27254
Make processor pipeline weight a constant.
CodeBlanch Mar 11, 2024
57cbc99
Tweak.
CodeBlanch Mar 11, 2024
356628e
Bug fix.
CodeBlanch Mar 12, 2024
3493219
Add tests.
CodeBlanch Mar 12, 2024
15ac47b
CHANGELOG update.
CodeBlanch Mar 12, 2024
f275ea2
Merge branch 'main' into otlp-crosscutting-extension
CodeBlanch Mar 12, 2024
c80c86f
Lint.
CodeBlanch Mar 12, 2024
74f24fe
Warning cleanup.
CodeBlanch Mar 12, 2024
ef38a9d
Warning cleanup.
CodeBlanch Mar 12, 2024
e767069
Docfx fix.
CodeBlanch Mar 12, 2024
f515cc0
Integration test fixes.
CodeBlanch Mar 12, 2024
b145007
Code review.
CodeBlanch Mar 12, 2024
16e0741
JSON config comment update.
CodeBlanch Mar 12, 2024
6f2c8e0
Code review.
CodeBlanch Mar 12, 2024
b75f139
Code review.
CodeBlanch Mar 12, 2024
0faac1a
CHANGELOG tweak.
CodeBlanch Mar 12, 2024
4bef0ef
Ctor tweak.
CodeBlanch Mar 13, 2024
956f537
Test tweaks.
CodeBlanch Mar 13, 2024
2c1cc37
Tweaked exposed overloads and refactored code.
CodeBlanch Mar 13, 2024
e577dc3
Test fixes.
CodeBlanch Mar 13, 2024
c4fe57c
Assert fix.
CodeBlanch Mar 13, 2024
084188a
Add XML docs to capture current naming behaviors.
CodeBlanch Mar 13, 2024
c0d8ba3
Tweak namespace.
CodeBlanch Mar 13, 2024
cc33b5a
Merge branch 'main' into otlp-crosscutting-extension
CodeBlanch Mar 13, 2024
c11e4a2
Warning cleanup.
CodeBlanch Mar 13, 2024
4f06ab8
Code review.
CodeBlanch Mar 13, 2024
cd7ca4e
Remove UseOtlpExporter overload which only changes protocol.
CodeBlanch Mar 13, 2024
01dba40
Code review.
CodeBlanch Mar 13, 2024
5dac98b
Merge from main.
CodeBlanch Mar 13, 2024
1c3258c
Cleanup and refactoring.
CodeBlanch Mar 13, 2024
ed09c06
Code review.
CodeBlanch Mar 13, 2024
31f7759
Refactor spec env var code and add test coverage for UseOtlpExporter …
CodeBlanch Mar 14, 2024
c3e0e37
Add a test collection for tests which modify envvars.
CodeBlanch Mar 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
OpenTelemetry.Exporter.OpenTelemetryBuilderOtlpExporterExtensions
static OpenTelemetry.Exporter.OpenTelemetryBuilderOtlpExporterExtensions.UseOtlpExporter(this OpenTelemetry.IOpenTelemetryBuilder! builder) -> OpenTelemetry.IOpenTelemetryBuilder!
static OpenTelemetry.Exporter.OpenTelemetryBuilderOtlpExporterExtensions.UseOtlpExporter(this OpenTelemetry.IOpenTelemetryBuilder! builder, OpenTelemetry.Exporter.OtlpExportProtocol protocol, System.Uri! baseEndpoint) -> OpenTelemetry.IOpenTelemetryBuilder!
static OpenTelemetry.Exporter.OpenTelemetryBuilderOtlpExporterExtensions.UseOtlpExporter(this OpenTelemetry.IOpenTelemetryBuilder! builder, System.Uri! baseEndpoint) -> OpenTelemetry.IOpenTelemetryBuilder!
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

#nullable enable

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation;
using OpenTelemetry.Internal;
using OpenTelemetry.Logs;
using OpenTelemetry.Metrics;
using OpenTelemetry.Trace;

namespace OpenTelemetry.Exporter;

/// <summary>
/// Contains extension methods to facilitate registration of the OpenTelemetry
/// Protocol (OTLP) exporter into an <see cref="IOpenTelemetryBuilder"/>
/// instance.
/// </summary>
public static class OpenTelemetryBuilderOtlpExporterExtensions
{
private const int DefaultProcessorPipelineWeight = 10_000;

/// <summary>
/// Uses OpenTelemetry Protocol (OTLP) exporter for all signals.
/// </summary>
/// <remarks>
/// Notes:
/// <list type="bullet">
/// <item>Calling this method automatically enables logging, metrics, and
/// tracing.</item>
/// <item>The exporter registered by this method will be added as the last
/// processor in the pipeline established for logging and tracing.</item>
/// <item>This method can only be called once. Subsequent calls will results
/// in a <see cref="NotSupportedException"/> being thrown.</item>
/// <item>This method cannot be called in addition to signal-specific
/// <c>AddOtlpExporter</c> methods. If this method is called signal-specific
/// <c>AddOtlpExporter</c> calls will result in a <see
/// cref="NotSupportedException"/> being thrown.</item>
/// </list>
/// </remarks>
/// <param name="builder"><see cref="IOpenTelemetryBuilder"/>.</param>
/// <returns>Supplied <see cref="IOpenTelemetryBuilder"/> for chaining calls.</returns>
public static IOpenTelemetryBuilder UseOtlpExporter(
this IOpenTelemetryBuilder builder)
=> UseOtlpExporter(builder, name: null);

/// <summary><inheritdoc cref="UseOtlpExporter(IOpenTelemetryBuilder)"/></summary>
/// <remarks><inheritdoc cref="UseOtlpExporter(IOpenTelemetryBuilder)" path="/remarks"/></remarks>
/// <returns><inheritdoc cref="UseOtlpExporter(IOpenTelemetryBuilder)" path="/returns"/></returns>
/// <param name="builder"><see cref="IOpenTelemetryBuilder"/>.</param>
/// <param name="baseEndpoint">The base endpoint to use. A signal-specific
/// path will be appended to the base endpoint for each signal
/// automatically if the protocol is set to <see cref="OtlpExportProtocol.HttpProtobuf"/>.</param>
public static IOpenTelemetryBuilder UseOtlpExporter(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like grpc protocol specific method to me. Do we need this as a separate method?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not following. What do you mean by grpc-specific? At the moment there are 3 overloads exposed:

// Everything is default or comes from IConfiguration/envvars
appBuilder.Services.AddOpenTelemetry()
    .UseOtlpExporter();

// Specify the endpoint. Everything else is default or comes from IConfiguration/envvars
appBuilder.Services.AddOpenTelemetry()
    .UseOtlpExporter(new Uri("http://my_otlp_endpoint/"));

// Specify the protocol + endpoint. Everything else is default or comes from IConfiguration/envvars
appBuilder.Services.AddOpenTelemetry()
    .UseOtlpExporter(OtlpExportProtocol.HttpProtobuf, new Uri("http://my_otlp_endpoint/"));

Copy link
Member

@vishweshbankwar vishweshbankwar Mar 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for this particular case

appBuilder.Services.AddOpenTelemetry()
    .UseOtlpExporter(new Uri("http://my_otlp_endpoint/"));

Are we expecting many users specifying the endpoint here and changing the protocol to http via environment variable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vishweshbankwar

I chatted with @alanwest a bit. We decided to tweak the overloads. They are now:

UseOtlpExporter();
UseOtlpExporter(OtlpExportProtocol protocol);
UseOtlpExporter(OtlpExportProtocol protocol, Uri baseEndpoint);

How you feel about that? TBH we weren't sure exactly what concern you were raising but these seemed more useful in the end 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chatted offline with @vishweshbankwar. We decided to drop the second overload. What we have now is:

UseOtlpExporter();
UseOtlpExporter(OtlpExportProtocol protocol, Uri baseEndpoint);

this IOpenTelemetryBuilder builder,
Uri baseEndpoint)
=> UseOtlpExporter(builder, OtlpExportProtocol.Grpc, baseEndpoint);

/// <summary><inheritdoc cref="UseOtlpExporter(IOpenTelemetryBuilder)"/></summary>
/// <remarks><inheritdoc cref="UseOtlpExporter(IOpenTelemetryBuilder)" path="/remarks"/></remarks>
/// <returns><inheritdoc cref="UseOtlpExporter(IOpenTelemetryBuilder)" path="/returns"/></returns>
/// <param name="builder"><see cref="IOpenTelemetryBuilder"/>.</param>
/// <param name="protocol"><see cref="OtlpExportProtocol"/>.</param>
/// <param name="baseEndpoint"><inheritdoc cref="UseOtlpExporter(IOpenTelemetryBuilder, Uri)" path="/param[@name='baseEndpoint']"/></param>
public static IOpenTelemetryBuilder UseOtlpExporter(
this IOpenTelemetryBuilder builder,
OtlpExportProtocol protocol,
Uri baseEndpoint)
{
Guard.ThrowIfNull(baseEndpoint);

return UseOtlpExporter(builder, name: null, configure: otlpBuilder =>
{
otlpBuilder.ConfigureDefaultExporterOptions(o =>
{
o.Protocol = protocol;
o.Endpoint = baseEndpoint;
});
});
}

internal static IOpenTelemetryBuilder UseOtlpExporter(
this IOpenTelemetryBuilder builder,
Action<OtlpExporterBuilder> configure)
{
Guard.ThrowIfNull(configure);

return UseOtlpExporter(builder, name: null, configure: configure);
}

internal static IOpenTelemetryBuilder UseOtlpExporter(
this IOpenTelemetryBuilder builder,
IConfiguration configuration)
{
Guard.ThrowIfNull(configuration);

return UseOtlpExporter(builder, name: null, configuration: configuration);
}

internal static IOpenTelemetryBuilder UseOtlpExporter(
this IOpenTelemetryBuilder builder,
string? name = null,
IConfiguration? configuration = null,
Action<OtlpExporterBuilder>? configure = null)
{
Guard.ThrowIfNull(builder);

if (configuration != null && string.IsNullOrEmpty(name))
{
name = "otlp";
}

var otlpBuilder = new OtlpExporterBuilder(builder.Services, name, configuration);

configure?.Invoke(otlpBuilder);

UseOtlpExporterInternal(builder, name);

return builder;
}

private static void UseOtlpExporterInternal(IOpenTelemetryBuilder builder, string? name)
{
name ??= Options.DefaultName;

// Note: We automatically turn on signals for "UseOtlpExporter"
builder
.WithLogging()
.WithMetrics()
.WithTracing();

var services = builder.Services;

// Note: UseOtlpExporterRegistration is added to the service collection
// to detect repeated calls to "UseOtlpExporter" and to throw if
// "AddOtlpExporter" extensions are called
services.AddSingleton<UseOtlpExporterRegistration>();

services.RegisterOptionsFactory(configuration => new SdkLimitOptions(configuration));
services.RegisterOptionsFactory(configuration => new ExperimentalOptions(configuration));
services.RegisterOptionsFactory((sp, configuration, name) => new OtlpExporterBuilderOptions(
configuration,
sp.GetRequiredService<IOptionsMonitor<SdkLimitOptions>>().CurrentValue,
sp.GetRequiredService<IOptionsMonitor<ExperimentalOptions>>().CurrentValue,
/* Note: We allow LogRecordExportProcessorOptions,
MetricReaderOptions, & ActivityExportProcessorOptions to be null
because those only exist if the corresponding signal is turned on.
Currently this extension turns on all signals so they will always be
there but that may change in the future so it is handled
defensively. */
sp.GetService<IOptionsMonitor<LogRecordExportProcessorOptions>>()?.Get(name),
sp.GetService<IOptionsMonitor<MetricReaderOptions>>()?.Get(name),
sp.GetService<IOptionsMonitor<ActivityExportProcessorOptions>>()?.Get(name)));

services.ConfigureOpenTelemetryLoggerProvider(
(sp, logging) =>
{
var builderOptions = GetBuilderOptionsAndValidateRegistrations(sp, name);
utpilla marked this conversation as resolved.
Show resolved Hide resolved

var processor = OtlpLogExporterHelperExtensions.BuildOtlpLogExporter(
sp,
builderOptions.LoggingOptionsInstance.ApplyDefaults(builderOptions.DefaultOptionsInstance),
builderOptions.LogRecordExportProcessorOptions ?? throw new InvalidOperationException("LogRecordExportProcessorOptions were missing with logging enabled"),
builderOptions.SdkLimitOptions,
builderOptions.ExperimentalOptions,
skipUseOtlpExporterRegistrationCheck: true);

processor.PipelineWeight = DefaultProcessorPipelineWeight;

logging.AddProcessor(processor);
});

services.ConfigureOpenTelemetryMeterProvider(
(sp, metrics) =>
{
var builderOptions = GetBuilderOptionsAndValidateRegistrations(sp, name);

metrics.AddReader(
OtlpMetricExporterExtensions.BuildOtlpExporterMetricReader(
sp,
builderOptions.MetricsOptionsInstance.ApplyDefaults(builderOptions.DefaultOptionsInstance),
builderOptions.MetricReaderOptions ?? throw new InvalidOperationException("MetricReaderOptions were missing with metrics enabled"),
skipUseOtlpExporterRegistrationCheck: true));
});

services.ConfigureOpenTelemetryTracerProvider(
(sp, tracing) =>
{
var builderOptions = GetBuilderOptionsAndValidateRegistrations(sp, name);

var processorOptions = builderOptions.ActivityExportProcessorOptions ?? throw new InvalidOperationException("ActivityExportProcessorOptions were missing with tracing enabled");

var processor = OtlpTraceExporterHelperExtensions.BuildOtlpExporterProcessor(
sp,
builderOptions.TracingOptionsInstance.ApplyDefaults(builderOptions.DefaultOptionsInstance),
builderOptions.SdkLimitOptions,
processorOptions.ExportProcessorType,
processorOptions.BatchExportProcessorOptions,
skipUseOtlpExporterRegistrationCheck: true);

processor.PipelineWeight = DefaultProcessorPipelineWeight;

tracing.AddProcessor(processor);
});

static OtlpExporterBuilderOptions GetBuilderOptionsAndValidateRegistrations(IServiceProvider sp, string name)
{
sp.EnsureSingleUseOtlpExporterRegistration();

return sp.GetRequiredService<IOptionsMonitor<OtlpExporterBuilderOptions>>().Get(name);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

using Microsoft.Extensions.DependencyInjection;
using OpenTelemetry.Exporter;

namespace System;
CodeBlanch marked this conversation as resolved.
Show resolved Hide resolved

internal static class OpenTelemetryBuilderServiceProviderExtensions
{
public static void EnsureSingleUseOtlpExporterRegistration(this IServiceProvider serviceProvider)
{
var registrations = serviceProvider.GetServices<UseOtlpExporterRegistration>();
if (registrations.Count() > 1)
{
throw new NotSupportedException("Multiple calls to UseOtlpExporter on the same IServiceCollection are not supported.");
}
}

public static void EnsureNoUseOtlpExporterRegistrations(this IServiceProvider serviceProvider)
{
var registrations = serviceProvider.GetServices<UseOtlpExporterRegistration>();
if (registrations.Any())
{
throw new NotSupportedException("Signal-specific AddOtlpExporter methods and the cross-cutting UseOtlpExporter method being invoked on the same IServiceCollection is not supported.");
}
}
}
Loading
Loading