[SDK + Otlp] Support loading envvars from IConfiguration #3760

CodeBlanch · 2022-10-12T21:30:28Z

Relates to #2980
Relates to #3720
Relates to #3639

Changes

Rename SdkConfiguration -> SdkLimitOptions & remove static coupling.
Support loading envvars for OtlpExporterOptions, SdkLimitOptions, & MetricReaderOptions from IConfiguration

TODOs

Appropriate CHANGELOG.md updated for non-trivial changes
Unit tests
Changes in public API reviewed

…ation.

codecov · 2022-10-12T21:41:15Z

Codecov Report

Merging #3760 (41c55d1) into main (0feebf3) will increase coverage by 0.17%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3760      +/-   ##
==========================================
+ Coverage   87.04%   87.21%   +0.17%     
==========================================
  Files         278      277       -1     
  Lines       10167    10209      +42     
==========================================
+ Hits         8850     8904      +54     
+ Misses       1317     1305      -12

Impacted Files	Coverage Δ
...metryProtocol/Implementation/ActivityExtensions.cs	`91.89% <100.00%> (+0.13%)`	⬆️
...etryProtocol/Implementation/LogRecordExtensions.cs	`85.88% <100.00%> (ø)`
...elemetryProtocol/Implementation/SdkLimitOptions.cs	`100.00% <100.00%> (ø)`
...orter.OpenTelemetryProtocol/OtlpExporterOptions.cs	`100.00% <100.00%> (ø)`
....Exporter.OpenTelemetryProtocol/OtlpLogExporter.cs	`75.00% <100.00%> (+5.00%)`	⬆️
...nTelemetryProtocol/OtlpMetricExporterExtensions.cs	`95.55% <100.00%> (+1.43%)`	⬆️
...xporter.OpenTelemetryProtocol/OtlpTraceExporter.cs	`65.38% <100.00%> (+29.02%)`	⬆️
...metryProtocol/OtlpTraceExporterHelperExtensions.cs	`94.73% <100.00%> (+2.73%)`	⬆️
...lder/ProviderBuilderServiceCollectionExtensions.cs	`100.00% <100.00%> (ø)`
...ry/Internal/PeriodicExportingMetricReaderHelper.cs	`100.00% <100.00%> (ø)`
... and 10 more

src/OpenTelemetry/.publicApi/netstandard2.1/PublicAPI.Unshipped.txt

src/OpenTelemetry/Metrics/PeriodicExportingMetricReaderOptions.cs

test/OpenTelemetry.Tests/SdkOptionsTests.cs

cijothomas · 2022-10-14T16:24:57Z

src/OpenTelemetry/SdkOptions.cs

+namespace OpenTelemetry;
+
+/// <summary>
+/// Contains options for the OpenTelemetry SDK.


How would users use this feature? Its a bit misleading, as the SDK itself does not enforce any of the limits from this class...

I reverted the SdkOptions changes on this PR. But I don't necessarily have an issue with the above. SDK is doing many things without enforcement! For example, up to each exporter author to correctly support every feature of SDK (Resource, Scopes, Histograms, etc.) 🤣 You know how the spec has the compliance matrix? Maybe we should have one too just for the different .NET bits floating around? Maintenance headache but there really should be some way for users to check what parts of the spec/SDK the thing they are consuming supports.

…guration

src/OpenTelemetry.Exporter.OpenTelemetryProtocol/OtlpLogExporter.cs

alanwest · 2022-10-17T21:54:19Z

src/OpenTelemetry.Exporter.OpenTelemetryProtocol/OtlpTraceExporterHelperExtensions.cs

+                }
+
+                services.RegisterOptionsFactory(configuration => new SdkLimitOptions(configuration));
+                services.RegisterOptionsFactory(configuration => new OtlpExporterOptions(configuration));


Just to clarify my understanding of things...

Let's say I set OTEL_EXPORTER_OTLP_ENDPOINT=http://my-collector.com and I also have some json config like:

{ "MyOtlpExporterOptions": { "Headers": "api-key=abc123" } }

Then I do this somewhere.

builder.Services.Configure<OtlpExporterOptions>(builder.Configuration.GetSection("MyOtlpExporterOptions"));

Will I get the headers setting from the json and the endpoint setting from the environment variable?

Will I get the headers setting from the json and the endpoint setting from the environment variable?

Yes!

alanwest

Just testing my understanding of things with a somewhat contrived question... but this LGTM!

* [Logs] Fix buffered log scopes being reused (#3731) * Fix buffered log scopes being reused. * CHANGELOG update. * Test fixes. Co-authored-by: Cijo Thomas <cithomas@microsoft.com> * clarify that Prometheus HttpListner is not for production at this moment (#3737) * [HttpClient] Export spans corresponding to retries (#3732) * Minor update to OTLP readme (#3739) * Nit fixes to prometheus readme * Add minor clarification about OTLP logs * Update workflow (#3741) * Minor improvement to log message (#3742) * [SDK + Jaeger] Support loading environment variables from IConfiguration in Traces & Metrics (#3720) * Support retrieval of environment variables through IConfiguration in SDK. * Update Jaeger to load environment variables through IConfiguration. * Warning fix. * CHANGELOG patch. * Bug fixes. * Warning cleanup. * Code review. * [Zipkin] Support loading envvars from IConfiguration (#3759) * Support loading envvars from IConfiguration in Zipkin exporter. * CHANGELOG patch. * SqlClient Instrumentation to leverage native Activity Status. (#3751) * [Metrics] Update default buckets for Explicit Bucket Histogram from spec (#3722) * [Logs] Fix: Respect AttributeValueLengthLimit when building otlp LogRecord data (#3684) * Respect SdkConfiguration.AttributeValueLengthLimit so otlp data points are not rejected by monitoring tools * Respect maxAttributeCount and use OtlpKeyValueTransformer in AddStringAttribute and in AddIntAttribute * Extend CHANGELOG.md Co-authored-by: Mikel Blanchard <mblanchard@macrosssoftware.com> * Bump actions/setup-dotnet from 3.0.1 to 3.0.2 (#3764) Bumps [actions/setup-dotnet](https://github.com/actions/setup-dotnet) from 3.0.1 to 3.0.2. - [Release notes](https://github.com/actions/setup-dotnet/releases) - [Commits](actions/setup-dotnet@v3.0.1...v3.0.2) --- updated-dependencies: - dependency-name: actions/setup-dotnet dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [Repo] Attempting to stabilize the API Compatibility CI job (#3766) * Attempting to stablize the API Compatibility CI. * Tweak. * More logging. * Increase build logging level for api compat ci job. * Tweak. * Tweak. * Revert extra logging in ci job. * Switched some logging back to debug. * Adding MinMax to Histograms (#2735) * Logging state during building of TracerProvider (#3746) * [HttpClient] Add unit tests for `RecordException` case (#3761) * Add a separate example project for Logs redaction (#3744) * Update CHANGELOG for 1.4.0-beta.2 release (#3772) * [SDK + Otlp] Support loading envvars from IConfiguration (#3760) * Updated Otlp Trace & Metrics exporters to load envvars from IConfiguration. * Patch CHANGELOGs. * Fix up otlp log exporter for SdkOptions changes. * Revert SdkOptions public api. * Restore tests. * Fix benchmarks. * SdkLimitOptions IConfiguration test. * OtlpExporterOptions IConfiguration test. * MetricReaderOptions IConfiguration test. * Bug fix. * Nit. * [SqlClient] Add support for Filter expression (#3743) * [SDK, Jaeger, Zipkin, & Otlp] Support loading envvars for BatchExportActivityProcessorOptions from IConfiguration (#3776) * Support loading envvars for ExportActivityProcessorOptions & BatchExportActivityProcessorOptions from IConfiguration. * Update src/OpenTelemetry/CHANGELOG.md Co-authored-by: Piotr Kiełkowicz <pkiekowicz@splunk.com> * Patch CHANGELOG. * Unit test. Co-authored-by: Cijo Thomas <cithomas@microsoft.com> Co-authored-by: Piotr Kiełkowicz <pkiekowicz@splunk.com> * Remove Env from CI `DOTNET_MULTILEVEL_LOOKUP = 1` (#3779) * Link to .NET docs about different ports (#3704) * Update DS to rc2 (#3781) * ConsoleLogExporter to output full exception (#3784) * [ASP.NET Core] Add back netstandard2.0 and 2.1 targets (#3755) * [HttpClient] Add back netstandard2.0 target (#3787) * Add back netstandard2.0 target * changelog * public api files * Bump System.Text.Json version due to CVE-2021-26701 (#3789) * Auto-generated Semantic Conventions (#2069) * [SDK] Support dependency injection in ResourceBuilder and load envvars from IConfiguration (#3782) * Add Vishwesh as an Approver (#3783) Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * Move more SDK docs to docs folder (#3794) * [Prometheus AspNetCore] Support named options in pipeline extensions (#3780) * Support named options in Prometheus AspNetCore pipeline extensions. * Patch CHANGELOG. Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * [Logs] UnitTest: LogRecord attribute limits (#3758) * Unittest for LogRecord attribute limits * Remove maxValueLength from LogRecordExtensions.AddIntAttribute. Change requested from #3684 (comment) * Pr commits addressed Co-authored-by: Mikel Blanchard <mblanchard@macrosssoftware.com> * Add MinMax to console and doc additions (#3795) * Mark private and internal classes as sealed (#3799) Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * Move more docs to docs folder (#3801) * [ASP.NET Core] Update enrich callbacks to use specific type. (#3749) * [Http] Update enrich callbacks for http (#3792) * [SDK] Support dependency injection in the GetDefaultResource API (#3798) * Support dependency injection in the GetDefaultResource API. * CHANGELOG patch. * Test tweaks. Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * Fix circular reference issue building up tracer provider. (#3803) * [SDK] Add some missing nullable annotations (#3796) * Added some missing nullable annotations in SDK. * Handle null span names in SamplingParameters. * Warning cleanup. * Warning cleanup. * Added link to issue in comment. Co-authored-by: Alan West <3676547+alanwest@users.noreply.github.com> * Guard.Range now uses invariant culture for error message (#3778) Co-authored-by: Utkarsh Umesan Pillai <utpilla@microsoft.com> * Fix circular reference issue building up meter provider. (#3806) Co-authored-by: Utkarsh Umesan Pillai <utpilla@microsoft.com> * Add missing end of code block backticks (#3807) * More doc tweaks (#3805) * More doc tweaks * remove draft staatus * Update grpc client enrich callbacks (#3804) * Port refactor from main-logs branch. (#3808) Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * Merge fixes. * Merge fixes. * Merge fix. Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Cijo Thomas <cithomas@microsoft.com> Co-authored-by: Reiley Yang <reyang@microsoft.com> Co-authored-by: Vishwesh Bankwar <vishweshbankwar@users.noreply.github.com> Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> Co-authored-by: Yun-Ting Lin <yunl@microsoft.com> Co-authored-by: Sebastian Schoder Moreno <35150382+schoder-moreno@users.noreply.github.com> Co-authored-by: Jonathan Wilhelm <Jonathan.wilhelm@sage.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michael Maxwell <micmax@microsoft.com> Co-authored-by: ggoel <gaurav.goel111@gmail.com> Co-authored-by: Utkarsh Umesan Pillai <utpilla@microsoft.com> Co-authored-by: Pavel Steinl <pavel.steinl@gmail.com> Co-authored-by: Piotr Kiełkowicz <pkiekowicz@splunk.com> Co-authored-by: aristotelos <arisvd@gmail.com> Co-authored-by: Joao Grassi <joao@joaograssi.com> Co-authored-by: Alan West <3676547+alanwest@users.noreply.github.com> Co-authored-by: benhall_io <3179852+benbhall@users.noreply.github.com>

Updated Otlp Trace & Metrics exporters to load envvars from IConfigur…

4617af5

…ation.

CodeBlanch requested a review from a team October 12, 2022 21:30

Patch CHANGELOGs.

8084d87

CodeBlanch mentioned this pull request Oct 13, 2022

[Logs] Fix: Respect AttributeValueLengthLimit when building otlp LogRecord data #3684

Merged

3 tasks

CodeBlanch added 3 commits October 13, 2022 10:18

Merge from main.

8cf3007

Fix up otlp log exporter for SdkOptions changes.

a982b96

Merge branch 'main' into oltpexporter-iconfiguration

82461a1

alanwest reviewed Oct 13, 2022

View reviewed changes

src/OpenTelemetry/.publicApi/netstandard2.1/PublicAPI.Unshipped.txt Outdated Show resolved Hide resolved

alanwest reviewed Oct 13, 2022

View reviewed changes

src/OpenTelemetry/Metrics/PeriodicExportingMetricReaderOptions.cs Show resolved Hide resolved

alanwest reviewed Oct 13, 2022

View reviewed changes

test/OpenTelemetry.Tests/SdkOptionsTests.cs Outdated Show resolved Hide resolved

cijothomas reviewed Oct 14, 2022

View reviewed changes

CodeBlanch added 8 commits October 17, 2022 10:44

Revert SdkOptions public api.

bae2b51

Merge remote-tracking branch 'upstream/main' into oltpexporter-iconfi…

0577fbf

…guration

Restore tests.

52a1840

Fix benchmarks.

ae109c8

SdkLimitOptions IConfiguration test.

7ace3cf

OtlpExporterOptions IConfiguration test.

01f5874

MetricReaderOptions IConfiguration test.

246e805

Bug fix.

b8e6bc0

alanwest reviewed Oct 17, 2022

View reviewed changes

src/OpenTelemetry.Exporter.OpenTelemetryProtocol/OtlpLogExporter.cs Outdated Show resolved Hide resolved

Nit.

0eb5204

alanwest reviewed Oct 17, 2022

View reviewed changes

alanwest approved these changes Oct 17, 2022

View reviewed changes

Merge branch 'main' into oltpexporter-iconfiguration

41c55d1

CodeBlanch merged commit 1a2103d into open-telemetry:main Oct 17, 2022

CodeBlanch deleted the oltpexporter-iconfiguration branch October 17, 2022 23:12

CodeBlanch mentioned this pull request Oct 24, 2022

[logs-branch] Support loading OtlpLogExporter envvars from IConfiguration #3812

Merged

1 task

moonbox3 mentioned this pull request Nov 14, 2022

FormatException is thrown when an invalid env var value is set #3690

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SDK + Otlp] Support loading envvars from IConfiguration #3760

[SDK + Otlp] Support loading envvars from IConfiguration #3760

CodeBlanch commented Oct 12, 2022 •

edited

Loading

codecov bot commented Oct 12, 2022 •

edited

Loading

cijothomas Oct 14, 2022

CodeBlanch Oct 17, 2022

alanwest Oct 17, 2022 •

edited

Loading

CodeBlanch Oct 17, 2022

alanwest left a comment

[SDK + Otlp] Support loading envvars from IConfiguration #3760

[SDK + Otlp] Support loading envvars from IConfiguration #3760

Conversation

CodeBlanch commented Oct 12, 2022 • edited Loading

Changes

TODOs

codecov bot commented Oct 12, 2022 • edited Loading

Codecov Report

cijothomas Oct 14, 2022

Choose a reason for hiding this comment

CodeBlanch Oct 17, 2022

Choose a reason for hiding this comment

alanwest Oct 17, 2022 • edited Loading

Choose a reason for hiding this comment

CodeBlanch Oct 17, 2022

Choose a reason for hiding this comment

alanwest left a comment

Choose a reason for hiding this comment

CodeBlanch commented Oct 12, 2022 •

edited

Loading

codecov bot commented Oct 12, 2022 •

edited

Loading

alanwest Oct 17, 2022 •

edited

Loading