[otlp] Remove the Google.Protobuf / Grpc packages, and replace the logs and metrics with the new implementation #6005

rajkumar-rangaraj · 2024-11-28T21:21:41Z

Fixes #5730
Design discussion issue #

Changes

Please provide a brief description of the changes here.

OpenTelemetry.Exporter.OpenTelemetryProtocol

Replaced the current logs and metrics implementation with the new one. Deleted all existing implementations and renamed the new implementation to the existing names.
Remove the following package references from the OTLP Exporter project
- Google.Protobuf
- Grpc
- Grpc.Net.Client
- Grpc.Tools
Deleted all proto files from the project

Tests / Benchmarks

Updated both tests and benchmarks project to include proto files and the following packages Google.Protobuf, Grpc, Grpc.Tools and Grpc.Net.Client.
Updated the tests.

Merge requirement checklist

CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
Unit tests added/updated
Appropriate CHANGELOG.md files updated for non-trivial changes
Changes in public API reviewed (if applicable)

rajkumar-rangaraj · 2024-11-28T21:22:55Z

...OpenTelemetry.Exporter.OpenTelemetryProtocol/Implementation/ExportClient/OtlpExportClient.cs

    {
        Guard.ThrowIfNull(options);
        Guard.ThrowIfNull(httpClient);
        Guard.ThrowIfNull(signalPath);

-        Uri exporterEndpoint = options.Endpoint.AppendPathIfNotPresent(signalPath);
+        Uri exporterEndpoint;
+        if (options.Protocol == OtlpExportProtocol.Grpc)


Added a logical condition here to honor the existing behavior.

Curious how it got by tests before?

New implementation was never tested on it. On the replacement we caught this issue.

The actual issue was with the HTTP implementation. In the gRPC case, we append the path in all scenarios. However, for HTTP, the path is appended only if it is provided by default or through environment variables. If an endpoint is explicitly set, we do not append the path in the HTTP case, but this behavior was not being followed in new implementation.

During my initial implementation, I mistakenly assumed this was a bug and removed the check. However, there was an existing test case that validated this behavior.

exporterEndpoint = options.AppendSignalPathToEndpoint ? options.Endpoint.AppendPathIfNotPresent(signalPath) : options.Endpoint;

test/Benchmarks/Benchmarks.csproj

codecov · 2024-11-28T21:27:54Z

Codecov Report

Attention: Patch coverage is 73.80952% with 22 lines in your changes missing coverage. Please review.

Project coverage is 86.23%. Comparing base (84e6afb) to head (c54a555).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
....Exporter.OpenTelemetryProtocol/OtlpLogExporter.cs	56.00%	11 Missing ⚠️
...porter.OpenTelemetryProtocol/OtlpMetricExporter.cs	52.17%	11 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6005      +/-   ##
==========================================
+ Coverage   85.15%   86.23%   +1.08%     
==========================================
  Files         272      257      -15     
  Lines       12420    11690     -730     
==========================================
- Hits        10576    10081     -495     
+ Misses       1844     1609     -235

Flag	Coverage Δ
unittests-Project-Experimental	`86.07% <73.80%> (+0.92%)`	⬆️
unittests-Project-Stable	`86.11% <73.80%> (+0.97%)`	⬆️
unittests-Solution	`86.13% <73.80%> (?)`
unittests-UnstableCoreLibraries-Experimental	`85.78% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ol/Implementation/ExportClient/OtlpExportClient.cs	`100.00% <100.00%> (ø)`
...mplementation/ExportClient/OtlpGrpcExportClient.cs	`66.66% <ø> (ø)`
...mplementation/ExportClient/OtlpHttpExportClient.cs	`100.00% <ø> (ø)`
...yProtocol/Implementation/ExportClient/OtlpRetry.cs	`85.71% <100.00%> (+1.84%)`	⬆️
...tlpExporterPersistentStorageTransmissionHandler.cs	`92.72% <100.00%> (+57.94%)`	⬆️
...ansmission/OtlpExporterRetryTransmissionHandler.cs	`80.00% <100.00%> (+60.00%)`	⬆️
...on/Transmission/OtlpExporterTransmissionHandler.cs	`100.00% <100.00%> (+29.41%)`	⬆️
...TelemetryProtocol/OtlpExporterOptionsExtensions.cs	`96.90% <100.00%> (+8.37%)`	⬆️
...lemetryProtocol/OtlpLogExporterHelperExtensions.cs	`94.26% <100.00%> (+0.67%)`	⬆️
...nTelemetryProtocol/OtlpMetricExporterExtensions.cs	`96.51% <100.00%> (+1.05%)`	⬆️
... and 3 more

... and 10 files with indirect coverage changes

Kielek · 2024-11-29T13:22:44Z

🎉 I do not have enough time to review it right now, but it is great to see this progress.

TimothyMothra · 2024-12-02T18:55:41Z

It looks like the proto files are copied into two directories, unit test & benchmarks.
Can these be stored in a common directory?

...etryProtocol/Implementation/Transmission/OtlpExporterPersistentStorageTransmissionHandler.cs

.editorconfig

CodeBlanch · 2024-12-02T22:10:57Z

...Telemetry.Exporter.OpenTelemetryProtocol/OpenTelemetry.Exporter.OpenTelemetryProtocol.csproj

-  <ItemGroup>
-    <PackageReference Include="Grpc.Net.Client" Condition="'$(TargetFramework)' != 'netstandard2.0' AND '$(TargetFramework)' != '$(NetFrameworkMinimumSupportedVersion)'" />
-    <PackageReference Include="Grpc" Condition="'$(TargetFramework)' == 'netstandard2.0' OR '$(TargetFramework)' == '$(NetFrameworkMinimumSupportedVersion)'" />
-    <PackageReference Include="Google.Protobuf" />
-    <PackageReference Include="Grpc.Tools" PrivateAssets="All" />
-  </ItemGroup>


Dropping these dependencies is probably worthy of a CHANGELOG entry? What you think @alanwest?

This definitely needs to be a part of the changelog. There are few things we need to cover in the changelog, planning to cover as follow-up.

Removal of this packages

Breaking change on PeerServiceResolver, in one the earlier PR got feedback from you and Alan and we decided to remove this look up as we aren't aware why this logic was present https://github.com/open-telemetry/opentelemetry-dotnet/blob/84e6afbebae67f4e9b498f1702edfe4ceccf34ec/src/Shared/PeerServiceResolver.cs

Call-out on the internal serialization change.

CodeBlanch

LGTM

rajkumar-rangaraj added 4 commits November 27, 2024 17:05

Replace logs with new impl

1f551af

Remove package reference, remove proto files, replace logs and metrics

9aa77bc

Update Benchmarks

cb52d26

Test updates

d7b7797

rajkumar-rangaraj requested a review from a team as a code owner November 28, 2024 21:21

github-actions bot added pkg:OpenTelemetry.Exporter.OpenTelemetryProtocol Issues related to OpenTelemetry.Exporter.OpenTelemetryProtocol NuGet package perf Performance related labels Nov 28, 2024

rajkumar-rangaraj commented Nov 28, 2024

View reviewed changes

test/Benchmarks/Benchmarks.csproj Outdated Show resolved Hide resolved

Lint fixes

8a0beb9

CodeBlanch reviewed Dec 2, 2024

View reviewed changes

...etryProtocol/Implementation/Transmission/OtlpExporterPersistentStorageTransmissionHandler.cs Outdated Show resolved Hide resolved

CodeBlanch added 2 commits December 2, 2024 13:25

Cleanup proto README.

0599950

Tweaks.

bf4becd

github-actions bot added the documentation Documentation related label Dec 2, 2024

CodeBlanch reviewed Dec 2, 2024

View reviewed changes

.editorconfig Show resolved Hide resolved

Lint.

d07f706

CodeBlanch changed the title ~~[otlp] Remove the Google.Protobuf / Grpc packages, and replace the logs and metrics with the new implementation.~~ [otlp] Remove the Google.Protobuf / Grpc packages, and replace the logs and metrics with the new implementation Dec 2, 2024

Clean up.

41dc362

CodeBlanch reviewed Dec 2, 2024

View reviewed changes

nits

c54a555

CodeBlanch approved these changes Dec 3, 2024

View reviewed changes

CodeBlanch merged commit b9be07a into open-telemetry:main Dec 3, 2024
49 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[otlp] Remove the Google.Protobuf / Grpc packages, and replace the logs and metrics with the new implementation #6005

[otlp] Remove the Google.Protobuf / Grpc packages, and replace the logs and metrics with the new implementation #6005

rajkumar-rangaraj commented Nov 28, 2024

rajkumar-rangaraj Nov 28, 2024

CodeBlanch Dec 2, 2024 •

edited

Loading

rajkumar-rangaraj Dec 2, 2024

rajkumar-rangaraj Dec 2, 2024

codecov bot commented Nov 28, 2024 •

edited

Loading

Kielek commented Nov 29, 2024

TimothyMothra commented Dec 2, 2024

CodeBlanch Dec 2, 2024

rajkumar-rangaraj Dec 2, 2024

CodeBlanch left a comment

[otlp] Remove the Google.Protobuf / Grpc packages, and replace the logs and metrics with the new implementation #6005

[otlp] Remove the Google.Protobuf / Grpc packages, and replace the logs and metrics with the new implementation #6005

Conversation

rajkumar-rangaraj commented Nov 28, 2024

Changes

OpenTelemetry.Exporter.OpenTelemetryProtocol

Tests / Benchmarks

Merge requirement checklist

rajkumar-rangaraj Nov 28, 2024

Choose a reason for hiding this comment

CodeBlanch Dec 2, 2024 • edited Loading

Choose a reason for hiding this comment

rajkumar-rangaraj Dec 2, 2024

Choose a reason for hiding this comment

rajkumar-rangaraj Dec 2, 2024

Choose a reason for hiding this comment

codecov bot commented Nov 28, 2024 • edited Loading

Codecov Report

Kielek commented Nov 29, 2024

TimothyMothra commented Dec 2, 2024

CodeBlanch Dec 2, 2024

Choose a reason for hiding this comment

rajkumar-rangaraj Dec 2, 2024

Choose a reason for hiding this comment

CodeBlanch left a comment

Choose a reason for hiding this comment

CodeBlanch Dec 2, 2024 •

edited

Loading

codecov bot commented Nov 28, 2024 •

edited

Loading