Auto-generated Semantic Conventions #2069

joaopgrassi · 2021-05-29T13:29:54Z

Changes

As discussed in #1658, this PR introduces a new package OpenTelemetry.SemanticConventions which aims to be the home of the auto-generated semantic convention code files. It uses the Semantic Convention Generator, similar as it's done in other languages.

Few things I noticed while doing this, which probably need some discussion and agreement:

The existing SemanticConventions.cs class. Looking at the other languages, they call it:
- Java: SemanticAttributes
- Python: SpanAttributes
- JS: SemanticAttributes
The existing ResourceSemanticConventions class. The other languages call it pretty much ResourceAttributes. But this name conflicts with ResourceAttributes from System.Reflection, so probably not a good name to use.

What should we use here? I went with TraceSemanticConventions, which is not similar to any of the above, but is similar to what we have with the resource one.

For now, I went with this:

SemanticConventions -> TraceSemanticConventions. Inside namespace OpenTelemetry.SemanticConventions.Trace
ResourceSemanticConventions -> same name. Inside namespace OpenTelemetry.SemanticConventions.Resource

The code files are generated via a script, manually for now. Once the naming above is agreed, I can add this new project to all others in the solution and start renaming and see if all works. Since they were internal files, I think it shouldn't be a problem?

linux-foundation-easycla · 2021-05-29T13:29:58Z

The committers are authorized under a signed CLA.

✅ Joao Grassi (4b91bdb4d7d99cf6610409cf150d040ff30a9f7a, ca2dd5ad3a54264620fab9278b642d2e4d649ad6, 2b897ac78ed27d9ee840fb0662283c160d2fd59e)

pcwiese · 2021-05-30T19:47:25Z

@cijothomas @reyang for guidance on naming here.

codecov · 2021-05-31T12:17:25Z

Codecov Report

Merging #2069 (459cf85) into main (6d6222f) will increase coverage by 0.06%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2069      +/-   ##
==========================================
+ Coverage   87.32%   87.38%   +0.06%     
==========================================
  Files         281      281              
  Lines       10758    10758              
==========================================
+ Hits         9394     9401       +7     
+ Misses       1364     1357       -7

Impacted Files	Coverage Δ
...metryProtocol/Implementation/ActivityExtensions.cs	`96.21% <0.00%> (+1.08%)`	⬆️
src/OpenTelemetry/BatchExportProcessor.cs	`84.11% <0.00%> (+1.86%)`	⬆️
...tpListener/Internal/PrometheusCollectionManager.cs	`75.82% <0.00%> (+2.19%)`	⬆️
...lementation/SqlClientInstrumentationEventSource.cs	`75.00% <0.00%> (+4.16%)`	⬆️

bruno-garcia

I think this is great stuff! Left some suggestions

src/OpenTelemetry.SemanticConventions/Resource/ResourceSemanticConventions.cs

src/OpenTelemetry.SemanticConventions/OpenTelemetry.SemanticConventions.csproj

src/OpenTelemetry.SemanticConventions/scripts/semantic-convetion/generate.sh

bruno-garcia · 2021-06-02T20:41:51Z

Also, this seems pretty mature already, any particular reason for it to be a draft?
You'll get the most reviews from folks once you drop the draft flag

joaopgrassi · 2021-06-02T21:44:58Z

@bruno-garcia I'll go through the comments, thanks!

Also, this seems pretty mature already, any particular reason for it to be a draft? You'll get the most reviews from folks once you drop the draft flag

It's mostly because I have all these naming stuff that I'm really not sure, so there's still quite some stuff to do. Once the namings are defined I'll have to go and change everywhere and see if everything still "works".

Then the idea is to merge this, and think how we will publish this package. Only then we'll use it in all the projects that currently use the internal class. At least that's what @pcwiese and I discussed in the linked issue and in Slack.

joaopgrassi · 2021-06-10T20:39:53Z

Publishing it now, so maybe we can get some feedback. As @bruno-garcia said I think now it's in a good state, just need to define the small details.

src/OpenTelemetry.SemanticConventions/OpenTelemetry.SemanticConventions.csproj

cijothomas · 2021-06-10T23:22:34Z

For now, I went with this:

SemanticConventions -> TraceSemanticConventions. Inside namespace OpenTelemetry.SemanticConventions.Trace
ResourceSemanticConventions -> same name. Inside namespace OpenTelemetry.SemanticConventions.Resource

TraceSemanticConventions inside Otel.SemanticConventions.Trace seems bit redundant. We may name the namespace as OpenTelemetry.SemanticConventions, and classes TraceSemanticConventions, MetricsSemanticConventions and ResourceSemanticConventions

cijothomas · 2021-06-11T00:09:33Z

For now, I went with this:
SemanticConventions -> TraceSemanticConventions. Inside namespace OpenTelemetry.SemanticConventions.Trace
ResourceSemanticConventions -> same name. Inside namespace OpenTelemetry.SemanticConventions.Resource

TraceSemanticConventions inside Otel.SemanticConventions.Trace seems bit redundant. We may name the namespace as OpenTelemetry.SemanticConventions, and classes TraceSemanticConventions, MetricsSemanticConventions and ResourceSemanticConventions

Alternately, if we re-use existing namespaces OpenTelemetry.Trace, Opentelemetry.Resource etc we can name classes as TraceSemanticConventions, MetricsSemanticConventions and ResourceSemanticConventions. This will avoid requires users to import more namespaces.

joaopgrassi · 2021-06-12T12:28:26Z

@cijothomas thanks for the review! I changed the namespace as your last comment.

What do you think regarding @bruno-garcia suggestion on having an inner class for the attributes?

What about an inner class Attribute and having these generated under it?
So we'd access it through:

ResourceSemanticConventions.Attribute.CloudProvider

And how do you think we can release this? Via CI? manual? @pcwiese had some comments in the original issue ticket... also how we will use this package later on the other projects? I presume since Semantic Conventions are not stable yet, we can't add it to the stable packages?

joaopgrassi · 2021-06-13T14:37:10Z

Something else I forgot: Should I add the files for .publicApi for this as well?

joaopgrassi · 2021-06-21T18:03:10Z

I added the .publicApi files. The sanitycheck/encoding CI task is failing due to a non-ascii char in one of the xml docs. It comes from here: https://github.com/open-telemetry/opentelemetry-specification/blob/d271b73d6b274bfd51eae9ed813586f0322342b7/semantic_conventions/trace/database.yaml#L65.

Not sure what to do about it.. any suggestions?

moonheart · 2021-06-22T03:42:55Z

I added the .publicApi files. The sanitycheck/encoding CI task is failing due to a non-ascii char in one of the xml docs. It comes from here: open-telemetry/opentelemetry-specification@d271b73/semantic_conventions/trace/database.yaml#L65.

Not sure what to do about it.. any suggestions?

'InterSystems Caché' é is the non-ascii char.

You can check it here.

src/OpenTelemetry.SemanticConventions/OpenTelemetry.SemanticConventions.csproj

joaopgrassi · 2021-07-01T10:51:32Z

'InterSystems Caché' é is the non-ascii char.

You can check it here.

@moonheart Yes, that's what I mentioned above. It is coming from the yaml file. None of the alternatives I thought seem to be good.

Send a PR to remove the non-ascii on the source probably not viable
Removing the xml comments for the auto-generated code files also not great

Thought of maybe defining some mapping and remove it during the code generation. For this case, substitute é with e, but also sounds like a hack. If tomorrow new ones are introduced, we need to change this again.

To me, the best option would be to disable the CI check for this package, but that is not up to me so waiting on others for directions.

Will be the home for the auto-generated semantic conventions constants.

Added instructions on how to update the PublicAPI files using the dotnet-format global tool.

cartermp · 2022-09-06T17:37:33Z

Makes sense! I'll defer to maintainers here on how to best proceed. It's not strictly urgent (just a docs update), but if we get an indication that the time to update w.r.t. the latest spec will be longer for reasons that can't be controlled here, I'd be happy to help with anything needed to get it merged+released as-is (and then documented).

src/OpenTelemetry.SemanticConventions/OpenTelemetry.SemanticConventions.csproj

github-actions · 2022-09-14T04:02:20Z

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

github-actions · 2022-09-21T04:04:38Z

Closed as inactive. Feel free to reopen if this PR is still being worked on.

And other minor improvements.

joaopgrassi · 2022-09-27T11:32:34Z

@cijothomas I think this is ready for a final review. I think the important thing is if we like the way the constants are defined. Also, only "enums" that have values of string type are supported. I can iterate later and see if we can improve this later as a follow up.

I updated it with the latest spec release (1.13), ironed out a few warnings and re-generated the public api files. The build seems to fail in Windows for net6.0, not sure why. I can't find a way to re-try failed jobs.

github-actions · 2022-10-18T04:03:53Z

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

cijothomas

thanks for getting this ready!
Lets try and incorporate this into existing projects one by one, and flush out any issues in that process, before officially releasing this as a nuget.

* [Logs] Fix buffered log scopes being reused (#3731) * Fix buffered log scopes being reused. * CHANGELOG update. * Test fixes. Co-authored-by: Cijo Thomas <cithomas@microsoft.com> * clarify that Prometheus HttpListner is not for production at this moment (#3737) * [HttpClient] Export spans corresponding to retries (#3732) * Minor update to OTLP readme (#3739) * Nit fixes to prometheus readme * Add minor clarification about OTLP logs * Update workflow (#3741) * Minor improvement to log message (#3742) * [SDK + Jaeger] Support loading environment variables from IConfiguration in Traces & Metrics (#3720) * Support retrieval of environment variables through IConfiguration in SDK. * Update Jaeger to load environment variables through IConfiguration. * Warning fix. * CHANGELOG patch. * Bug fixes. * Warning cleanup. * Code review. * [Zipkin] Support loading envvars from IConfiguration (#3759) * Support loading envvars from IConfiguration in Zipkin exporter. * CHANGELOG patch. * SqlClient Instrumentation to leverage native Activity Status. (#3751) * [Metrics] Update default buckets for Explicit Bucket Histogram from spec (#3722) * [Logs] Fix: Respect AttributeValueLengthLimit when building otlp LogRecord data (#3684) * Respect SdkConfiguration.AttributeValueLengthLimit so otlp data points are not rejected by monitoring tools * Respect maxAttributeCount and use OtlpKeyValueTransformer in AddStringAttribute and in AddIntAttribute * Extend CHANGELOG.md Co-authored-by: Mikel Blanchard <mblanchard@macrosssoftware.com> * Bump actions/setup-dotnet from 3.0.1 to 3.0.2 (#3764) Bumps [actions/setup-dotnet](https://github.com/actions/setup-dotnet) from 3.0.1 to 3.0.2. - [Release notes](https://github.com/actions/setup-dotnet/releases) - [Commits](actions/setup-dotnet@v3.0.1...v3.0.2) --- updated-dependencies: - dependency-name: actions/setup-dotnet dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [Repo] Attempting to stabilize the API Compatibility CI job (#3766) * Attempting to stablize the API Compatibility CI. * Tweak. * More logging. * Increase build logging level for api compat ci job. * Tweak. * Tweak. * Revert extra logging in ci job. * Switched some logging back to debug. * Adding MinMax to Histograms (#2735) * Logging state during building of TracerProvider (#3746) * [HttpClient] Add unit tests for `RecordException` case (#3761) * Add a separate example project for Logs redaction (#3744) * Update CHANGELOG for 1.4.0-beta.2 release (#3772) * [SDK + Otlp] Support loading envvars from IConfiguration (#3760) * Updated Otlp Trace & Metrics exporters to load envvars from IConfiguration. * Patch CHANGELOGs. * Fix up otlp log exporter for SdkOptions changes. * Revert SdkOptions public api. * Restore tests. * Fix benchmarks. * SdkLimitOptions IConfiguration test. * OtlpExporterOptions IConfiguration test. * MetricReaderOptions IConfiguration test. * Bug fix. * Nit. * [SqlClient] Add support for Filter expression (#3743) * [SDK, Jaeger, Zipkin, & Otlp] Support loading envvars for BatchExportActivityProcessorOptions from IConfiguration (#3776) * Support loading envvars for ExportActivityProcessorOptions & BatchExportActivityProcessorOptions from IConfiguration. * Update src/OpenTelemetry/CHANGELOG.md Co-authored-by: Piotr Kiełkowicz <pkiekowicz@splunk.com> * Patch CHANGELOG. * Unit test. Co-authored-by: Cijo Thomas <cithomas@microsoft.com> Co-authored-by: Piotr Kiełkowicz <pkiekowicz@splunk.com> * Remove Env from CI `DOTNET_MULTILEVEL_LOOKUP = 1` (#3779) * Link to .NET docs about different ports (#3704) * Update DS to rc2 (#3781) * ConsoleLogExporter to output full exception (#3784) * [ASP.NET Core] Add back netstandard2.0 and 2.1 targets (#3755) * [HttpClient] Add back netstandard2.0 target (#3787) * Add back netstandard2.0 target * changelog * public api files * Bump System.Text.Json version due to CVE-2021-26701 (#3789) * Auto-generated Semantic Conventions (#2069) * [SDK] Support dependency injection in ResourceBuilder and load envvars from IConfiguration (#3782) * Add Vishwesh as an Approver (#3783) Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * Move more SDK docs to docs folder (#3794) * [Prometheus AspNetCore] Support named options in pipeline extensions (#3780) * Support named options in Prometheus AspNetCore pipeline extensions. * Patch CHANGELOG. Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * [Logs] UnitTest: LogRecord attribute limits (#3758) * Unittest for LogRecord attribute limits * Remove maxValueLength from LogRecordExtensions.AddIntAttribute. Change requested from #3684 (comment) * Pr commits addressed Co-authored-by: Mikel Blanchard <mblanchard@macrosssoftware.com> * Add MinMax to console and doc additions (#3795) * Mark private and internal classes as sealed (#3799) Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * Move more docs to docs folder (#3801) * [ASP.NET Core] Update enrich callbacks to use specific type. (#3749) * [Http] Update enrich callbacks for http (#3792) * [SDK] Support dependency injection in the GetDefaultResource API (#3798) * Support dependency injection in the GetDefaultResource API. * CHANGELOG patch. * Test tweaks. Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * Fix circular reference issue building up tracer provider. (#3803) * [SDK] Add some missing nullable annotations (#3796) * Added some missing nullable annotations in SDK. * Handle null span names in SamplingParameters. * Warning cleanup. * Warning cleanup. * Added link to issue in comment. Co-authored-by: Alan West <3676547+alanwest@users.noreply.github.com> * Guard.Range now uses invariant culture for error message (#3778) Co-authored-by: Utkarsh Umesan Pillai <utpilla@microsoft.com> * Fix circular reference issue building up meter provider. (#3806) Co-authored-by: Utkarsh Umesan Pillai <utpilla@microsoft.com> * Add missing end of code block backticks (#3807) * More doc tweaks (#3805) * More doc tweaks * remove draft staatus * Update grpc client enrich callbacks (#3804) * Port refactor from main-logs branch. (#3808) Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> * Merge fixes. * Merge fixes. * Merge fix. Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Cijo Thomas <cithomas@microsoft.com> Co-authored-by: Reiley Yang <reyang@microsoft.com> Co-authored-by: Vishwesh Bankwar <vishweshbankwar@users.noreply.github.com> Co-authored-by: Cijo Thomas <cijo.thomas@gmail.com> Co-authored-by: Yun-Ting Lin <yunl@microsoft.com> Co-authored-by: Sebastian Schoder Moreno <35150382+schoder-moreno@users.noreply.github.com> Co-authored-by: Jonathan Wilhelm <Jonathan.wilhelm@sage.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michael Maxwell <micmax@microsoft.com> Co-authored-by: ggoel <gaurav.goel111@gmail.com> Co-authored-by: Utkarsh Umesan Pillai <utpilla@microsoft.com> Co-authored-by: Pavel Steinl <pavel.steinl@gmail.com> Co-authored-by: Piotr Kiełkowicz <pkiekowicz@splunk.com> Co-authored-by: aristotelos <arisvd@gmail.com> Co-authored-by: Joao Grassi <joao@joaograssi.com> Co-authored-by: Alan West <3676547+alanwest@users.noreply.github.com> Co-authored-by: benhall_io <3179852+benbhall@users.noreply.github.com>

bruno-garcia reviewed Jun 2, 2021

View reviewed changes

joaopgrassi marked this pull request as ready for review June 10, 2021 20:40

joaopgrassi requested review from a team and bruno-garcia June 10, 2021 20:40

cijothomas reviewed Jun 10, 2021

View reviewed changes

src/OpenTelemetry.SemanticConventions/OpenTelemetry.SemanticConventions.csproj Outdated Show resolved Hide resolved

joaopgrassi requested a review from cijothomas June 12, 2021 12:28

cijothomas reviewed Jun 22, 2021

View reviewed changes

src/OpenTelemetry.SemanticConventions/OpenTelemetry.SemanticConventions.csproj Outdated Show resolved Hide resolved

joaopgrassi force-pushed the feat/autogen_semconv branch from dbaf800 to a278335 Compare August 31, 2021 19:45

joaopgrassi force-pushed the feat/autogen_semconv branch from 69564d2 to 59d801e Compare September 13, 2021 19:14

joaopgrassi force-pushed the feat/autogen_semconv branch 3 times, most recently from 8ff294b to d92fce5 Compare October 17, 2021 11:45

Introduce SemanticConventions project

7e2b190

Will be the home for the auto-generated semantic conventions constants.

joaopgrassi force-pushed the feat/autogen_semconv branch from d92fce5 to 7e2b190 Compare October 17, 2021 11:47

cijothomas and others added 3 commits October 18, 2021 10:30

Merge branch 'main' into feat/autogen_semconv

4a0305f

Merge branch 'main' into feat/autogen_semconv

a26aa03

Add missing PublicAPI entries

03ffc5d

Added instructions on how to update the PublicAPI files using the dotnet-format global tool.

cijothomas reviewed Sep 6, 2022

View reviewed changes

src/OpenTelemetry.SemanticConventions/OpenTelemetry.SemanticConventions.csproj Outdated Show resolved Hide resolved

github-actions bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Sep 14, 2022

github-actions bot closed this Sep 21, 2022

cijothomas reopened this Sep 21, 2022

Merge branch 'main' into feat/autogen_semconv

3b99dc8

github-actions bot removed the Stale Issues and pull requests which have been flagged for closing due to inactivity label Sep 22, 2022

joaopgrassi added 3 commits September 26, 2022 23:13

Use spec version 1.13, net462, update PublicApi files

c558b57

And other minor improvements.

Appease build with missing blank line

cf422d9

Merge branch 'main' into feat/autogen_semconv

d850232

cijothomas and others added 3 commits September 30, 2022 07:42

Merge branch 'main' into feat/autogen_semconv

af99f52

Merge branch 'main' into feat/autogen_semconv

6494bd4

Merge branch 'main' into feat/autogen_semconv

7cbfc4c

github-actions bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Oct 18, 2022

Merge branch 'main' into feat/autogen_semconv

3d7956b

cijothomas approved these changes Oct 18, 2022

View reviewed changes

cijothomas added 2 commits October 18, 2022 14:26

Merge branch 'main' into feat/autogen_semconv

110db72

Merge branch 'main' into feat/autogen_semconv

d3af237

github-actions bot removed the Stale Issues and pull requests which have been flagged for closing due to inactivity label Oct 19, 2022

Merge branch 'main' into feat/autogen_semconv

459cf85

cijothomas merged commit 2377d00 into open-telemetry:main Oct 19, 2022

CodeBlanch mentioned this pull request Oct 24, 2022

[Repo] Revert OpenTelemetry.Instrumentation.AspNet OpenTelemetry.proj change #3811

Merged

joaopgrassi deleted the feat/autogen_semconv branch October 28, 2022 15:17

tshowers-bt mentioned this pull request Jan 24, 2023

SemanticConventions attribute name constants differ from spec #4101

Closed

joaopgrassi mentioned this pull request Aug 23, 2023

Improve documentation of code generator open-telemetry/build-tools#198

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Auto-generated Semantic Conventions #2069

Auto-generated Semantic Conventions #2069

joaopgrassi commented May 29, 2021 •

edited

Loading

linux-foundation-easycla bot commented May 29, 2021 •

edited

Loading

pcwiese commented May 30, 2021

codecov bot commented May 31, 2021 •

edited

Loading

bruno-garcia left a comment

bruno-garcia commented Jun 2, 2021

joaopgrassi commented Jun 2, 2021 •

edited

Loading

joaopgrassi commented Jun 10, 2021

cijothomas commented Jun 10, 2021

cijothomas commented Jun 11, 2021

joaopgrassi commented Jun 12, 2021 •

edited

Loading

joaopgrassi commented Jun 13, 2021

joaopgrassi commented Jun 21, 2021

moonheart commented Jun 22, 2021 •

edited

Loading

joaopgrassi commented Jul 1, 2021 •

edited

Loading

cartermp commented Sep 6, 2022

github-actions bot commented Sep 14, 2022

github-actions bot commented Sep 21, 2022

joaopgrassi commented Sep 27, 2022 •

edited

Loading

github-actions bot commented Oct 18, 2022

cijothomas left a comment

Auto-generated Semantic Conventions #2069

Auto-generated Semantic Conventions #2069

Conversation

joaopgrassi commented May 29, 2021 • edited Loading

Changes

linux-foundation-easycla bot commented May 29, 2021 • edited Loading

pcwiese commented May 30, 2021

codecov bot commented May 31, 2021 • edited Loading

Codecov Report

bruno-garcia left a comment

Choose a reason for hiding this comment

bruno-garcia commented Jun 2, 2021

joaopgrassi commented Jun 2, 2021 • edited Loading

joaopgrassi commented Jun 10, 2021

cijothomas commented Jun 10, 2021

cijothomas commented Jun 11, 2021

joaopgrassi commented Jun 12, 2021 • edited Loading

joaopgrassi commented Jun 13, 2021

joaopgrassi commented Jun 21, 2021

moonheart commented Jun 22, 2021 • edited Loading

joaopgrassi commented Jul 1, 2021 • edited Loading

cartermp commented Sep 6, 2022

github-actions bot commented Sep 14, 2022

github-actions bot commented Sep 21, 2022

joaopgrassi commented Sep 27, 2022 • edited Loading

github-actions bot commented Oct 18, 2022

cijothomas left a comment

Choose a reason for hiding this comment

joaopgrassi commented May 29, 2021 •

edited

Loading

linux-foundation-easycla bot commented May 29, 2021 •

edited

Loading

codecov bot commented May 31, 2021 •

edited

Loading

joaopgrassi commented Jun 2, 2021 •

edited

Loading

joaopgrassi commented Jun 12, 2021 •

edited

Loading

moonheart commented Jun 22, 2021 •

edited

Loading

joaopgrassi commented Jul 1, 2021 •

edited

Loading

joaopgrassi commented Sep 27, 2022 •

edited

Loading