sdk: expose hidden Tracer::should_sample() method

[djc]

Changes

Unfortunately while working on #1934 I missed that tracing-opentelemetry needs one additional API method. Add that here. I will put up a draft tracing-opentelemetry PR after this to make sure this is complete.

Merge requirement checklist

• CONTRIBUTING guidelines followed
• Unit tests added/updated (if applicable)
• Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
• Changes in public API reviewed (if applicable)

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 74.9%. Comparing base (a9b8621) to head (44a0ff9).

Files	Patch %	Lines
opentelemetry-sdk/src/trace/tracer.rs	0.0%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #1937     +/-   ##
=======================================
- Coverage   74.9%   74.9%   -0.1%     
=======================================
  Files        122     122             
  Lines      20311   20314      +3     
=======================================
  Hits       15217   15217             
- Misses      5094    5097      +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[djc]

Downstream PR in tokio-rs/tracing-opentelemetry#155. It doesn't compile yet, but it's late here. If anyone wants to help figure this out, would appreciate it.

(At this time, not sure if this PR is necessary or sufficient, probably better not to merge it yet.)

[lalitb]

sdk::trace::Config was made non-exhaustive in #1755, so it's direct initialization is not possible. Also AttributeSet should be replaced with Vec<KeyValue> (refer #1792).

[cijothomas]

Downstream PR in tokio-rs/tracing-opentelemetry#155. It doesn't compile yet, but it's late here. If anyone wants to help figure this out, would appreciate it.

(At this time, not sure if this PR is necessary or sufficient, probably better not to merge it yet.)

Will hold off until we get confirmation that this will unblock tracing-opentelemetry.

[djc]

I've spent some time staring at the downstream conversion and have been having a hard time making sense of how all the bits are supposed to fit together. Apparently stuff like opentelemetry_otlp::new_pipeline() now yields a TracerProvider instead of a Tracer, the caller has to explicitly install that, apparently it should clone() but this is not enforced by the API, then it gets a GlobalTracerProvider... and I'm not sure where to go from there. Honestly the documentation doesn't seem that great either, there's a bunch of examples code but not much reference documentation that explains from first principles who responsibilities are divided up over different parts of the API.

I'm honestly wondering whether it wouldn't make more sense to have a separate tracing_subscriber::Subscriber implementation with an OTLP client, mostly bypassing the opentelemetry stack.

[cijothomas]

@djc
Unfortunately, there is no one from tracing-opentelemetry repo who reviews PRs in this repo. Though we need to solve the long standing problem of 2 competing APIs, for short term, it'd be super helpful if someone experienced in tracing-opentelemetry can also review changes here. @djc if you can help with this, that'd be great.

We are constantly seeing issues with opentelemetry and tracing, most recent being #1938
#1571 must be resolved to address the root of the issue.

For the specific changes with OTLPExporter returning tracer vs tracerprovider - See if the examples in this repo https://github.com/open-telemetry/opentelemetry-rust/blob/main/examples/tracing-jaeger/src/main.rs#L13 are helpful.

I'm honestly wondering whether it wouldn't make more sense to have a separate tracing_subscriber::Subscriber implementation with an OTLP client, mostly bypassing the opentelemetry stack.

Looks like you are leaning towards Option 4 from #1571, where otel and tracing just operates as 2 independent libraries, no special casing to support each other. (TBH, I haven't seen anyone supporting that approach so far. Apologies if I misinterpreted your statement.)

If #1689 is adopted, then there won't be need for a separate bridge, as OTel will natively understand Spans from tracing, and whole class of issues like this will just disappear.

[lalitb]

@djc - I updated the example code in tracing-opentelemetry fork to demonstrate the fix to be done
https://github.com/tokio-rs/tracing-opentelemetry/compare/v0.1.x...lalitb:tracing-opentelemetry:v0.24-fix?expand=1

refer to the changes in the examples/ directory. I believe rest of the CI failures are simple to fix, so ignored them for now. Let me know if this helps.

Also +1 to have someone from tracing-opentelemetry to review the tracing related PRs in opentelemetry-rust repo. So that these issues can be caught upfront.

[djc]

@djc Unfortunately, there is no one from tracing-opentelemetry repo who reviews PRs in this repo. Though we need to solve the long standing problem of 2 competing APIs, for short term, it'd be super helpful if someone experienced in tracing-opentelemetry can also review changes here. @djc if you can help with this, that'd be great.

Honestly, I'm a "reluctant" maintainer of tracing-opentelemetry -- I'm mainly helping out because I have a use case at work for funnelling tracing-rs spans into (a) Datadog (via OTLP), (b) Google Cloud Trace (née Stackdriver) and (c) Slack (via a custom SpanExporter) and (d) CLI tooling (via tracing_subscriber::fmt::layer()), and at the time it seemed to make sense to adopt the OpenTelemetry standards to make this easier.

I previously found being involved in the opentelemetry-rust project pretty frustrating, so I don't think I want to ramp that back up. I do think the opentelemetry-rust maintainers should consider tracing-opentelemetry an integral part of the project since, as far as I can tell, a majority of the opentelemetry-rust tracing consumers probably use it. So I would suggest we invert the burden of maintenance and say that openteleemtry-rust maintenance responsibility should extend out towards tracing-opentelemetry at least for trace-related public API changes.

I'm honestly wondering whether it wouldn't make more sense to have a separate tracing_subscriber::Subscriber implementation with an OTLP client, mostly bypassing the opentelemetry stack.

Looks like you are leaning towards Option 4 from #1571, where otel and tracing just operates as 2 independent libraries, no special casing to support each other. (TBH, I haven't seen anyone supporting that approach so far. Apologies if I misinterpreted your statement.)

If #1689 is adopted, then there won't be need for a separate bridge, as OTel will natively understand Spans from tracing, and whole class of issues like this will just disappear.

#1689 seems like the right direction to me -- I think that a way to provide tracing_subscriber::Subscriber implementations that bridge to other OTel-enabled tooling would effectively come down to the same thing?

[djc]

@lalitb thanks for the help. Based on your changes, I was able to update tokio-rs/tracing-opentelemetry#155 so that it passes CI. I think it would be great if this can be merged and published.

[lalitb]

Thanks for the PR @djc. Patch release is done for opentelemetry_sdk.

[djc]

Thanks!

[cijothomas]

I previously found being involved in the opentelemetry-rust project pretty frustrating

I'd be really interested to know more on this. The current maintainers (myself included!) would definitely benefit learning about this from folks who previously maintained the library.
Are the otel specs part frustrating? Or something else?

[cijothomas]

I do think the opentelemetry-rust maintainers should consider tracing-opentelemetry an integral part of the project since, as far as I can tell, a majority of the opentelemetry-rust tracing consumers probably use it. So I would suggest we invert the burden of maintenance and say that openteleemtry-rust maintenance responsibility should extend out towards tracing-opentelemetry at least for trace-related public API changes

I am very supportive of the idea of making sure tracing users are happy when integrating with OpenTelemetry. But the current state of special casing for tracing-opentelemetry is not sustainable, without a clear long term plan how these 2 ecosystems want to happily co-exist. And that is why #1571 is opened to find out that long term plan.

[djc]

I previously found being involved in the opentelemetry-rust project pretty frustrating

I'd be really interested to know more on this. The current maintainers (myself included!) would definitely benefit learning about this from folks who previously maintained the library. Are the otel specs part frustrating? Or something else?

IMO low-level libraries like this benefit from careful design, and this hasn't always happened in the past. I feel that current maintainers have been more focused on the specs than on providing a good experience for downstream consumers (especially considering that I feel like most downstream consumers are instrumenting their applications with the tracing crate). I also dislike the move to synchronous meetings (with, AFAICT, not much in the way of written notes/discussion) and the Slack workspace because they make it harder to participate and seem to shift the focus away from a technical project that tries to provide building blocks for projects that need observability.

[cijothomas]

IMO low-level libraries like this benefit from careful design, and this hasn't always happened in the past.

Agree! We are trying out best to fix that problem. Its still in-progress, but most work has been in Logs, Metrics area so far.

feel that current maintainers have been more focused on the specs than on providing a good experience for downstream consumers

I don't observe that anymore. I personally treat spec as a guidance, not something I follow literally, but I absorb its "spirit". This is what I have seen other maintainers also doing!

If there is a specific instance of not focusing on "good experience for downstream consumers", very glad to work on rectifying that!

I also dislike the move to synchronous meetings (with, AFAICT, not much in the way of written notes/discussion)

I don't see that problem. Sync meetings are optional, but is very good to sort out hard PRs/issues. The meetings are recorded always, and the meeting notes are also publicly available.

the Slack workspace because they make it harder to participate

I am not sure why. Some people don't use slack I understand, but OpenTelemetry ecosystem prefers slack. (However, I believe most of OTel Rust folks can be reached out in Discord too!)

seem to shift the focus away from a technical project that tries to provide building blocks for projects that need observability.

Not sure I am following this. Maybe you can clarify with some specifics where focus is not there!

Thanks a lot for sharing your thoughts/feedback. If there are things that is within the power of current maintainers (@open-telemetry/rust-maintainers ) to make this community more welcoming/less frustrating, we are always open!