How should OpenTelemetry-internal metrics be exposed?

[toumorokoshi]

What are you trying to achieve?

From a conversation in #942, there is a need to have metrics internal to opentelemetry. This is a ticket to start the discussion of how that would work.

Design decisions include:

• what meter would those metrics emit to?
• how would components of opentelemetry emit them?

This may also be better started in a SIG discussion.

[jkwatson]

Just as a reference the Java SDK is currently emitting metrics for our BatchSpanProcessor: https://github.com/open-telemetry/opentelemetry-java/blob/ddb38970fabadd82e7fa875789d74b9bd29fd257/sdk/tracing/src/main/java/io/opentelemetry/sdk/trace/export/BatchSpanProcessor.java#L140-L152

[lzchen]

@jkwatson
Couple of questions regarding what Java is doing.

1. How is the SpanProcessor able to see spans that are dropped/sampled out? According to the spec, if they are dropped the processor should not be receive them. How are you able to get the "droppedSpans" counter in this case?
2. Should this be present for all SpanProcessor s?

[Oberon00]

@lzchen I think that code is from the time when DROP was called NOT_RECORD, so when it uses "dropped" it talks about (sampled) spans being dropped from the queue of the batch span processor (due to it being full), not spans being dropped by the sampler.

[toumorokoshi]

Following back up on this one. Looking at the java implementation, the answers to my questions above are:

• use whatever MeterProvider is set to create meters
• use the package or module namespace as the name

One question I would wonder: how would one turn off all opentelemetry metrics, in such a setup? do exporters or MeterProviders provide some mechanism to no-op a whole namespace?

[lzchen]

it talks about (sampled) spans being dropped from the queue of the batch span processor (due to it being full), not spans being dropped by the sampler.

@jkwatson
I see. So does that mean that the implementation is only keeping a count of spans that are sampled (spans that are not "dropped" by the sampler). I'm wondering would it even make sense to keep a count of ALL spans, regardless of whether or not they are sampled.

[jkwatson]

it talks about (sampled) spans being dropped from the queue of the batch span processor (due to it being full), not spans being dropped by the sampler.

@jkwatson
I see. So does that mean that the implementation is only keeping a count of spans that are sampled (spans that are not "dropped" by the sampler). I'm wondering would it even make sense to keep a count of ALL spans, regardless of whether or not they are sampled.

Perhaps, but that's not the metric that I'm recording in the BatchSpanProcessor. :)
There are probably more metrics needed in the SDK...I just implemented the one I did because I needed it, and thought it would be useful.

[objectiser]

Is there an intention to define a common set of internal metrics across all SDK impls?

@jkwatson currently I don't think the java sdk is recording a metric related to queue size. Do you think this would be generally useful (i.e. to track how close a tracer is getting to dropping spans)? If so, I'll create an issue in the java repo.