Clarify MetricReader parameters and temporality controls

[jmacd]

Fixes #2403.
Replaces #2314.
Replaces #2407.
Fixes #2065.
Part of #2072.

Changes

Clarify the role of the MetricReader in determining the default aggregation, temporality, and default_enabled properties. Clarify the Metric Exporter's "associated" MetricReader and that when auto-configuring an exporter, the Metric Reader's defaults MAY be determined in ways specific to the exporter.

The present form of this PR has a single environment variable to control variable temporality of the OTLP exporter. Note that with 5 instruments having temporality and 2 choices, there are 32 possible settings for temporality across the instruments. I would prefer not to have 5 environment variables, because not very many of the configurations are actually expected.

Therefore, this proposal uses a single environment variable with two specified values, CUMULATIVE (the default), DELTA (where Counters, Async Counters and Histograms all use Delta, UpDownCounter and Async UpDownCounter uses Cumulative).

There's a third value I've proposed in #2314, that would be STATELESS, where synchronous instruments choose Delta and asynchronous instruments choose Cumulative. I do not think we should have 5 environment variables, because not many more than 3 out of the possible 32 combinations are actually going to happen. I'm happy to leave STATELESS out of the current proposal because (speaking as a vendor), my needs are met without it. Given that a MetricReader supports configuring the default temporality as a function of instrument kind, we're able to fill in any of the 29 unspecified temporality combinations through a programmatic interface. It means we will be able to set the STATELESS behavior in our launchers even if the environment variable does not recognize that setting.

[bogdandrutu]

LGTM, I think this clarifies more, but I have a small question about the need of having "preferred" temporality at all.

[jmacd]

I have prepared #2407, which is an alternative to this one. This specifies more, #2407 specifies less.

[codeboten]

Just a minor comment, otherwise i think this provides a good path moving forward.

[alanwest]

I'm liking the direction of this PR. I just had a few questions for clarification.

#2407 alone as discussed in today's SIG meeting won't work - that is we need to continue to maintain some way to configure aggregation temporality. even if the intent was for 2407 to be an intermediate step.

[jmacd]

FYI I will work to clarify the questions posed in this PR today.

[jmacd]

@open-telemetry/specs-metrics-approvers I've expanded the scope of this PR following Tuesday's SIG meeting. Mainly, I restored the OTLP exporter environment variable that expresses what was formerly known as a "preference". Here I'm defining this as the "default aggregation temporality policy", because the default aggregation temporality is a MetricReader setting that is made "on the basis of instrument kind".

In the latest round of editing, I tried to strengthen our understanding of MetricReader. It's not an interface the way I understand it, it's more like an abstract base class for the thing that registers exporters and provides their default settings. Please take a look.

[cijothomas]

LGTM.

[bogdandrutu]

This PR definitely clarifies who is responsible of doing what. I like that MetricExporter does not have a "preferred temporality API" (is this true?) anymore.

[jmacd]

@bogdandrutu and @cijothomas please see if efb040f helps.

I like that MetricExporter does not have a "preferred temporality API" (is this true?) anymore.

Yes, there's no API like that in the exporter in my understanding. However, when the SDK auto-configures an exporter there may be a need to configure temporality on a case-by-case basis, with the OTLP exporter being a prime example.

[dyladan]

@bogdandrutu and @cijothomas please see if efb040f helps.

I like that MetricExporter does not have a "preferred temporality API" (is this true?) anymore.

Yes, there's no API like that in the exporter in my understanding. However, when the SDK auto-configures an exporter there may be a need to configure temporality on a case-by-case basis, with the OTLP exporter being a prime example.

This is the only point I'm still not completely sure on. Why would we make users configure the temporality manually with the metric reader rather than the metric reader getting temporality configuration from the exporter?

To be clear I am not trying to block or delay this PR at all with this question, merely understand the reasoning for that change.

[cijothomas]

@bogdandrutu and @cijothomas please see if efb040f helps.

I like that MetricExporter does not have a "preferred temporality API" (is this true?) anymore.

Yes, there's no API like that in the exporter in my understanding. However, when the SDK auto-configures an exporter there may be a need to configure temporality on a case-by-case basis, with the OTLP exporter being a prime example.

This is the only point I'm still not completely sure on. Why would we make users configure the temporality manually with the metric reader rather than the metric reader getting temporality configuration from the exporter?

To be clear I am not trying to block or delay this PR at all with this question, merely understand the reasoning for that change.

There is the following wording in the spec "OpenTelemetry language implementations MAY support automatically configuring the MetricReader to use for an Exporter".
Which I interpret as - If an SDK wants to make it easier for users, it can, by allowing Reader to automatically infer the temporality from the Exporter. Spec is not prohibiting you from doing that. Neither it is enforcing to offer that capability.

Also spec is silent on "How to infer the temporality from an Exporter". I think this is intentional, as languages may have some idiomatic way of expressing it.

[jmacd]

@dyladan I like @cijothomas's answer. It does feel natural for an exporter to make available its preferred temporality, but I'm not sure about making it a requirement; auto-configuring exporters would have a degree of consistency that I suspect you're after.

Since we have some exporters where there are more than one reasonable temporality configuration, I feel that forcing the exporter to provide a preference makes more work for exporters, since if the user wants to change the configuration they have to have a way to change every specific exporter's configuration. Since exporters are 1:1 with readers, it seems simpler to specify that readers support this configuration change so that every exporter doesn't have to. Still, I can see adding a mechanism to get the preferred default when auto-configuring exporters. For the OTLP Exporter, it would interpret the environment variable to decide. This sounds like a good design, not sure it has to be specified this way.

[bogdandrutu]

@cijothomas

Which I interpret as - If an SDK wants to make it easier for users, it can, by allowing Reader to automatically infer the temporality from the Exporter. Spec is not prohibiting you from doing that. Neither it is enforcing to offer that capability.

Also spec is silent on "How to infer the temporality from an Exporter". I think this is intentional, as languages may have some idiomatic way of expressing it.

I would definitely not do that by having a new concept "preferred" temporality with some values that are not defined by the spec. But that is just me, I would simply expose a otlp.NewReader() a "factory method" to create reader + exporter in that exporter package instead.