Add details for Aggregator in Metric spec

[victlu]

Add into Metric spec details for defining a Metric Aggregator.

[reyang]

Do we intend to cover MIN and MAX?

[reyang]

Some high level questions:

1. Is Aggregator intended to be exposed by the SDK as a type/class? (my answer would be NO, we don't need such type/class, we probably want some enum/flags that we can consume while defining the View).
2. If the answer to 1) is yes, do we intend to provide extensibility (e.g. one could build their own Aggregator)? (my answer would be eventually YES, for the first stable release probably NO)
3. If the answer to 1) is no, what's the purpose or it is just the SDK internal details?
4. Do we allow multiple aggregators to co-exist for the same event(s)? For example, if I have temperature, could I have Histogram, Min/Max and Average at the same time? If the answer is yes, do we end up with one stream or multiple streams? (my answer would be - we'll have multiple streams)

[jkwatson]

I was surprised not to see any mention of attributes/labels in here, and how an Aggregator needs to handle them. Is this something that is just so obvious that it doesn't need mentioning, or was this missed in writing up the details?

[victlu]

Do we intend to cover MIN and MAX?

An Aggregator is intended to target a Metric Point in Metrics Data Model. Thus, an Aggregator internally (in-memory state) stores a composite of data required to generate a Metric Point (Sums, Gauge, Histogram).

Currently, our OTLP protocol and Metric Points do not require MIN and/or MAX. ("Legacy" Summary Metric Point has that concept but wrapped as Quantiles).

[victlu]

I was surprised not to see any mention of attributes/labels in here, and how an Aggregator needs to handle them.

Handling of Attributes is indeed a central concept. I did have a few mention and examples of how to address this. But, it likely needs it's own section to clarify. In the meantime, I did add a statement regarding SDK needing to route/map measurements to aggregators. It needs to be more flushed out in spec-level details.

In summary, A MeasurementProcessor ("Aggregator MeasurementProcessor") will route a measurement to zero or more pre-configured Aggregators. This Processor supports a mapping scheme which maps Attributes to Key/Value pair combinations. And for support of multiple exporters, mapping scheme which map per collection frequency.

At the last SIG meeting, I asked where this "routing" task should reside. As part of SDK (which I currently have spec) or part of a MeasurmentProcessor (call it the "Aggregator" measurement processor?). This is unfortunately confusing between a "Aggregator MeasurementProcessor" and an Aggregator.

Aggregator MeasurmentProcessor = concern with taking a measurement and breaking it up to zero or more instances.
Aggregator = concern with how to "aggregate" (sum/count/bucket/etc...) values.

[victlu]

1. Is Aggregator intended to be exposed by the SDK as a type/class? (my answer would be NO, we don't need such type/class, we probably want some enum/flags that we can consume while defining the View).

I think this is more of a style question. In the end, we need to expose the concept of an aggregation/aggregator and potentially any configuration (e.g. Monotonic and/or Cumulative/Delta and/or Bucket scaling).

Enum

  .AddView(..., ENUM_AGGREGATOR_SUM);      // Use "default"
  .AddView(..., ENUM_AGGREGATOR_SUM_MONOTONIC_DELTA);

pro:

• Only need to expose one ENUM type
• If we support file based configs (i.e. json/xml), ENUMs are easier to encode.

con:

• Need to enumerate all possible configurations upfront.
• Need to update ENUM if we extend in future. And any mapping code for Enum to aggregator.
• Need different method to pass in hints.

Class/Type

  .AddView(..., new SumAggregator());     // Use "default"
  .AddView(..., new SumAggregator(monotonic=true, temporality=DELTA));
  .AddView(..., new HistogramAggregator(hint=...));    // Take in hint

pro:

• Allows for finer control (e.g. new Histogram(bucket_boundaries=new int[] { 1, 10, 100 } ;)
• Potential for easy extensions in future.

con:

• Need to expose several classes/types.

[victlu]

4. Do we allow multiple aggregators to co-exist for the same event(s)? For example, if I have temperature, could I have Histogram, Min/Max and Average at the same time? If the answer is yes, do we end up with one stream or multiple streams? (my answer would be - we'll have multiple streams)

I also think it would be valuable to allow one measurement to generate multiple metrics. (i.e. a Temperature Instrument can produce a Gauge timeseries and a Histogram timeseries and a Summary timeseries.

An Aggregator as defined here, does not care, as each instance of an aggregator represent one timeseries. The SDK is poised to route a measurement to zero or more aggregators as configured by the View API.

[weyert]

Do we want to specify a default unit of measure for this?

[victlu]

After discussion in SIG meeting, a new PR is created #1842 to continue with this spec.