Add predicate to MetricReader and MetricProducer

[asafm]

Fixes #3324

Changes

As described in #3324 , there is a need to have the ability to filter out selected (instrument, attribute) pairs at the MetricReader Collect operation. For efficiency reasons, this needs to be pushed down to the MetricProducer(s) - both the SDK's one and the registered external ones. Implementations are encouraged to use the predicate to avoid allocating memory allocation for the resulting data point if the predicate filters it out.

TODO

• Fill CHANGELOG
• Fill spec compliance matrix
• Update TOC of sdk.md

[asafm]

@reyang @jack-berg Following our Zoom call, here is the PR I've prepared to match the proposal detailed in the issue #3324.

@reyang In our discussion you mentioned a different model where we will have a processor which will have an optional filter associated with it, so we can know whether we should stream metric data point to the processor, from the metric producer. You said you will think about it.
In theory we can in the future just add processor "function" which will be executed only on data points matching the filter, so I think this proposal doesn't block that option from happening in the future.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[asafm]

I'm working on a big adjustment to it, please don't close it

[asafm]

@reyang @jack-berg I've revamped the PR in the last commit, following the filter rules approach. Your feedback is much appreciated.

[marcalff]

Thanks for driving this, I support this change.

My only comment here is about naming, to open up the possibility to also have predicates and filters on meters as a whole, not individual metrics.

Please consider renaming like:

• Filter -> InstrumentFilter or MetricFilter
• Predicate -> InstrumentPredicate or MetricPredicate

etc.

Of course, Meter predicates will be an entirely separate discussion and PR, not to take place here, but I just would like to prevent future naming collisions.

[jack-berg]

Does a prototype exist for this anywhere? The contribution guidelines specify that prototypes are required to help evaluate. We recently re-affirmed this expectation in #3574.

I ask because the motivation for this is primarily performance oriented. As we've discussed, there's presumably some performance overhead for a reader accessing the aggregated state of all instruments, and for invoking all the callbacks. The predicate stands to avoid some of this overhead. The iteration of this PR which includes rules seems adds a lot of surface area relative to the original version which just required implementations to expose some way to specify a predicate to readers. I want to know what kind of performance improvement we stand to gain by introducing this new surface area.

To be frank, its a lot of new surface area / concepts for what I currently view as a niche use case.

[asafm]

@dashpole Did I add the changelog and compliance matrix correctly?

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[asafm]

@dashpole Did I add the changelog and compliance matrix correctly?

[dashpole]

Changelog and matrix look correct.

[asafm]

So, what is the next step before merging @dashpole ?