New metrics SDK #1000

jtescher · 2023-03-19T04:49:12Z

This patch updates the metrics SDK to the latest spec. The following breaking changes are introduced.

Metrics API changes:

Move AttributeSet to SDK as it's not mentioned in the spec or used in the api
Consolidate AsyncCounter, AsyncUpDownCounter, and AsyncGauge into AsyncInstrument trait and add downcasting for observer callbacks.
Add AsyncInstrumentBuilder to allow per-instrument callback configuration.
Allow metric name and description fields to be Cow<'static, str>
Warn on metric misconfiguration when using instrument builder init rather than returning error
Update Meter::register_callback to take a list of async instruments and validate they are registered in the callback through the associated Observer
Allow registered callbacks to be unregistered.

Metrics SDK changes:

Introduce Scope as type alias for InstrumentationLibrary
Update Aggregation to match aggregation spec
Refactor BasicController to spec compliant ManualReader
Refactor PushController to spec compliant PeriodicReader
Update metric data fields to match spec, including exemplars.
Split MetricsExporter into Readers and PushMetricExporters
Add View implementation
Remove AtomicNumbers
Refactor Processors into Pipeline

Metrics exporter changes:

Update otlp exporter to match new metrics data
Update otlp exporter configuration to allow aggregation and temporality selectors to be optional.
Update prometheus exporter to match new metrics data

Example changes:

Update otlp metrics and prometheus examples.
Remove basic example as we should be focusing on the OTLP variants

Given the size of the changes I would have preferred to land them individually, but as they all rely on each other it was not practical.

Resolves #897

This patch updates the metrics SDK to the latest spec. The following breaking changes are introduced. Metrics API changes: * Move `AttributeSet` to SDK as it's not mentioned in the spec or used in the api * Consolidate `AsyncCounter`, `AsyncUpDownCounter`, and `AsyncGauge` into `AsyncInstrument` trait and add downcasting for observer callbacks. * Add `AsyncInstrumentBuilder` to allow per-instrument callback configuration. * Allow metric `name` and `description` fields to be `Cow<'static, str>` * Warn on metric misconfiguration when using instrument builder `init` rather than returning error * Update `Meter::register_callback` to take a list of async instruments and validate they are registered in the callback through the associated `Observer` * Allow registered callbacks to be unregistered. Metrics SDK changes: * Introduce `Scope` as type alias for `InstrumentationLibrary` * Update `Aggregation` to match aggregation spec * Refactor `BasicController` to spec compliant `ManualReader` * Refactor `PushController` to spec compliant `PeriodicReader` * Update metric data fields to match spec, including exemplars. * Split `MetricsExporter` into `Reader`s and `PushMetricExporter`s * Add `View` implementation * Remove `AtomicNumber`s * Refactor `Processor`s into `Pipeline` Metrics exporter changes: * Update otlp exporter to match new metrics data * Update otlp exporter configuration to allow aggregation and temporality selectors to be optional. * Update prometheus exporter to match new metrics data Example changes: * Update otlp metrics and prometheus examples. * Remove basic example as we should be focusing on the OTLP variants

opentelemetry-otlp/src/metric.rs

opentelemetry-sdk/src/metrics/aggregation.rs

examples/basic-otlp/src/main.rs

examples/basic/src/main.rs

…acks

examples/basic-otlp/Cargo.toml

opentelemetry-otlp/Cargo.toml

hdost

Great work overall! Don't see any blockers, and I think anything that we missed on this go we can iterate on 😄

TommyCpp · 2023-03-23T21:43:17Z

Sorry was little busy recently. I will finish reviewing this weekend if that's OK. Or we can merge this and address any issue in the followup PR. Overall great work 🎉

hdost · 2023-03-24T08:14:04Z

Sorry was little busy recently. I will finish reviewing this weekend if that's OK. Or we can merge this and address any issue in the followup PR. Overall great work tada

I guess I thought we were waiting until after 0.19.0 to merge this? Or are we just not releasing 0.19.0 😅

TommyCpp · 2023-03-25T19:37:39Z

Sorry was little busy recently. I will finish reviewing this weekend if that's OK. Or we can merge this and address any issue in the followup PR. Overall great work tada

I guess I thought we were waiting until after 0.19.0 to merge this? Or are we just not releasing 0.19.0 😅

Haha yeah, we want to after 0.19. I was worried I will block others from merging the release PR and then this PR. I will #1002 today

# Conflicts: # opentelemetry-dynatrace/Cargo.toml # opentelemetry-otlp/Cargo.toml # opentelemetry-prometheus/Cargo.toml

TommyCpp

Mostly nits and can be addressed in follow-up PRs

opentelemetry-sdk/src/metrics/view.rs

opentelemetry-api/src/metrics/meter.rs

opentelemetry-prometheus/src/lib.rs

opentelemetry-sdk/src/metrics/data/mod.rs

opentelemetry-sdk/src/metrics/pipeline.rs

Co-authored-by: Zhongyang Wu <zhongyang.wu@outlook.com>

codecov · 2023-03-27T18:44:24Z

Codecov Report

Patch coverage: 39.1% and project coverage change: -12.8 ⚠️

Comparison is base (879d6ff) 69.9% compared to head (0ea2e6d) 57.2%.

Additional details and impacted files

@@           Coverage Diff            @@
##            main   #1000      +/-   ##
========================================
- Coverage   69.9%   57.2%   -12.8%     
========================================
  Files        116     143      +27     
  Lines       9027   17461    +8434     
========================================
+ Hits        6316    9999    +3683     
- Misses      2711    7462    +4751

Impacted Files	Coverage Δ
opentelemetry-api/src/lib.rs	`100.0% <ø> (ø)`
...entelemetry-api/src/metrics/instruments/counter.rs	`30.3% <0.0%> (-19.7%)`	⬇️
opentelemetry-api/src/metrics/instruments/gauge.rs	`0.0% <0.0%> (-42.9%)`	⬇️
...try-api/src/metrics/instruments/up_down_counter.rs	`27.7% <0.0%> (-6.9%)`	⬇️
opentelemetry-api/src/metrics/mod.rs	`7.0% <0.0%> (-4.4%)`	⬇️
opentelemetry-api/src/metrics/noop.rs	`5.6% <0.0%> (-28.7%)`	⬇️
opentelemetry-otlp/src/lib.rs	`29.7% <ø> (ø)`
opentelemetry-otlp/src/metric.rs	`0.0% <0.0%> (ø)`
opentelemetry-otlp/src/span.rs	`34.9% <ø> (+34.9%)`	⬆️
opentelemetry-otlp/src/transform/metrics.rs	`0.0% <0.0%> (ø)`
... and 29 more

... and 104 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

jtescher · 2023-03-27T19:33:57Z

@TommyCpp any other comments or we good to merge?

TommyCpp · 2023-03-28T03:22:38Z

@TommyCpp any other comments or we good to merge?

Nope good to merge. Great work!

ref: open-telemetry/opentelemetry-rust#1000 ref: open-telemetry/opentelemetry-rust#2221 ref: open-telemetry/opentelemetry-rust#2085

* chore(deps): cargo * wip: use axum & cargo fmt * wip: remove `once_cell` * wip: upgrade opentelemetry ref: open-telemetry/opentelemetry-rust#1000 ref: open-telemetry/opentelemetry-rust#2221 ref: open-telemetry/opentelemetry-rust#2085 * chore: cargo clippy * chore(deps): upgrade go * version: upgrade to 0.6.0 * chore(deps): use rkyv * chore(deps): update * chore(deps): update * wip: split mods into crates * wip: remove unused dependencies * chore(ci): upgrade rust version in Dockerfile * fix(ci): update cargo install path * chore: tidy up main.rs * fix(test): review snaps & fix tests * chore(deps): update * fix(cgroup): early stop when use `map_while` * chore: init logger before setup cgroup * chore(deps): ebpf version restore to v0.12.3 * chore(lazylock): use `LazyLock::force` to init cache * fix(cache): remove unnecessary async * chore(config): make default functions const

jtescher requested a review from a team March 19, 2023 04:49

jtescher force-pushed the new-metrics branch 2 times, most recently from b04f672 to 970884a Compare March 19, 2023 05:30

jtescher force-pushed the new-metrics branch from 970884a to e242ae0 Compare March 19, 2023 05:30

BrynCooke reviewed Mar 19, 2023

View reviewed changes

opentelemetry-otlp/src/metric.rs Show resolved Hide resolved

jtescher added 2 commits March 19, 2023 11:25

Update metric benchmarks

5874ac9

Fix periodic reader mutex locks

be4b49f

jtescher mentioned this pull request Mar 20, 2023

release(v0.19): add initial change log #965

Merged

jtescher added 2 commits March 19, 2023 19:36

Add more benchmark tests

e6a5afb

Merge branch 'main' into new-metrics

8d80bce

cijothomas reviewed Mar 21, 2023

View reviewed changes

opentelemetry-sdk/src/metrics/aggregation.rs Outdated Show resolved Hide resolved

Rename histogram aggregation field to record_min_max

16f9907

cijothomas reviewed Mar 21, 2023

View reviewed changes

examples/basic-otlp/src/main.rs Show resolved Hide resolved

cijothomas reviewed Mar 21, 2023

View reviewed changes

examples/basic/src/main.rs Show resolved Hide resolved

jtescher added 2 commits March 21, 2023 13:11

Remove context references in callbacks per the spec

4e5eeb2

Implement MeterProvider shutdown spec and individual instrument callb…

6ca2749

…acks

hdost reviewed Mar 23, 2023

View reviewed changes

examples/basic-otlp/Cargo.toml Show resolved Hide resolved

opentelemetry-otlp/Cargo.toml Outdated Show resolved Hide resolved

hdost approved these changes Mar 23, 2023

View reviewed changes

hdost mentioned this pull request Mar 23, 2023

docs: Followup updates to #989 #1004

Closed

hdost mentioned this pull request Mar 24, 2023

Proposal: BasicController to MeterProvider #976

Closed

jtescher added 6 commits March 26, 2023 16:49

Merge branch 'main' into new-metrics

2ffa68a

# Conflicts: # opentelemetry-dynatrace/Cargo.toml # opentelemetry-otlp/Cargo.toml # opentelemetry-prometheus/Cargo.toml

Remove sleep from examples

6d722b8

Fix tests

7ab6437

Fix coverage

c5718ba

Switch to all workspace coverage

e91b7cf

Fix clippy lints

c57d90b

Switch to llvm-cov for coverage

0599689

TommyCpp reviewed Mar 27, 2023

View reviewed changes

jtescher and others added 3 commits March 27, 2023 09:45

Review comment

98d0c02

Co-authored-by: Zhongyang Wu <zhongyang.wu@outlook.com>

Address comments from review

3f280a6

Fix noop docs

ccefdb6

Ignore proto files in coverage

0ea2e6d

hdost approved these changes Mar 28, 2023

View reviewed changes

jtescher merged commit e4ef2a9 into open-telemetry:main Mar 28, 2023

jtescher deleted the new-metrics branch March 28, 2023 18:31

cijothomas mentioned this pull request Jun 8, 2023

New release #1100

Closed

jtescher mentioned this pull request Jul 19, 2023

Prometheus exporter: Use OTel Resource for target_info #929

Closed

Matthias247 mentioned this pull request Aug 23, 2023

Reset ObservableGauge for all cached attributes sets to 0 #1221

Closed

GZTimeWalker added a commit to darkyzhou/seele that referenced this pull request Jan 5, 2025

wip: upgrade opentelemetry

09da1ec

ref: open-telemetry/opentelemetry-rust#1000 ref: open-telemetry/opentelemetry-rust#2221 ref: open-telemetry/opentelemetry-rust#2085

GZTimeWalker mentioned this pull request Jan 5, 2025

Upgrade deps and dump version to 0.6.0 darkyzhou/seele#18

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New metrics SDK #1000

New metrics SDK #1000

jtescher commented Mar 19, 2023

hdost left a comment

TommyCpp commented Mar 23, 2023

hdost commented Mar 24, 2023

TommyCpp commented Mar 25, 2023

TommyCpp left a comment

codecov bot commented Mar 27, 2023 •

edited

Loading

jtescher commented Mar 27, 2023

TommyCpp commented Mar 28, 2023

New metrics SDK #1000

New metrics SDK #1000

Conversation

jtescher commented Mar 19, 2023

hdost left a comment

Choose a reason for hiding this comment

TommyCpp commented Mar 23, 2023

hdost commented Mar 24, 2023

TommyCpp commented Mar 25, 2023

TommyCpp left a comment

Choose a reason for hiding this comment

codecov bot commented Mar 27, 2023 • edited Loading

Codecov Report

jtescher commented Mar 27, 2023

TommyCpp commented Mar 28, 2023

codecov bot commented Mar 27, 2023 •

edited

Loading