Add support for configurable OTLP export strategy with cumulative default strategy #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
huyan0
requested review from
alolita,
ankit-bhargava,
Brandon-Kimberly,
connorlindsey,
CunjunWang,
dabangarang,
EdZou,
ercl,
erichsueh3 and
HudsonHumphries
connorlindsey
approved these changes
Aug 5, 2020
huyan0
changed the title
connorlindsey
approved these changes
Aug 7, 2020
jmacd
approved these changes
Aug 8, 2020
|
|
||
| To support Prometheus as well as delta backends, the SDK should be configurable and supports OTLP exporter to have both cumulative default or delta export behavior. The implication is that the OTLP metric protocol should support both cumulative and delta report strategy. | ||
|
|
||
| Users should be allowed to declare an environment variable or configuration field that determines this setting for OTLP exporters. |
huyan0
commented
Aug 10, 2020
|
|
||
| To support cumulative export strategy, the SDK needs to maintain state of each cumulative metrics. This means users with high-cardinality metrics can experience high memory usage. | ||
|
|
||
| This problem could be addressed by adding support in the Collector, when configured as an Agent, to support converting delat OTLP to Cumulative OTLP, but this requires a single agent for each metric generating client so that all delta values of a metric are converted at the same location. |
dabangarang
reviewed
Aug 10, 2020
|
|
||
| ## Motivation | ||
|
|
||
| 1. Metric backends such as Prometheus and Prometheus Remote Write integrated storages requires cumulative values for cumulative/adding metrics rather than delta value per collection interval. Thus, to export metrics generated by the SDK using the collector, incoming values from the SDK should be in cumulative form. However, backends like Statsd expects delta value of each collection interval. To support different backends, OTLP metric export behavior should be configurable, with cumulative export as default. The metrics SIG meeting concluded that cumulative export behavior is more reliable, as explained in the following section. |
There was a problem hiding this comment.
Maybe say "backends that make use of the Prometheus Remote Write API"
There was a problem hiding this comment.
OTLP Configurable Export Behavior
Add support for configurable behavior in OTLP exporters
Based on discussions from the Metrics SIG meeting, configurable export behavior from the OTLP exporter is desired to support different mertic backends
Based on discussions from the Metrics SIG meeting, configurable export behavior from the OTLP exporter is desired to support different mertic backends
open-telemetry#126) * Add a proposal for SDK configurations for metric aggregation. * rename file to match the PR # * fix markdown lint issues * clarify that this applies to the default SDK, and fill out the open questions * update the java example to fix the naming changes * Add another open question to the list. * Update text/0126-Configurable-Metric-Aggregations.md Co-authored-by: Chris Kleinknecht <libc@google.com> * Update to use the 'view' terminology * another configuration/view replacement * Add a few more open questions, and a note that they will be resolved in the spec. * Update text/0126-Configurable-Metric-Aggregations.md Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com> Co-authored-by: Chris Kleinknecht <libc@google.com> Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com> Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
…0126-Configurable-Metric-Aggregations.md (open-telemetry#132)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR describes and records discussions around spec #731, #725 and the overall conclusion from Metrics SIG meetings.
cc: @alolita @huyan0