-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[exporter/datadogexporter] Retry per network call #6412
[exporter/datadogexporter] Retry per network call #6412
Conversation
…tings into account
a4c6d66
to
2366398
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This comment has been minimized.
This comment has been minimized.
@@ -156,7 +156,8 @@ func createMetricsExporter( | |||
pushMetricsFn, | |||
// explicitly disable since we rely on http.Client timeout logic. | |||
exporterhelper.WithTimeout(exporterhelper.TimeoutSettings{Timeout: 0 * time.Second}), | |||
exporterhelper.WithRetry(cfg.RetrySettings), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like a breaking change. Could you add this to the changelog? Make sure also to add this to the readme. Is there a way to determine whether the retry settings were customized by users, and warn them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Retry settings will still be used here
opentelemetry-collector-contrib/exporter/datadogexporter/internal/utils/retrier.go
Lines 53 to 61 in 3e21ff7
expBackoff := backoff.ExponentialBackOff{ | |
InitialInterval: r.cfg.InitialInterval, | |
RandomizationFactor: backoff.DefaultRandomizationFactor, | |
Multiplier: backoff.DefaultMultiplier, | |
MaxInterval: r.cfg.MaxInterval, | |
MaxElapsedTime: r.cfg.MaxElapsedTime, | |
Stop: backoff.Stop, | |
Clock: backoff.SystemClock, | |
} |
ConsumeMetrics
function (so, we disable the exporterhelper
retry mechanism). I believe this change should be transparent to users and it will probably align closer with what one would expect from the retry settings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it! Makes sense then.
* [exporter/datadogexporter] Refactor metadata retrier to take RetrySettings into account * [exporter/datadogexporter] Do retries per network call on the metrics side * Address review comment * Fix lint
Signed-off-by: Bogdan <bogdandrutu@gmail.com>
Description:
Our
Push[Metric/Trace]Data
sometimes do several network calls per OTLP payload.In the case of metrics these are safe to retry so we retry them per network call.
In the case of traces it's unclear when it is safe to retry so I am disabling them for now.
Link to tracking Issue: n/a
Testing: Added unit tests. Manual tests to be done are:
Documentation: Added a comment about retry settings on traces.