Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backend OTLP: Adding OTLP as a configurable backend for GoStatsD #656

Merged
merged 8 commits into from
May 24, 2024

Conversation

MovieStoreGuy
Copy link
Contributor

@MovieStoreGuy MovieStoreGuy commented Jan 11, 2024

Context

This adds in support for OTLP as a supported backend for Gostatsd however it does come with some limitations:

  • Events are no (currently) supported
  • Slight processing overhead converting from internal metrics values to OTLP values

Events will need to be exported as logs which currently isn't written yet and ignored for this PR.

Migration

To ensure the highest amount of compatibility with other configurable backends, timers can either exported as Gauges with the calculated value added as suffix to the metric. This should ensure that existing metrics being emitted to your configured backend should maintain the same semantic value when visualising / evaluating within metrics platform.

Outstanding

  • Adding tests for Backend
  • Limiting the amount of concurrent requests being made
  • Respecting batching limits
  • Cleanup unused helper functionality

pkg/backends/otlp/backend.go Show resolved Hide resolved
pkg/backends/otlp/group.go Outdated Show resolved Hide resolved
@MovieStoreGuy MovieStoreGuy marked this pull request as ready for review January 15, 2024 13:02
@MovieStoreGuy MovieStoreGuy force-pushed the msg/add-consuming-metrics branch from 008923b to 69a647a Compare January 15, 2024 13:08
@MovieStoreGuy MovieStoreGuy force-pushed the msg/add-consuming-metrics branch from 69a647a to 3a97091 Compare January 15, 2024 13:09
@MovieStoreGuy
Copy link
Contributor Author

I've parked the additional actions, I would like to follow up on those to be done within the transport instead of within the encoding step.

@MovieStoreGuy MovieStoreGuy force-pushed the msg/add-consuming-metrics branch from 3a97091 to 7d31e1d Compare May 23, 2024 04:02
@MovieStoreGuy MovieStoreGuy requested a review from Excaleo May 23, 2024 04:06
func newDefaultConfig() *Config {
return &Config{
Transport: "default",
MaxRequests: runtime.NumCPU() * 2,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this too conservative?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it really depends on the backend supporting out of order writes, if it does, you can open this up wide, otherwise, restricting it is a good idea.

Limit the amount of concurrent request to `maxRequests`
@MovieStoreGuy MovieStoreGuy merged commit 93434ad into master May 24, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants