Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup OpenTelemetry exporters when initializing prefect #15946

Merged
merged 5 commits into from
Nov 8, 2024

Conversation

bunchesofdonald
Copy link
Contributor

This PR adds OpenTelemetry support to Prefect Cloud by setting up exporters for traces, metrics, and logs during initialization when enabled.

Key changes:

  • Adds experimental setting to enable/disable telemetry
  • Add OTLP exporters for traces, metrics, and logs
  • Implement custom InFlightSpanProcessor for handling long-running spans
  • Configure telemetry endpoint URL and auth based on existing Prefect settings

@bunchesofdonald bunchesofdonald added the development Tech debt, refactors, CI, tests, and other related work. label Nov 7, 2024
@github-actions github-actions bot added docs upstream dependency An upstream issue caused by a bug in one of our dependencies labels Nov 7, 2024
Copy link

codspeed-hq bot commented Nov 7, 2024

CodSpeed Performance Report

Merging #15946 will not alter performance

Comparing CLOUD-563-config (7d612f3) with main (2624409)

Summary

✅ 3 untouched benchmarks

Copy link
Collaborator

@chrisguidry chrisguidry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great tests! I'm thinking we need to move some of the imports conditionally into functions?

requirements-dev.txt Outdated Show resolved Hide resolved
@bunchesofdonald
Copy link
Contributor Author

Great tests! I'm thinking we need to move some of the imports conditionally into functions?

Yeah, I'm working on something like that, having a bootstrap.py that handles knowing if the imports should happen.

if server_type != ServerType.CLOUD:
return None, None, None

assert settings.api.key
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need an API key for this to work, can we raise a friendly error when the API key is missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I have a ticket to come through and make this a more friendly thing in general: https://linear.app/prefect/issue/CLOUD-675/make-telemetry-failures-more-friendly

src/prefect/telemetry/instrumentation.py Outdated Show resolved Hide resolved
Co-authored-by: Alexander Streed <desertaxle@users.noreply.github.com>
@bunchesofdonald bunchesofdonald merged commit d6a4842 into main Nov 8, 2024
37 checks passed
@bunchesofdonald bunchesofdonald deleted the CLOUD-563-config branch November 8, 2024 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development Tech debt, refactors, CI, tests, and other related work. docs upstream dependency An upstream issue caused by a bug in one of our dependencies
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants