Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send worker metadata with first heartbeat #15898

Merged
merged 10 commits into from
Nov 4, 2024
Merged

Conversation

abrookins
Copy link
Collaborator

@abrookins abrookins commented Nov 1, 2024

Send worker metadata with the first worker heartbeat. For now, this metadata includes the Prefect integrations installed in the current process.

NOTE: This PR only sends the data if we're connected to Cloud. Support for open-source Prefect servers will follow.

Closes CLOUD-542.

Checklist

  • This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • If this pull request adds new functionality, it includes unit tests that cover the changes
  • If this pull request removes docs files, it includes redirect settings in mint.json.
  • If this pull request adds functions or classes, it includes helpful docstrings.

Copy link

codspeed-hq bot commented Nov 1, 2024

CodSpeed Performance Report

Merging #15898 will not alter performance

Comparing cloud-542-worker-metadata (329c093) with main (d332172)

Summary

✅ 3 untouched benchmarks

@abrookins abrookins marked this pull request as ready for review November 1, 2024 22:25
@abrookins abrookins changed the title WIP on collecting integration versions for worker metadata Send worker metadata with first heartbeat Nov 1, 2024
Copy link
Member

@desertaxle desertaxle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

I left one suggestion to send the prefect version with the heartbeat in service of #12126, but feel free to punt on that for now and we'll come back to it.

src/prefect/workers/base.py Outdated Show resolved Hide resolved
@abrookins
Copy link
Collaborator Author

Good suggestion! Here was my thinking on excluding it. Currently, the client already sends its version in an HTTP header, so we're capturing it that way to store in metadata server-side in the heartbeat endpoint. Does that work? We'll plan to port that to open-source when we bring the feature over.

@desertaxle
Copy link
Member

Good suggestion! Here was my thinking on excluding it. Currently, the client already sends its version in an HTTP header, so we're capturing it that way to store in metadata server-side in the heartbeat endpoint. Does that work? We'll plan to port that to open-source when we bring the feature over.

Yeah, that makes perfect sense to me!

@abrookins abrookins merged commit 190c994 into main Nov 4, 2024
34 checks passed
@abrookins abrookins deleted the cloud-542-worker-metadata branch November 4, 2024 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants