Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add protobuf support for onlyspans ingestion #3044

Merged
merged 12 commits into from
Feb 9, 2024

Conversation

sl0thentr0py
Copy link
Member

@sl0thentr0py sl0thentr0py commented Feb 1, 2024

  • use official prost and serde compatible structs from opentelemetry-proto
  • use axum-extra with the protobuf feature for the protobuf extractor
  • removed our old custom structs and replaced the transformation logic

TODO

  • trace_id/span_id are Vec<u8>, fix that conversion
  • the date conversion helper from strings is not added to their serde derives, so I've ignored this test for now will need to do upstream later

@sl0thentr0py sl0thentr0py force-pushed the neel/onlyspans-protobuf branch from a7dd2fd to d71949c Compare February 1, 2024 17:13
@sl0thentr0py sl0thentr0py self-assigned this Feb 5, 2024
@sl0thentr0py sl0thentr0py force-pushed the neel/onlyspans-protobuf branch 4 times, most recently from 3f10db7 to f8a5997 Compare February 6, 2024 15:46
@sl0thentr0py sl0thentr0py requested a review from phacops February 6, 2024 15:48
@sl0thentr0py sl0thentr0py marked this pull request as ready for review February 6, 2024 15:48
@sl0thentr0py sl0thentr0py requested a review from a team as a code owner February 6, 2024 15:48
@phacops phacops requested a review from jjbayer February 6, 2024 23:53
Copy link
Contributor

@iker-barriocanal iker-barriocanal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me, leaving the final review to @jjbayer as I don't have enough context to provide proper feedback.

@sl0thentr0py sl0thentr0py force-pushed the neel/onlyspans-protobuf branch 2 times, most recently from a525fa1 to 1f9b731 Compare February 7, 2024 14:00
Copy link
Member

@Dav1dde Dav1dde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely have to make sure to not have an unpinned git dependency. Also would like to get a review from @jjbayer, I am missing context and was just looking for Rust stuff.

Copy link
Member

@jjbayer jjbayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sl0thentr0py thanks for doing this! Nice to see some LoCs removed. Please add a test for binary protobuf ingestion to test_store.py. See

def test_span_ingestion(

@sl0thentr0py sl0thentr0py force-pushed the neel/onlyspans-protobuf branch from 1f9b731 to 2c9d457 Compare February 8, 2024 15:21
Copy link
Member

@Dav1dde Dav1dde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks looks good to me! Left a small improvement to the handle function.

It would also be nice to have at least one integration test which makes sure we're parsing the data correctly.

the date conversion helper from strings is not added to their serde derives, so I've ignored this test for now

To you still want to address this todo from your PR description?

@sl0thentr0py sl0thentr0py force-pushed the neel/onlyspans-protobuf branch from b0c0351 to 423f82a Compare February 9, 2024 14:36
@sl0thentr0py
Copy link
Member Author

added integration test
the date stuff will need to be done upstream later, I think it's fine to remove that functionality for now since we're relying on official otel structs anyway and if it's in the spec they will eventually add it.

@sl0thentr0py sl0thentr0py force-pushed the neel/onlyspans-protobuf branch from 423f82a to 117a2a0 Compare February 9, 2024 14:51
Copy link
Member

@Dav1dde Dav1dde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Co-authored-by: Joris Bayer <joris.bayer@sentry.io>
@sl0thentr0py sl0thentr0py enabled auto-merge (squash) February 9, 2024 15:36
@sl0thentr0py sl0thentr0py disabled auto-merge February 9, 2024 15:36
@sl0thentr0py sl0thentr0py enabled auto-merge (squash) February 9, 2024 15:36
@sl0thentr0py sl0thentr0py merged commit 53bd97c into master Feb 9, 2024
20 checks passed
@sl0thentr0py sl0thentr0py deleted the neel/onlyspans-protobuf branch February 9, 2024 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants