Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(spans): Do not trim essential fields #3670

Merged
merged 12 commits into from
Jun 4, 2024
Merged

Conversation

jjbayer
Copy link
Member

@jjbayer jjbayer commented May 31, 2024

Since #3645, we're seeing incomplete or missing trace IDs on spans (see linked sentry issues). What happens is that after visiting a long span.description, there is no budget left for essential fields and the trimming processor drops or trims them.

  • Introduce trim = "false" field attribute to disallow trimming of specific fields.
  • Apply the new field to span.trace_id and other essential fields to prevent them from being partially scrubbed or dropped entirely.

Current limitations:

  • Not trimming these fields leads to over-accepting bytes. This should not matter as long as the fields that have trim = "false" are limited in size (which they are currently not, see ref(protocol): Represent SpanId and TraceId as bytes #3535 as follow-up).
    • The more correct solution would be to subtract the length of trim = "false" fields before trimming any other fields.
    • Alternatively, we could decide to never over-accept, and drop items after visiting them if the budget is negative. I implemented this in 1136f76 but it resulted in more failing tests, because the current trimming processor already over-accepts a little bit (key lengths), and I did not want to touch the default behavior. cc @Dav1dde.

Fixes RELAY-2NY7, RELAY-2NY8, SNUBA-552.

@jjbayer jjbayer self-assigned this May 31, 2024
@jjbayer jjbayer changed the title fix(spans): Do not trim trace ID fix(spans): Do not trim essential fields Jun 4, 2024
@jjbayer jjbayer marked this pull request as ready for review June 4, 2024 08:16
@jjbayer jjbayer requested a review from a team as a code owner June 4, 2024 08:16
Copy link
Member

@Dav1dde Dav1dde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a

relay-event-derive/src/lib.rs Show resolved Hide resolved
relay-event-derive/src/lib.rs Outdated Show resolved Hide resolved
@jjbayer jjbayer merged commit 909fea6 into master Jun 4, 2024
22 checks passed
@jjbayer jjbayer deleted the fix/spans-trim-trace-id branch June 4, 2024 10:48
jjbayer added a commit that referenced this pull request Jun 5, 2024
Follow-up to #3670: Also stop
trimming span timestamps.
@jjbayer jjbayer mentioned this pull request Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants