Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka Headers #3014

Merged
merged 6 commits into from
Feb 19, 2024
Merged

Kafka Headers #3014

merged 6 commits into from
Feb 19, 2024

Conversation

iancooper
Copy link
Member

Our implementation of a header builder for Kafka lacked tests and had bugs around GUIDs. Adding tests revealed naive thinking about translating headers to and from Kafka.

  • A Kafka header a string key and a byte[] value
  • If it is a Brighter header, we can convert a type to a byte[] and back.
  • If it a user defined header (the bag) we can at best convert the type to a string and byte[] and then back to a string
    • We would need to know the serialized type to convert back into the same type
    • Adding type information has implications for security if implemented (see retirement of BinaryFormatter)
    • We can in this case expect the consumer to convert back from a string to a type they know

We used unix timestamps for our own Header timestamp. There is no need to do this, and it simplifies not tol

@holytshirt holytshirt changed the title Kakfa Headers Kafka Headers Feb 13, 2024
@iancooper iancooper merged commit afa2d92 into master Feb 19, 2024
18 of 19 checks passed
@iancooper iancooper deleted the guid_in_header branch March 27, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant