Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support incremental validation of fragmented messages sent by client #671

Closed
8 tasks done
jfallows opened this issue Jan 1, 2024 · 0 comments
Closed
8 tasks done
Assignees
Labels
story Feature description from user's perspective

Comments

@jfallows
Copy link
Contributor

jfallows commented Jan 1, 2024

Describe the desired outcome from the user's perspective
Large messages are often fragmented into smaller partial messages, but should still be validated according to the same rules as if they are observed as a single unified value.

Acceptance criteria

  • Fragmented HTTP request messages can be validated or rejected; string, integer, json
  • Fragmented MQTT publish messages can be validated or rejected; string, integer, json
  • Fragmented Kafka produce messages can be validated or rejected; string, integer, json
  • Fragmented messages should not require buffering into a complete message before validation

Additional context
Conversion from json to avro (for example) is much more challenging to implement incrementally without buffering the complete message, so we should separate the concepts of validator and converter, supporting converters only at the kafka cache where we already have a view of the complete message by design.

Tasks

  • Separate the concepts of Validator and Converter in Java APIs
  • avro converter should only be configurable in kafka cache by schema (not other protocols)
  • protobuf converter should only be configurable in kafka cache by schema (not other protocols)
  • all validators should only be configurable on HTTP and MQTT by schema
  • both validators and converters should be configurable on Kafka by schema
  • implement string fragmented validator
  • implement integer fragmented validator
  • implement json fragmented validator
@jfallows jfallows added the story Feature description from user's perspective label Jan 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
story Feature description from user's perspective
Projects
Status: Done
Development

No branches or pull requests

2 participants