feat: Adds Gzip Processor and Content Inspector #2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds two new features:
Motivation and Context
Substation assumes that all data coming into and going out of a pipeline is JSON, but there are use cases for converting bodies of data to and from different formats. This PR adds support for Gzip and inspection of any MIME type, but the design should be reused in future "data conversion" processors. These could include:
For processor options, the simplest way to describe conversion is with an option called
direction
that accepts one of two strings: "to" and "from". This should be applicable to any type of data conversion (e.g., "from base64", "to zlib", "from CSV", "to XML") and I think it's better than having bespoke options ("decode base64", "compress zlib", etc.), but I'm open to alternative suggestions.How Has This Been Tested?
Added a bunch of unit tests:
Types of changes
Checklist: