Add Request Decompression middleware

[david-acker]

Adds middleware for decompressing HTTP requests.

Problem

In the past, the developer of the server would need to manually handle requests with compressed content by checking if a request was compressed and then decompressing the request body stream as appropriate.

Solution

This middleware uses the Content-Encoding header automatically identify and decompress requests with compressed content, so that the developer of the server does not need to handle this themselves.

The request decompression middleware is added using the UseRequestDecompression extension method for IApplicationBuilder and the AddRequestDecompression extension method for IServiceCollection.

Supported Content Encodings

Support for Brotli (br), Deflate (deflate), and GZip (gzip) are included out of the box.

Support for other content encodings can be added by registering a custom decompression provider class, which implements the IDecompressionProvider interface, along with a Content-Encoding header value in RequestDecompressionOptions.

Request Size Limits

The request body size limit enforced by the server (MaxRequestBodySize) and/or the endpoint (RequestSizeLimitAttribute) will also be enforced on the decompressed request body.

If the amount of data read from the decompressed request body exceeds the size limit, an InvalidOperationException is thrown to prevent any more data from being read. This protects against the threat of uncontrolled resource consumption from zip bombs

Edge Cases

• If a request uses an unsupported Content-Encoding or has multiple encodings, an appropriate message will be logged, and the request will be passed on without being decompressed.
• If the compressed content is invalid, an exception will be thrown when attempting to read the (decompressed) request body stream.

Fixes #40080

[sebastienros]

I wish my next middleware PR is as polished as this one.

[pranavkm]

Random thought: what if we designed the feature like so: https://gist.github.com/pranavkm/8932a862a701061f43b6e64cc0b18931. It removes some of the newly introduced types at the expense of slightly more code in each of the compression providers.

@sebastienros / @Tratcher?

[david-acker]

Trimming needs to be enabled on this new project. I suggest creating an issue to follow up and do it later. See #41704 as an example.

I've created a follow up issue (#41721) for this.

[halter73]

Thanks @david-acker! I should have paid more attention to the actual code on my last review.

[david-acker]

@halter73 No worries! Thanks for the help with the IRequestSizeLimitMetadata-related changes!

[davidfowl]

Can we make sure we have basic benchmarks setup so we can measure the overhead here?

[david-acker]

I've added some benchmarks which cover both compressed and uncompressed requests as well as the scenarios where the IRequestSizeLimitMetadata is/isn't present.

[Tratcher]

This whole thing looks really clean, nice job.

[adityamandaleeka]

Congrats @david-acker! Happy to see this merged.

[ghost]

@david-acker, this change will be considered for inclusion in the blog post for the release it'll ship in. Nice work!

Please ensure that the original comment in this thread contains a clear explanation of what the change does, why it's important (what problem does it solve?), and, if relevant, include things like code samples and/or performance numbers.

This content may not be exactly what goes into the blog post, but it will help the team putting together the announcement.

Thanks!