Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filters: add generic compressor filter #7057

Merged
merged 94 commits into from
Mar 5, 2020
Merged
Show file tree
Hide file tree
Changes from 89 commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
321d72e
gzip: generalize gzip into a general compression filter
May 22, 2019
86d8f82
fix clang-tidy warnings
May 23, 2019
fd920c4
fix spellcheck errors
May 23, 2019
6a1f90f
optimize iterating over allowed compressors
May 29, 2019
207d983
tests: verify removeAcceptEncoding is not used when compressor is not…
May 29, 2019
eeb4c4f
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
May 29, 2019
054be50
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
rojkov May 30, 2019
1a8853b
move common config from gzip.proto to compressor_common.proto
Jun 20, 2019
5124466
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Jun 25, 2019
0405f89
fix unit test after merging master
Jun 24, 2019
ae31978
update CODEOWNERS
Jun 25, 2019
c05bcbd
fix typo
Jun 25, 2019
a239574
documment new field in Gzip message
Jun 25, 2019
6a373cd
fix clang_tidy check
Jun 25, 2019
07b4974
rename compressor_common package to compressor
Jun 25, 2019
d27b53e
actually generate docs for Compressor
Jun 25, 2019
36cd2d3
fix formatting
Jun 25, 2019
a71137c
reset field numbering in compress.proto
Jul 1, 2019
a8c86ef
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Jul 3, 2019
7cbdadd
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Jul 29, 2019
19a2cfa
reorganize imports in compressor.proto
Aug 1, 2019
f7960f9
add deprecated option to deprected fields in gzip.proto
Aug 1, 2019
5942a8e
compressor.proto: drop unused import and option
Aug 1, 2019
12e6281
add missing new line
Aug 1, 2019
bf2710a
gzip.proto: fix formatting
Aug 1, 2019
bb5b24b
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Aug 26, 2019
a6c6c46
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Sep 18, 2019
3164191
fix format checks by adding api_proto_package() to BUILD file
Sep 18, 2019
ff14601
fix API checks
Sep 18, 2019
134187b
Fix failing integration test
Sep 19, 2019
a8522dd
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Sep 24, 2019
decc691
fix proto format check
Sep 24, 2019
ffbd399
fix error reported by proto_format.sh
Sep 24, 2019
b961274
remove limits from compression filter configs
Sep 27, 2019
e7a928f
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Sep 27, 2019
59db2c2
add compressor.proto for v3alpha
Sep 27, 2019
dec3407
add BUILD file to compressor API module
Sep 27, 2019
f376068
fix gzip.proto dependency for v3alpha
Sep 27, 2019
2a9a12a
fix clang_tidy check
Sep 27, 2019
9f389b0
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Oct 7, 2019
ce23b42
fix formatting
Oct 7, 2019
676d676
get rid of static non-pod
Oct 8, 2019
c68e451
reformat comments to use wider column limit
Oct 8, 2019
1e6f7bf
add explanation on client's reqs and server's caps
Oct 8, 2019
5f9d88d
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Oct 8, 2019
e8bab57
fix format checks
Oct 8, 2019
edaecd2
capture defaultContentEncoding() as a temp
Oct 9, 2019
96dc9d4
guard registered compressors with mutex
Oct 8, 2019
dd64763
replace vectors with map to avoid a double loop
Oct 9, 2019
1f3aa75
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Oct 11, 2019
5296095
api: minified v2 -> v3alpha upgrade via type dependency analysis.
Oct 11, 2019
e7c5fc0
run proto_format.sh fix
Oct 11, 2019
98533ba
improve readability by spelling the type explicitly
Oct 15, 2019
b034b87
instantiate compressor as late as possible
Oct 16, 2019
8693478
drop handled TODO comment
Oct 15, 2019
0e8656f
avoid double map lookup
Oct 14, 2019
93ebd44
maintain singleton map instead of its construction for every request
Oct 15, 2019
41f9843
get rid of total sort
Oct 15, 2019
859bb4c
reuse iterator instead of repeated map lookup
Oct 23, 2019
f408dc6
handle Accept-Encoding only once per request
Oct 23, 2019
bed9247
add a comment for the copied map
Oct 23, 2019
da69508
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Oct 23, 2019
09d0ca8
fix proto_format.sh
Oct 23, 2019
afa48c6
address minor issues
Nov 6, 2019
54b47e9
get rid of global mutex
Nov 12, 2019
8ceb6d8
fix proto_format check
Nov 12, 2019
dbc2e91
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Dec 23, 2019
adb9ce6
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Jan 16, 2020
9ea11ca
fix function signature to accommodate v2 to v3 API transition
Jan 16, 2020
6b98708
run proto_format.sh fix
Jan 16, 2020
d7bd763
gzip.proto: remove deprecation warning from v2 as gzip moved to v3 al…
Jan 16, 2020
9fe7a41
move compressor.proto to v3
Jan 17, 2020
e99b4c1
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Jan 17, 2020
8b1ff4e
gzip: mark deprecated fields as deprecated again
Jan 20, 2020
0b38c92
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Feb 10, 2020
4cb988e
compress: adapt to API changes in FilterState
Feb 10, 2020
ad73448
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Feb 12, 2020
2bbbeb2
Merge remote-tracking branch 'upstream/master' into brotli-multi-filters
Feb 19, 2020
ee50131
gzip: clarify the doc
Feb 19, 2020
d414cb8
extend the list of text content types with known synonyms
Feb 19, 2020
3d50681
delay encoding decision until content-type is known
Feb 19, 2020
3d6bbb5
address review comments
Feb 19, 2020
0df08ac
run check_format.py fix
Feb 19, 2020
3d8d556
run proto_format.py fix
Feb 19, 2020
68e3a60
use correct type for HeaderMap
Feb 19, 2020
1264376
remove unused headers
Feb 26, 2020
4461a7a
drop refactoring of gzip filter
Feb 26, 2020
6b3a23e
Merge remote-tracking branch 'upstream/master' into brotli-multi-filt…
Feb 27, 2020
1f89041
describe chooseEncoding() in details
Mar 2, 2020
4a70888
use excplicit kill switch defined in the config
Mar 4, 2020
50e7300
fix check_spelling_pedantic.py errors
Mar 4, 2020
31324f2
Merge remote-tracking branch 'upstream/master' into brotli-multi-filt…
Mar 4, 2020
70ef5fc
fix check_format.py errors
Mar 4, 2020
d2c385d
use deflate instead of brotli in examples
Mar 5, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ extensions/filters/common/original_src @snowp @klarose
/*/extensions/filters/network/dubbo_proxy @zyfjeff @lizan
# thrift_proxy extension
/*/extensions/filters/network/thrift_proxy @zuercher @brian-pane
# compressor used by http compression filters
/*/extensions/filters/http/common/compressor @gsagula @rojkov @dio
# jwt_authn http filter extension
/*/extensions/filters/http/jwt_authn @qiwzhang @lizan
# grpc_http1_reverse_bridge http filter extension
Expand Down
2 changes: 2 additions & 0 deletions api/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ proto_library(
"//envoy/config/filter/http/aws_request_signing/v2alpha:pkg",
"//envoy/config/filter/http/buffer/v2:pkg",
"//envoy/config/filter/http/cache/v2alpha:pkg",
"//envoy/config/filter/http/compressor/v2:pkg",
"//envoy/config/filter/http/cors/v2:pkg",
"//envoy/config/filter/http/csrf/v2:pkg",
"//envoy/config/filter/http/dynamic_forward_proxy/v2alpha:pkg",
Expand Down Expand Up @@ -161,6 +162,7 @@ proto_library(
"//envoy/extensions/filters/http/aws_request_signing/v3:pkg",
"//envoy/extensions/filters/http/buffer/v3:pkg",
"//envoy/extensions/filters/http/cache/v3alpha:pkg",
"//envoy/extensions/filters/http/compressor/v3:pkg",
"//envoy/extensions/filters/http/cors/v3:pkg",
"//envoy/extensions/filters/http/csrf/v3:pkg",
"//envoy/extensions/filters/http/dynamic_forward_proxy/v3:pkg",
Expand Down
1 change: 1 addition & 0 deletions api/docs/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ proto_library(
"//envoy/config/filter/http/aws_request_signing/v2alpha:pkg",
"//envoy/config/filter/http/buffer/v2:pkg",
"//envoy/config/filter/http/cache/v2alpha:pkg",
"//envoy/config/filter/http/compressor/v2:pkg",
"//envoy/config/filter/http/cors/v2:pkg",
"//envoy/config/filter/http/csrf/v2:pkg",
"//envoy/config/filter/http/dynamic_forward_proxy/v2alpha:pkg",
Expand Down
9 changes: 9 additions & 0 deletions api/envoy/config/filter/http/compressor/v2/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# DO NOT EDIT. This file is generated by tools/proto_sync.py.

load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")

licenses(["notice"]) # Apache 2

api_proto_package(
deps = ["@com_github_cncf_udpa//udpa/annotations:pkg"],
)
39 changes: 39 additions & 0 deletions api/envoy/config/filter/http/compressor/v2/compressor.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
syntax = "proto3";

package envoy.config.filter.http.compressor.v2;

import "google/protobuf/wrappers.proto";

import "udpa/annotations/migrate.proto";

option java_package = "io.envoyproxy.envoy.config.filter.http.compressor.v2";
option java_outer_classname = "CompressorProto";
option java_multiple_files = true;
option (udpa.annotations.file_migrate).move_to_package =
"envoy.extensions.filters.http.compressor.v3";

// [#protodoc-title: Compressor]

message Compressor {
// Minimum response length, in bytes, which will trigger compression. The default value is 30.
mattklein123 marked this conversation as resolved.
Show resolved Hide resolved
google.protobuf.UInt32Value content_length = 1;

// Set of strings that allows specifying which mime-types yield compression; e.g.,
// application/json, text/html, etc. When this field is not defined, compression will be applied
// to the following mime-types: "application/javascript", "application/json",
mattklein123 marked this conversation as resolved.
Show resolved Hide resolved
// "application/xhtml+xml", "image/svg+xml", "text/css", "text/html", "text/plain", "text/xml"
// and their synonyms.
repeated string content_type = 2;

// If true, disables compression when the response contains an etag header. When it is false, the
// filter will preserve weak etags and remove the ones that require strong validation.
bool disable_on_etag_header = 3;

// If true, removes accept-encoding from the request headers before dispatching it to the upstream
// so that responses do not get compressed before reaching the filter.
// .. attention:
//
// To avoid interfering with other compression filters in the same chain use this option in
// the filter closest to the upstream.
bool remove_accept_encoding_header = 4;
}
12 changes: 12 additions & 0 deletions api/envoy/extensions/filters/http/compressor/v3/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# DO NOT EDIT. This file is generated by tools/proto_sync.py.

load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")

licenses(["notice"]) # Apache 2

api_proto_package(
deps = [
"//envoy/config/filter/http/compressor/v2:pkg",
"@com_github_cncf_udpa//udpa/annotations:pkg",
],
)
40 changes: 40 additions & 0 deletions api/envoy/extensions/filters/http/compressor/v3/compressor.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
syntax = "proto3";

package envoy.extensions.filters.http.compressor.v3;

import "google/protobuf/wrappers.proto";

import "udpa/annotations/versioning.proto";

option java_package = "io.envoyproxy.envoy.extensions.filters.http.compressor.v3";
option java_outer_classname = "CompressorProto";
option java_multiple_files = true;

// [#protodoc-title: Compressor]

message Compressor {
option (udpa.annotations.versioning).previous_message_type =
"envoy.config.filter.http.compressor.v2.Compressor";

// Minimum response length, in bytes, which will trigger compression. The default value is 30.
google.protobuf.UInt32Value content_length = 1;

// Set of strings that allows specifying which mime-types yield compression; e.g.,
// application/json, text/html, etc. When this field is not defined, compression will be applied
// to the following mime-types: "application/javascript", "application/json",
// "application/xhtml+xml", "image/svg+xml", "text/css", "text/html", "text/plain", "text/xml"
// and their synonyms.
repeated string content_type = 2;

// If true, disables compression when the response contains an etag header. When it is false, the
// filter will preserve weak etags and remove the ones that require strong validation.
bool disable_on_etag_header = 3;

// If true, removes accept-encoding from the request headers before dispatching it to the upstream
// so that responses do not get compressed before reaching the filter.
// .. attention:
//
// To avoid interfering with other compression filters in the same chain use this option in
// the filter closest to the upstream.
bool remove_accept_encoding_header = 4;
}
2 changes: 2 additions & 0 deletions generated_api_shadow/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ proto_library(
"//envoy/config/filter/fault/v2:pkg",
"//envoy/config/filter/http/adaptive_concurrency/v2alpha:pkg",
"//envoy/config/filter/http/buffer/v2:pkg",
"//envoy/config/filter/http/compressor/v2:pkg",
"//envoy/config/filter/http/cors/v2:pkg",
"//envoy/config/filter/http/csrf/v2:pkg",
"//envoy/config/filter/http/dynamic_forward_proxy/v2alpha:pkg",
Expand Down Expand Up @@ -129,6 +130,7 @@ proto_library(
"//envoy/extensions/filters/common/fault/v3:pkg",
"//envoy/extensions/filters/http/adaptive_concurrency/v3:pkg",
"//envoy/extensions/filters/http/buffer/v3:pkg",
"//envoy/extensions/filters/http/compressor/v3:pkg",
"//envoy/extensions/filters/http/cors/v3:pkg",
"//envoy/extensions/filters/http/csrf/v3:pkg",
"//envoy/extensions/filters/http/dynamic_forward_proxy/v3:pkg",
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 25 additions & 0 deletions source/extensions/filters/http/common/compressor/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
licenses(["notice"]) # Apache 2

load(
"//bazel:envoy_build_system.bzl",
"envoy_cc_library",
"envoy_package",
)

envoy_package()

envoy_cc_library(
name = "compressor_lib",
srcs = ["compressor.cc"],
hdrs = ["compressor.h"],
deps = [
"//include/envoy/compressor:compressor_interface",
"//include/envoy/stats:stats_macros",
"//include/envoy/stream_info:filter_state_interface",
"//source/common/buffer:buffer_lib",
"//source/common/http:header_map_lib",
"//source/common/protobuf",
"//source/extensions/filters/http/common:pass_through_filter_lib",
"@envoy_api//envoy/extensions/filters/http/compressor/v3:pkg_cc_proto",
],
)
Loading