Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http: support creating filters with match tree #14430

Merged
merged 37 commits into from
Jan 12, 2021
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
ecf47e2
http: add support for skip filter match action
snowp Dec 4, 2020
910d3f1
support skipping filter execution
snowp Dec 4, 2020
5734a5c
move protos
snowp Dec 7, 2020
e63ff82
Merge branch 'master' of github.com:envoyproxy/envoy into hcm-int
snowp Dec 7, 2020
ce3e503
move protos
snowp Dec 7, 2020
6b407f5
spelling + title
snowp Dec 7, 2020
6fec69f
add to toc
snowp Dec 8, 2020
baff1bd
templating to help clang do the conversions
snowp Dec 8, 2020
c39c870
fix name collision
snowp Dec 8, 2020
571f096
update mocks
snowp Dec 8, 2020
e83b3b2
remove accidental line
snowp Dec 8, 2020
9afefee
wip filter for creating match trees
snowp Dec 8, 2020
3d223ad
wire things up a bit
snowp Dec 11, 2020
d124637
Merge remote-tracking branch 'envoy/master' into matching-filter
snowp Dec 15, 2020
ecb49e7
add test coverage
snowp Dec 15, 2020
13a552d
remove old protos
snowp Dec 15, 2020
a66c51f
cleanup
snowp Dec 15, 2020
b66ef32
more cleanups
snowp Dec 15, 2020
01fced2
format
snowp Dec 15, 2020
142c70b
remove more old protos
snowp Dec 15, 2020
9ce8523
add comments
snowp Dec 16, 2020
be5f50b
test coverage
snowp Dec 16, 2020
711114b
update comments
snowp Dec 17, 2020
d8e5cde
use better helpers
snowp Dec 17, 2020
465f9a4
comments + format
snowp Dec 17, 2020
e88be1d
move protos out of hcm config
snowp Dec 17, 2020
487822f
move to envoy/type/matcher
snowp Dec 18, 2020
6cb618f
update docs
snowp Dec 18, 2020
2cfb10c
format
snowp Dec 18, 2020
0fb52fb
fix test failures
snowp Dec 18, 2020
91e62af
fix last type url
snowp Dec 18, 2020
b2c884f
clang-tidy
snowp Dec 18, 2020
e4987b5
add include
snowp Dec 18, 2020
8868cd2
Merge remote-tracking branch 'envoy/master' into matching-filter
snowp Dec 29, 2020
9abac4b
Merge remote-tracking branch 'envoy/master' into matching-filter
snowp Jan 4, 2021
899a365
introduce templated base class for factory
snowp Jan 11, 2021
193a95c
Merge remote-tracking branch 'envoy/master' into matching-filter
snowp Jan 11, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -838,3 +838,17 @@ message RequestIDExtension {
// Request ID extension specific configuration.
google.protobuf.Any typed_config = 1;
}

// Match input indicates that matching should be done on a specific request header.
// TODO(snowp): Link to unified matching docs.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use the doc comment format here and below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you talking about /** */ comments or something else? iirc we just use // for all proto comments

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean [#comment (see other examples). Can you check CI also?

/wait

message HttpRequestHeaderMatchInput {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this also be needed in the route table? If so, should we put this somewhere more generic, so that it can be used in both places?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume yes if/when we get around to modifying the route table to use some of this stuff. I agree it might be worthwhile to make a new proto for some of these fields so it can be shared.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was imagining that we wouldn't depend on this directly in the route table if we ended up using the matching API for it? Wouldn't we just refer to this via opaque TypedExtensionConfigs?

Any suggestions for where this would fit better? I can dump it into the extensions/common/matcher package I put the ExtensionWithMatcher, but there might be a better location for it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we'd use TypedExtensionConfig to plug this into the matcher proto. But if we're matching against HTTP headers in both cases, then we'd presumably want to use the same proto to plugin to the TypedExtensionConfig in both the filter matcher case and the route table case.

Maybe a better location would be type/matcher/http_header.proto?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this handle multi-valued headers? I think it should concatenate multiple values with ',' delimiters, and we should document that behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed we should be very specific about this. In the current impl (given the previous CVE) we would do that concat that you comment on. I would be explicit about that in the docs.

// The request header to match on.
string header_name = 1;
}

// Match input indicating that matching should be done on a specific response header.
// TODO(snowp): Link to unified matching docs.
message HttpResponseHeaderMatchInput {
// The response header to match on.
string header_name = 1;
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions include/envoy/http/filter.h
Original file line number Diff line number Diff line change
Expand Up @@ -867,6 +867,8 @@ using StreamFilterSharedPtr = std::shared_ptr<StreamFilter>;

class HttpMatchingData {
public:
static absl::string_view name() { return "http"; }

virtual ~HttpMatchingData() = default;

virtual RequestHeaderMapOptConstRef requestHeaders() const PURE;
Expand Down
2 changes: 2 additions & 0 deletions source/common/http/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ envoy_cc_library(
"//source/common/local_reply:local_reply_lib",
"//source/common/matcher:matcher_lib",
"@envoy_api//envoy/extensions/filters/common/matcher/action/v3:pkg_cc_proto",
"@envoy_api//envoy/extensions/filters/network/http_connection_manager/v3:pkg_cc_proto",
],
)

Expand Down Expand Up @@ -242,6 +243,7 @@ envoy_cc_library(
"//source/common/http/http2:codec_lib",
"//source/common/http/http3:quic_codec_factory_lib",
"//source/common/http/http3:well_known_names",
"//source/common/http/match_wrapper:config",
"//source/common/network:utility_lib",
"//source/common/router:config_lib",
"//source/common/router:scoped_rds_lib",
Expand Down
2 changes: 2 additions & 0 deletions source/common/http/filter_manager.cc
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ namespace Envoy {
namespace Http {

namespace {
REGISTER_FACTORY(HttpRequestHeadersDataInputFactory, Matcher::DataInputFactory<HttpMatchingData>);
REGISTER_FACTORY(SkipActionFactory, Matcher::ActionFactory);

template <class T> using FilterList = std::list<std::unique_ptr<T>>;

Expand Down
47 changes: 45 additions & 2 deletions source/common/http/filter_manager.h
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
#pragma once

#include <memory>

#include "envoy/extensions/filters/common/matcher/action/v3/skip_action.pb.h"
#include "envoy/extensions/filters/network/http_connection_manager/v3/http_connection_manager.pb.h"
#include "envoy/http/filter.h"
#include "envoy/http/header_map.h"
#include "envoy/matcher/matcher.h"
Expand Down Expand Up @@ -103,6 +102,23 @@ class HttpRequestHeadersDataInput : public HttpHeadersDataInputBase<RequestHeade
}
};

class HttpRequestHeadersDataInputFactory : public Matcher::DataInputFactory<HttpMatchingData> {
public:
std::string name() const override { return "request-headers"; }
Matcher::DataInputPtr<HttpMatchingData>
createDataInput(const Protobuf::Message& config) override {
const auto& typed_config =
dynamic_cast<const envoy::extensions::filters::network::http_connection_manager::v3::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want to downcastAndValidate here? Same elsewhere.

HttpRequestHeaderMatchInput&>(config);

return std::make_unique<HttpRequestHeadersDataInput>(typed_config.header_name());
};
ProtobufTypes::MessagePtr createEmptyConfigProto() override {
return std::make_unique<envoy::extensions::filters::network::http_connection_manager::v3::
HttpRequestHeaderMatchInput>();
}
};

class HttpResponseHeadersDataInput : public HttpHeadersDataInputBase<ResponseHeaderMap> {
public:
explicit HttpResponseHeadersDataInput(const std::string& name) : HttpHeadersDataInputBase(name) {}
Expand All @@ -113,9 +129,36 @@ class HttpResponseHeadersDataInput : public HttpHeadersDataInputBase<ResponseHea
}
};

class HttpResponseHeadersDataInputFactory : public Matcher::DataInputFactory<HttpMatchingData> {
public:
std::string name() const override { return "response-headers"; }
Matcher::DataInputPtr<HttpMatchingData>
createDataInput(const Protobuf::Message& config) override {
const auto& typed_config =
dynamic_cast<const envoy::extensions::filters::network::http_connection_manager::v3::
HttpResponseHeaderMatchInput&>(config);

return std::make_unique<HttpResponseHeadersDataInput>(typed_config.header_name());
};
ProtobufTypes::MessagePtr createEmptyConfigProto() override {
return std::make_unique<envoy::extensions::filters::network::http_connection_manager::v3::
HttpResponseHeaderMatchInput>();
}
};

class SkipAction : public Matcher::ActionBase<
envoy::extensions::filters::common::matcher::action::v3::SkipFilter> {};

class SkipActionFactory : public Matcher::ActionFactory {
public:
std::string name() const override { return "skip"; }
Matcher::ActionFactoryCb createActionFactoryCb(const Protobuf::Message&) override {
return []() { return std::make_unique<SkipAction>(); };
}
ProtobufTypes::MessagePtr createEmptyConfigProto() override {
return std::make_unique<envoy::extensions::filters::common::matcher::action::v3::SkipFilter>();
}
};
/**
* Base class wrapper for both stream encoder and decoder filters.
*
Expand Down
23 changes: 23 additions & 0 deletions source/common/http/match_wrapper/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
load(
"//bazel:envoy_build_system.bzl",
"envoy_cc_library",
"envoy_package",
)

licenses(["notice"]) # Apache 2

envoy_package()

envoy_cc_library(
name = "config",
srcs = ["config.cc"],
hdrs = ["config.h"],
deps = [
"//include/envoy/registry",
"//include/envoy/server:filter_config_interface",
"//source/common/matcher:matcher_lib",
"//source/extensions/filters/http:well_known_names",
"//source/extensions/filters/http/common:factory_base_lib",
"@envoy_api//envoy/extensions/common/matching/v3:pkg_cc_proto",
],
)
92 changes: 92 additions & 0 deletions source/common/http/match_wrapper/config.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
#include "common/http/match_wrapper/config.h"

#include "envoy/http/filter.h"
#include "envoy/matcher/matcher.h"
#include "envoy/registry/registry.h"

#include "common/matcher/matcher.h"

namespace Envoy {
namespace Common {
namespace Http {
namespace MatchWrapper {

namespace {
struct DelegatingFactoryCallbacks : public Envoy::Http::FilterChainFactoryCallbacks {
DelegatingFactoryCallbacks(Envoy::Http::FilterChainFactoryCallbacks& delegated_callbacks,
Matcher::MatchTreeSharedPtr<Envoy::Http::HttpMatchingData> match_tree)
: delegated_callbacks_(delegated_callbacks), match_tree_(std::move(match_tree)) {}

void addStreamDecoderFilter(Envoy::Http::StreamDecoderFilterSharedPtr filter) override {
delegated_callbacks_.addStreamDecoderFilter(std::move(filter), match_tree_);
}
void addStreamDecoderFilter(
Envoy::Http::StreamDecoderFilterSharedPtr filter,
Matcher::MatchTreeSharedPtr<Envoy::Http::HttpMatchingData> match_tree) override {
delegated_callbacks_.addStreamDecoderFilter(std::move(filter), std::move(match_tree));
}
void addStreamEncoderFilter(Envoy::Http::StreamEncoderFilterSharedPtr filter) override {
delegated_callbacks_.addStreamEncoderFilter(std::move(filter), match_tree_);
}
void addStreamEncoderFilter(
Envoy::Http::StreamEncoderFilterSharedPtr filter,
Matcher::MatchTreeSharedPtr<Envoy::Http::HttpMatchingData> match_tree) override {
delegated_callbacks_.addStreamEncoderFilter(std::move(filter), std::move(match_tree));
}
void addStreamFilter(Envoy::Http::StreamFilterSharedPtr filter) override {
delegated_callbacks_.addStreamFilter(std::move(filter), match_tree_);
}
void
addStreamFilter(Envoy::Http::StreamFilterSharedPtr filter,
Matcher::MatchTreeSharedPtr<Envoy::Http::HttpMatchingData> match_tree) override {
delegated_callbacks_.addStreamFilter(std::move(filter), std::move(match_tree));
}
void addAccessLogHandler(AccessLog::InstanceSharedPtr handler) override {
delegated_callbacks_.addAccessLogHandler(std::move(handler));
}

Envoy::Http::FilterChainFactoryCallbacks& delegated_callbacks_;
Matcher::MatchTreeSharedPtr<Envoy::Http::HttpMatchingData> match_tree_;
};
} // namespace

Envoy::Http::FilterFactoryCb MatchWrapperConfig::createFilterFactoryFromProtoTyped(
const envoy::extensions::common::matching::v3::ExtensionWithMatcher& proto_config,
const std::string& prefix, Server::Configuration::FactoryContext& context) {

if (proto_config.has_extension_config()) {
auto& factory =
Config::Utility::getAndCheckFactory<Server::Configuration::NamedHttpFilterConfigFactory>(
proto_config.extension_config());

auto message = factory.createEmptyConfigProto();
proto_config.extension_config().typed_config();
Config::Utility::translateOpaqueConfig(proto_config.extension_config().typed_config(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can use translateAnyToFactoryConfig here?

ProtobufWkt::Struct(),
context.messageValidationVisitor(), *message);
auto filter_factory = factory.createFilterFactoryFromProto(*message, prefix, context);

auto match_tree =
Matcher::MatchTreeFactory<Envoy::Http::HttpMatchingData>(context.messageValidationVisitor())
.create(proto_config.matcher());

return
[filter_factory, match_tree](Envoy::Http::FilterChainFactoryCallbacks& callbacks) -> void {
DelegatingFactoryCallbacks delegated_callbacks(callbacks, match_tree);

return filter_factory(delegated_callbacks);
};
}

return [](Envoy::Http::FilterChainFactoryCallbacks&) -> void {};
}

/**
* Static registration for the Lua filter. @see RegisterFactory.
*/
REGISTER_FACTORY(MatchWrapperConfig, Server::Configuration::NamedHttpFilterConfigFactory){"blah"};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused. Where is all of this code actually used? Are you registering this as an HTTP filter factory? (I think so). If so can you add more comments here and do something other than "blah"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woops yeah this should have a better name for sure. Yeah I'm making use of the fact that by registering it as a filter all the locations that try to instantiate a filter will end up looking up this factory that unwraps the proto and allocates the matcher. I'll update all of this and add some more documentation


} // namespace MatchWrapper
} // namespace Http
} // namespace Common
} // namespace Envoy
28 changes: 28 additions & 0 deletions source/common/http/match_wrapper/config.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#pragma once

#include "envoy/extensions/common/matching/v3/extension_matcher.pb.validate.h"
#include "envoy/server/filter_config.h"

#include "extensions/filters/http/common/factory_base.h"
#include "extensions/filters/http/well_known_names.h"

namespace Envoy {
namespace Common {
namespace Http {
namespace MatchWrapper {

class MatchWrapperConfig : public Extensions::HttpFilters::Common::FactoryBase<
envoy::extensions::common::matching::v3::ExtensionWithMatcher> {
public:
MatchWrapperConfig() : FactoryBase("match-wrapper") {}

private:
Envoy::Http::FilterFactoryCb createFilterFactoryFromProtoTyped(
const envoy::extensions::common::matching::v3::ExtensionWithMatcher& proto_config,
const std::string&, Server::Configuration::FactoryContext& context) override;
};

} // namespace MatchWrapper
} // namespace Http
} // namespace Common
} // namespace Envoy
3 changes: 3 additions & 0 deletions test/integration/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -882,6 +882,8 @@ envoy_cc_test(
"//test/integration/filters:encoder_decoder_buffer_filter_lib",
"//test/integration/filters:invalid_header_filter_lib",
"//test/integration/filters:process_context_lib",
"//test/integration/filters:set_response_code_filter_config_proto_cc_proto",
"//test/integration/filters:set_response_code_filter_lib",
"//test/integration/filters:stop_iteration_and_continue",
"//test/mocks/http:http_mocks",
"//test/test_common:utility_lib",
Expand Down Expand Up @@ -1083,6 +1085,7 @@ envoy_cc_test(
"//test/integration/filters:set_response_code_filter_config_proto_cc_proto",
"//test/integration/filters:set_response_code_filter_lib",
"//test/test_common:utility_lib",
"@envoy_api//envoy/extensions/common/matching/v3:pkg_cc_proto",
"@envoy_api//envoy/extensions/filters/network/http_connection_manager/v3:pkg_cc_proto",
"@envoy_api//envoy/service/extension/v3:pkg_cc_proto",
],
Expand Down
Loading