-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fault injection filter #219
Conversation
Runtime support is now available to override the configuration defaults: Header matches specified in the config block cannot be overridden via runtime, at this time. |
@rshriram sorry this is going to have to be rebased on current master to use the new header map API. It should be a pretty small change. I will comment on the rest of it. |
return nullptr; | ||
} | ||
|
||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: // comments inside functions, just one line // TODO: ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to clarify.. are you saying use only the // comments inside functions and not the multiline comment syntax?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes (putting multi line comments inside .cc files makes it difficult to bulk comment out code, in general always prefer // comments in .cc files)
#include "common/http/headers.h" | ||
|
||
namespace Http { | ||
FaultFilter::FaultFilter(FaultFilterConfigPtr config) : config_(config) {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: newline before this line
callbacks_ = &callbacks; | ||
callbacks_->addResetStreamCallback([this]() -> void { onResetStream(); }); | ||
} | ||
} // Http |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: newline before this line
StreamDecoderFilterCallbacks* callbacks_{}; | ||
Event::TimerPtr delay_timer_; | ||
}; | ||
} // Http |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: newline before this line
} | ||
|
||
// header match semantics in fault filter works is same as the one in route block | ||
bool FaultFilter::matches(const Http::HeaderMap& headers) const { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
instead of copying this code from the route table, can we move the code into a static helper, perhaps in ConfigUtility class, and then call from both places, just passing in the headers and the headers list.
*/ | ||
struct FaultFilterConfig { | ||
FaultFilterConfig(uint64_t abort_enabled, uint64_t abort_code, uint64_t delay_enabled, | ||
uint64_t delay_duration, std::vector<FaultFilterHeaders> fault_filter_headers, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const std::vector& fault_filter_headers
At a high level code looks good. There are a few things we might want to do later like we have a specific access log error code for fault injection (see UC, UF, etc. in access log code) but we can do that in a follow up. Please work on adding tests (check with coverage build to make sure you have 100% coverage). You can follow along with other filter tests to see how it's done. |
+cc thurston Nomenclature nits... probabilities should range 0.0 - 1.0, rename to percentage? For abort codes we need to be cognizant of protocol specific abort
On Tue, Nov 15, 2016 at 8:33 AM, Matt Klein notifications@github.com
|
At Netflix we propagate a failure injection context that travels along in a header and identifies when to fail (for example, fail all calls from any service when calling service A), as well as a configurable delay. I see you have one form of the filter that will be applied when a header is present with a specific value, would it be possible to also read the delay in milliseconds and failure percentages out of a header? It'd be best if we could even do some logic on headers as they're filtered, but in the absence of that we could do the logic in the host language of the gRPC client and just add the headers. |
re: naming and percents Typically for time units we have been doing "_ms" postfix for ms units and "_s" postfix for second units. I think doing "_percent" postfix is fine for percent units. Internally within Envoy we don't typically use floating point. Everything ultimately gets converted to an integer typically in the range 0-10000 before computing chance tests. I don't have a strong preference as to whether from a config perspective we support floating point. Whatever people prefer is fine with me. If we start using floating point in these cases we will need to slightly modify runtime to also pre-fill floating point values (easy). re: abort types, per @louiscryan it makes sense to at least make the config for this future proof even if we don't do a lot with it initially. E.g.,
re: header configuration, yes, ultimately we should definitely allow configuration via trusted headers. We already do this extensively within Envoy for timeouts, retries, etc. and developers find this very easy to work with. See https://lyft.github.io/envoy/docs/configuration/http_filters/router_filter.html#http-headers. Would love to see us allow header config here also, but I would recommend we do that in a follow up. |
@jhspaybar We do something similar internally at IBM. The current form of this PR is coarse grained. It would affect all traffic exiting an instance or entering an instance. But its a first cut :). My plan was to push the fault blocks into the routing section so as to allow the fault injection to be performed on a per source-destination basis. @louiscryan @mattklein123 will look into the error codes. |
On the delay specification do we care about using a distribution (uniform, On Wed, Nov 16, 2016 at 12:40 PM, Shriram Rajagopalan <
|
Yeah maybe we do:
for now to make that future proof then we can add other things later also. |
Why should it be restricted to just delays? |
Not sure exponential would make sense for faults which are binary decisions On Wed, Nov 16, 2016 at 3:59 PM, Shriram Rajagopalan <
|
8947845
to
5e79344
Compare
@louiscryan I got confused by a recent comment on delay spec: "delay_specification": {
"type": "fixed",
"fixed_duration_ms" : 5000
} and thought that instead of the the Just to consolidate the feedback, here is the final config format. Let me know if this looks okay ( @mattklein123 @louiscryan @jhspaybar ) "config" : {
"headers" : {
"name" : "value"
}
"abort_percent" : 100,
"abort_specification" : {
"tcp_reset" : "true",
"grpc_code" : <status_code_string>,
"http2_code": <status_code_string>,
"http1_code" : <status_code_int>
},
"delay_percent" : 100,
"delay_specification" : {
"distribution" : "fixed|exponential|others_in_future"
"fixed_delay_ms" : 5000,
"other_fields_for_exponential_distribution_in_future" : 0
} If tcp_reset is set to true, then the connection will be abruptly terminated (irrespective of the underlying protocol). [a future addition]. The config parameters can be overridden by runtime parameters: [ where can I find the integer status codes for HTTP2 and gRPC? It would be good to avoid string matches in the fault filter] In future iterations, we could look into adding support for obtaining the fault configuration from the request headers themselves. However, we need to identify the priority of the request header configuration over that of the runtime parameters. For TCP (in future), we could have the abort action be a connection reset, and the delay action be some form of bandwidth throttling. |
Shriram, Some notes inline below On Fri, Nov 18, 2016 at 8:41 AM, Shriram Rajagopalan <
https://tools.ietf.org/html/rfc7540#section-7 Maybe call it "http2_error" (terminology used in spec)
"delay_specification" : {
For TCP I'd be interested in the effects of delaying the termination of the
Agreed
|
@louiscryan with respect to strings vs int, having strings in config is definitely the way to go. I was pointing to Envoy's ability to load these parameters dynamically at runtime by reading the keys from filesystem. [e.g., A sample code snippet that illustrates the point (ignore the syntax errors). const std::string grpc_status_str = runtime.snapshot().getString("fault.http.abort.grpc_status");
uint64_t grpc_status_code = someHashMap[grpc_status_str]; This can be optimized by pre-processing these strings and converting them into their integer equivalents every time a new snapshot is read from the filesystem. The conversions would be specific to every filter. This might require lot more changes in Envoy. Am I missing something @mattklein123 ? |
re: strings vs. ints, sure strings are better. Feel free to use them. @rshriram let's try to keep this initial change as scoped as possible (I prefer to do things incrementally), so if you need to use a map in each request for right now that's fine. We can separately add a runtime snapshot update callback that can be subscribed to so that this kind of behavior can be made more efficient in the future. (We have this internally in our Go port of the runtime library). |
"config" : {
"headers" : {
"name" : "value"
}
"abort" : {
"abort_percent" : 100,
"tcp_reset" : <future>,
"grpc_status" : <status_code_string>,
"http2_error": <status_code_string>,
"http_status" : <status_code_int>
},
"delay" : {
"type" : "fixed|exponential|others_in_future",
"fixed_delay_percent" : 100,
"fixed_duration_ms" : 5000,
"other_fields_for_exponential_distribution_in_future" : 0
} |
LGTM On Fri, Nov 18, 2016 at 11:20 AM, Shriram Rajagopalan <
|
5e79344
to
415321a
Compare
@mattklein123 I have addressed your comments in the recent commits (last 4) including the new config format, test cases and documentation. Note: the current version implements a fixed delay function, and supports aborts with HTTP Status only. As we discussed earlier, support for grpc_status and http2_error could be added in future iterations. |
const std::string& stat_prefix, Stats::Store& stats) | ||
: runtime_(runtime), stats_{ALL_FAULT_FILTER_STATS(POOL_COUNTER_PREFIX(stats, stat_prefix))} { | ||
|
||
abort_percent_ = 0UL; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Just initialize all 4 of this in the header file by default initialization. E.g.,
uint64_t abort_percent_{};
|
||
if (config_->runtime().snapshot().featureEnabled("fault.http.abort.abort_percent", | ||
config_->abortPercent())) { | ||
// todo check http status codes obtained from runtime |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: TODO:
// Delays can be followed by aborts | ||
if (config_->runtime().snapshot().featureEnabled("fault.http.abort.abort_percent", | ||
config_->abortPercent())) { | ||
Http::HeaderMapPtr response_headers{new HeaderMapImpl{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have a helper function that sends the response that is called from both here as well as decodeHeaders() so we avoid the code duplication.
@rshriram a few small comments. Overall very well done. Thanks. Can you please sign CLA (see contribution readme) |
@mattklein123 addressed your comments and signed the CLAs. |
{Headers::get().Status, std::to_string(config_->runtime().snapshot().getInteger( | ||
"fault.http.abort.http_status", config_->abortCode()))}}}; | ||
callbacks_->encodeHeaders(std::move(response_headers), true); | ||
abortWithHTTPStatus(); | ||
config_->stats().aborts_injected_.inc(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should go in abortWithHTTPStatus()
Http::HeaderMapPtr response_headers{new HeaderMapImpl{ | ||
{Headers::get().Status, std::to_string(config_->runtime().snapshot().getInteger( | ||
"fault.http.abort.http_status", config_->abortCode()))}}}; | ||
abortWithHTTPStatus(); | ||
config_->stats().aborts_injected_.inc(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should go in abortWithHTTPStatus()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason I left it out of that function was because in future, we might add abortWithGRPCStatus
and abortWithHTTP2Error
. Rather than duplicating the stats increment counter in all 3 functions, I thought its better to keep it here. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would just move it for now. We can deal with the future when it happens. :)
done |
* Send delta metrics for intermediate reports. * Move last_request_bytes/last_response_bytes to RequestContext. * Handle final report. * Address comment.
* Created check security rules file and a few dummy/helper functions. (envoyproxy#40) * Created check security rules file and a few dummy/helper functions. And added it to check work flow. * Fix format. * Firebase: Merge from master. (envoyproxy#53) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (envoyproxy#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (envoyproxy#41) * Update prototype to use iptables (envoyproxy#42) * Rebase to fixed Envoy (envoyproxy#43) * Handle HEAD request. (envoyproxy#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (envoyproxy#48) * Try again (envoyproxy#49) * Enable ESP to invoke Firebase Security rules. (envoyproxy#54) * Enable ESP to invoke Firebase Security rules. * Address code review comments. * Remove some debug logs * Add proto file to capture TestRulesetRequest. * clang-format files * Resolve a merge issue with previous commit * Allow security rules to disabled via serverconfig * format file * Addressed Wayne's review comments. * Add firebase server to Server Config. * Address Lizan's review comments * Address review comments. * Disable check rules service by default. * Address more review comments. * Fix a check. * Delete unwanted constant. * Address Wayne's comments and add a simple config test. * Address a review comment. * Add negative test case for config * Address code review * Remove unwanted const std::string * Merge from master into firebase (envoyproxy#65) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (envoyproxy#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (envoyproxy#41) * Update prototype to use iptables (envoyproxy#42) * Rebase to fixed Envoy (envoyproxy#43) * Handle HEAD request. (envoyproxy#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (envoyproxy#48) * Try again (envoyproxy#49) * Integrate with mixer client. (envoyproxy#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (envoyproxy#56) * Add uuid-dev dependency in README.md (envoyproxy#45) * Extract originIp and OriginHost. (envoyproxy#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (envoyproxy#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (envoyproxy#59) * Use envoy new access_log handler for sending Report. (envoyproxy#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with envoyproxy#396. (envoyproxy#61) * Fix tclap dependency fetching error (envoyproxy#62) * Update the auth checke to use service.experimental.authorization.providerwq! * Update the auth check to use service.experimental.authorization.provider * Update the auth check to use service.experimental.authorization.provider (envoyproxy#67) * Update the auth check to use service.experimental.authorization.provider * Address comments and revert accidental change. * Remove unnecessary added accidentally. * Another patch * fix the logic * fix lint * Fix broken test and add unit tests * Fix comments * Fix style check * revert style for raw string * fix small lint * fix small lint * fix small lint * Unit tests for check security rules. (envoyproxy#75) * Unit tests for check security rules. * format * Address review comments. * Fix typos * Merge from master to firebase (envoyproxy#143) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (envoyproxy#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (envoyproxy#41) * Update prototype to use iptables (envoyproxy#42) * Rebase to fixed Envoy (envoyproxy#43) * Handle HEAD request. (envoyproxy#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (envoyproxy#48) * Try again (envoyproxy#49) * Integrate with mixer client. (envoyproxy#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (envoyproxy#56) * Add uuid-dev dependency in README.md (envoyproxy#45) * Extract originIp and OriginHost. (envoyproxy#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (envoyproxy#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (envoyproxy#59) * Use envoy new access_log handler for sending Report. (envoyproxy#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with envoyproxy#396. (envoyproxy#61) * Fix tclap dependency fetching error (envoyproxy#62) * Integrate mixer client directly with envoy. (envoyproxy#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (envoyproxy#68) * Push tar.gz to GCS (envoyproxy#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (envoyproxy#72) * Update mixer client SHA. (envoyproxy#74) * Update readme. (envoyproxy#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (envoyproxy#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (envoyproxy#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (envoyproxy#78) * Add script to build docker image. (envoyproxy#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (envoyproxy#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (envoyproxy#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (envoyproxy#82) * Fix src/envoy/mixer/README.md (envoyproxy#85) * Get attributes from envoy config. (envoyproxy#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (envoyproxy#94) * Disable travis on stable branches (envoyproxy#96) * Publish debug binaries (no release yet) (envoyproxy#98) * Copies the binary instead of linking for release (envoyproxy#102) * Not to use api_key if its service is not actived. (envoyproxy#109) * Update envoy and add c-ares (envoyproxy#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (envoyproxy#110) * Add send_attribute filter. (envoyproxy#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (envoyproxy#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (envoyproxy#125) * Send headers as string map. (envoyproxy#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (envoyproxy#127) * update base debug docker image reference (envoyproxy#133) * Update postsubmit to create docker images (envoyproxy#132) * Adding config release for bazel build (envoyproxy#135) * Fix mixer client crash. (envoyproxy#136) * Get mixerclient with response parsing. (envoyproxy#138) * Update nghttp2 to sync with envoy (envoyproxy#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Merge from master to firebase (envoyproxy#159) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (envoyproxy#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (envoyproxy#41) * Update prototype to use iptables (envoyproxy#42) * Rebase to fixed Envoy (envoyproxy#43) * Handle HEAD request. (envoyproxy#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (envoyproxy#48) * Try again (envoyproxy#49) * Integrate with mixer client. (envoyproxy#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (envoyproxy#56) * Add uuid-dev dependency in README.md (envoyproxy#45) * Extract originIp and OriginHost. (envoyproxy#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (envoyproxy#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (envoyproxy#59) * Use envoy new access_log handler for sending Report. (envoyproxy#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with envoyproxy#396. (envoyproxy#61) * Fix tclap dependency fetching error (envoyproxy#62) * Integrate mixer client directly with envoy. (envoyproxy#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (envoyproxy#68) * Push tar.gz to GCS (envoyproxy#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (envoyproxy#72) * Update mixer client SHA. (envoyproxy#74) * Update readme. (envoyproxy#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (envoyproxy#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (envoyproxy#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (envoyproxy#78) * Add script to build docker image. (envoyproxy#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (envoyproxy#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (envoyproxy#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (envoyproxy#82) * Fix src/envoy/mixer/README.md (envoyproxy#85) * Get attributes from envoy config. (envoyproxy#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (envoyproxy#94) * Disable travis on stable branches (envoyproxy#96) * Publish debug binaries (no release yet) (envoyproxy#98) * Copies the binary instead of linking for release (envoyproxy#102) * Not to use api_key if its service is not actived. (envoyproxy#109) * Update envoy and add c-ares (envoyproxy#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (envoyproxy#110) * Add send_attribute filter. (envoyproxy#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (envoyproxy#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (envoyproxy#125) * Send headers as string map. (envoyproxy#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (envoyproxy#127) * update base debug docker image reference (envoyproxy#133) * Update postsubmit to create docker images (envoyproxy#132) * Adding config release for bazel build (envoyproxy#135) * Fix mixer client crash. (envoyproxy#136) * Get mixerclient with response parsing. (envoyproxy#138) * Update nghttp2 to sync with envoy (envoyproxy#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Populate origin.user attribute from the SAN field of client cert (envoyproxy#142) * Test * test * test * revert file * address comments * test * fix typo * fix format * fix format * Update to latest mixer_client. (envoyproxy#145) * Update to latest mixer_client. * Updated the sha. * Not call report if decodeHeaders is not called. (envoyproxy#150) * Update mixerclient with sync-ed grpc write and fail-fast. (envoyproxy#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (envoyproxy#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Update the Commit id for envoy * Allow for HTTP based function from Firebase rules (envoyproxy#202) * Allow for HTTP based function from Firebase rules * Fix code style check * Added more comments. * Fix style issues. * Address code review comments from Limin and Lizan. * Add more comments and address CR comments. * Fix a typo. * Address Wayne's CR comments. * Merge from master to firebase (envoyproxy#237) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (envoyproxy#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (envoyproxy#41) * Update prototype to use iptables (envoyproxy#42) * Rebase to fixed Envoy (envoyproxy#43) * Handle HEAD request. (envoyproxy#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (envoyproxy#48) * Try again (envoyproxy#49) * Integrate with mixer client. (envoyproxy#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (envoyproxy#56) * Add uuid-dev dependency in README.md (envoyproxy#45) * Extract originIp and OriginHost. (envoyproxy#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (envoyproxy#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (envoyproxy#59) * Use envoy new access_log handler for sending Report. (envoyproxy#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with envoyproxy#396. (envoyproxy#61) * Fix tclap dependency fetching error (envoyproxy#62) * Integrate mixer client directly with envoy. (envoyproxy#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (envoyproxy#68) * Push tar.gz to GCS (envoyproxy#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (envoyproxy#72) * Update mixer client SHA. (envoyproxy#74) * Update readme. (envoyproxy#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (envoyproxy#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (envoyproxy#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (envoyproxy#78) * Add script to build docker image. (envoyproxy#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (envoyproxy#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (envoyproxy#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (envoyproxy#82) * Fix src/envoy/mixer/README.md (envoyproxy#85) * Get attributes from envoy config. (envoyproxy#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (envoyproxy#94) * Disable travis on stable branches (envoyproxy#96) * Publish debug binaries (no release yet) (envoyproxy#98) * Copies the binary instead of linking for release (envoyproxy#102) * Not to use api_key if its service is not actived. (envoyproxy#109) * Update envoy and add c-ares (envoyproxy#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (envoyproxy#110) * Add send_attribute filter. (envoyproxy#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (envoyproxy#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (envoyproxy#125) * Send headers as string map. (envoyproxy#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (envoyproxy#127) * update base debug docker image reference (envoyproxy#133) * Update postsubmit to create docker images (envoyproxy#132) * Adding config release for bazel build (envoyproxy#135) * Fix mixer client crash. (envoyproxy#136) * Get mixerclient with response parsing. (envoyproxy#138) * Update nghttp2 to sync with envoy (envoyproxy#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Populate origin.user attribute from the SAN field of client cert (envoyproxy#142) * Test * test * test * revert file * address comments * test * fix typo * fix format * fix format * Update to latest mixer_client. (envoyproxy#145) * Update to latest mixer_client. * Updated the sha. * Not call report if decodeHeaders is not called. (envoyproxy#150) * Update mixerclient with sync-ed grpc write and fail-fast. (envoyproxy#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (envoyproxy#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Uses a specific version of the Shared Pipeline lib (envoyproxy#158) * Update lyft/envoy commit Id to latest. (envoyproxy#161) * Update lyft/envoy commit Id to latest. * Remove the comment about pull request * Add new line - will delete in next commit. * Update repositories.bzl (envoyproxy#169) * Always set response latency (envoyproxy#172) * Update mixerclient to sync_transport change. (envoyproxy#178) * Use opaque config to turn on/off forward attribute and mixer filter (envoyproxy#179) * Modify mixer filter * Swap defaults * Make the filter decoder only * cache mixer disabled decision * Fix a bug in opaque config change and test it out (envoyproxy#182) * Fix a bug and test it out * Update filter type * Update README.md * Update mixer client to mixer api with gogoproto. (envoyproxy#184) * Move .bazelrc to tools/bazel.rc (envoyproxy#186) * Move .bazelrc to tools/bazel.rc * Update Jenkinsfile with latest version of pipeline * Support apikey based traffic restriction (envoyproxy#189) * b/36368559 support apikey based traffic restriction * Fixed code formatting * Fix crash in unreachable/overloaded RDS (envoyproxy#190) * Add mixer client end to end integration test. (envoyproxy#177) * Add mixer client end to end integration test. * Split some repositories into a separate file. * use real mixer for fake mixer_server. * Test repository * use mixer bzl file. * Use mixer repositories * Not to use mixer repository. * Add return line at the end of WORKSPACE. * Fix broken link (envoyproxy#193) * Make quota call (envoyproxy#192) * hookup quota call * Make quota call. * Update indent. * Update envoy and update configs (envoyproxy#195) * Update envoy and update configs * Use gcc-4.9 for travis * Use bazel 0.4.5 * Fix SHA of lightstep-tracer-common * Enable check cache and refactory mixer config loading (envoyproxy#197) * Refactory the mixer config loading. * fix format * Add integration test. * updated README.md * s/send/sent/ * Split into separate tests. (envoyproxy#201) * Update README on how to enable check cache. (envoyproxy#204) * Update README on how to enable check cache. * Update the comment. * build: support Envoy native Bazel build. (envoyproxy#210) * build: support Envoy native Bazel build. This patch switches the Envoy build from src/envoy/repositories.bzl to using the upstream native build. See envoyproxy#663 for the corresponding changes on the Envoy side. * Use Envoy master with BUILD.wip rename merged. * Fix clang-format issues. * Fixes bazel.rc issues (envoyproxy#212) * Fixes bazel rc issues * Update Jenkins to latest pipeline version * Fix go build (envoyproxy#224) * Use TranscoderInputStream to reduce confusion around ByteCount() (envoyproxy#225) * Add TranscoderInputStream to reduce confusion * fix_format * Merge latest changes from rate_limiting to master (envoyproxy#221) * Point to googleapi in service control client. (envoyproxy#91) * Point to googleapi in service control client. * Use git repository for service-control-client. * Merge latest changes from master (envoyproxy#104) * Get attributes from envoy config. (envoyproxy#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (envoyproxy#94) * Disable travis on stable branches (envoyproxy#96) * Publish debug binaries (no release yet) (envoyproxy#98) * Copies the binary instead of linking for release (envoyproxy#102) * Extract quota config from service config. (envoyproxy#101) * Add metric_cost in config. * Remove group rules. * Call loadQuotaConfig in config::create. * Update latest update from master branch (envoyproxy#106) * Get attributes from envoy config. (envoyproxy#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (envoyproxy#94) * Disable travis on stable branches (envoyproxy#96) * Publish debug binaries (no release yet) (envoyproxy#98) * Copies the binary instead of linking for release (envoyproxy#102) * Added quota contoll without the service control client library (envoyproxy#93) * Added quota contoll without the service control client library * Applied code review * Applied code review * Resolve conflicts * Resolve conflicts * Fixed format error reported by script/check-style * Fixed a bug at Aggregated::GetAuthToken that causes Segmentation Fault * Changed usage of template funcion * Applied latest changes from the repo * Applied latest changes from the repo * Applied latest changes from the repo * Adde comments * Updated log information * Applied envoyproxy#101 * Changed metric_cost_map to metric_cost_vector * Fixed test case compilation error * Fixed test case compilation error * Add unit test for quota config. (envoyproxy#108) * Add unit test for quota config. * Add comments. * Update test specifics. * Merge latest changes from master branch (envoyproxy#112) * Get attributes from envoy config. (envoyproxy#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (envoyproxy#94) * Disable travis on stable branches (envoyproxy#96) * Publish debug binaries (no release yet) (envoyproxy#98) * Copies the binary instead of linking for release (envoyproxy#102) * Not to use api_key if its service is not actived. (envoyproxy#109) * If QuotaControl service is not available, return utils::Status::OK (envoyproxy#113) * If QuotaControl service is not available, return utils::Status::OK * Updated comment * Return HTTP status code 429 on google.rpc.Code.RESOURCE_EXHAUSTED (envoyproxy#119) * Fixed incorrectly resolved conflicts (envoyproxy#123) * Added unit test cases for rate limiting (envoyproxy#124) * Fixed incorrectly resolved conflicts * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Rename response.http.code (envoyproxy#125) (envoyproxy#128) * Added handling of error code QUOTA_SYSTEM_UNAVAILABLE (envoyproxy#148) * Integrated service control client library with quota cache aggregation (envoyproxy#149) * Fixed error on merge (envoyproxy#151) * Integrated service control client library with quota cache aggregation * Fixed error on merge * Fixed the compatibility issue with the latest update on esp (envoyproxy#152) * Removed copied proto files (envoyproxy#208) * Set default allocate quota request timeout to 1sec and applied latest service control client library change (envoyproxy#211) * Merged key_restriction related changes from master (envoyproxy#213) * Merge latest changes from master branch (envoyproxy#217) * Not call report if decodeHeaders is not called. (envoyproxy#150) * Update mixerclient with sync-ed grpc write and fail-fast. (envoyproxy#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (envoyproxy#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Uses a specific version of the Shared Pipeline lib (envoyproxy#158) * Update lyft/envoy commit Id to latest. (envoyproxy#161) * Update lyft/envoy commit Id to latest. * Remove the comment about pull request * Add new line - will delete in next commit. * Update repositories.bzl (envoyproxy#169) * Always set response latency (envoyproxy#172) * Update mixerclient to sync_transport change. (envoyproxy#178) * Use opaque config to turn on/off forward attribute and mixer filter (envoyproxy#179) * Modify mixer filter * Swap defaults * Make the filter decoder only * cache mixer disabled decision * Fix a bug in opaque config change and test it out (envoyproxy#182) * Fix a bug and test it out * Update filter type * Update README.md * Update mixer client to mixer api with gogoproto. (envoyproxy#184) * Move .bazelrc to tools/bazel.rc (envoyproxy#186) * Move .bazelrc to tools/bazel.rc * Update Jenkinsfile with latest version of pipeline * Support apikey based traffic restriction (envoyproxy#189) * b/36368559 support apikey based traffic restriction * Fixed code formatting * Fix crash in unreachable/overloaded RDS (envoyproxy#190) * Add mixer client end to end integration test. (envoyproxy#177) * Add mixer client end to end integration test. * Split some repositories into a separate file. * use real mixer for fake mixer_server. * Test repository * use mixer bzl file. * Use mixer repositories * Not to use mixer repository. * Add return line at the end of WORKSPACE. * Fix broken link (envoyproxy#193) * Make quota call (envoyproxy#192) * hookup quota call * Make quota call. * Update indent. * Update envoy and update configs (envoyproxy#195) * Update envoy and update configs * Use gcc-4.9 for travis * Use bazel 0.4.5 * Fix SHA of lightstep-tracer-common * Enable check cache and refactory mixer config loading (envoyproxy#197) * Refactory the mixer config loading. * fix format * Add integration test. * updated README.md * s/send/sent/ * Split into separate tests. (envoyproxy#201) * Update README on how to enable check cache. (envoyproxy#204) * Update README on how to enable check cache. * Update the comment. * build: support Envoy native Bazel build. (envoyproxy#210) * build: support Envoy native Bazel build. This patch switches the Envoy build from src/envoy/repositories.bzl to using the upstream native build. See envoyproxy#663 for the corresponding changes on the Envoy side. * Use Envoy master with BUILD.wip rename merged. * Fix clang-format issues. * Fixes bazel.rc issues (envoyproxy#212) * Fixes bazel rc issues * Update Jenkins to latest pipeline version * Updated the commit id of cloudendpoints/service-control-client-cxx (envoyproxy#218) * Update commitid of cloudendpoints/service-control-client-cxx repo (envoyproxy#220) * Send delta metrics for intermediate reports. (envoyproxy#219) * Send delta metrics for intermediate reports. * Move last_request_bytes/last_response_bytes to RequestContext. * Handle final report. * Address comment. * Update attributes to match the canonical attribute list. (envoyproxy#232) * Update response.http.code to response.code and response.latency to response.duration to line up with the canonical attributes in istio/istio.github.io/docs/concepts/attributes.md * Format according to clang-format * Add envoy Buffer based TranscoderInputStream (envoyproxy#231) * Add envoy Buffer based TranscoderInputStream * fix format * A few doc changes for consistency across repos. (envoyproxy#235) * Add repositories.bzl * Added missing export setting in bazel configuration (envoyproxy#236) * Added export missing in bazel configuration * Added export missing in bazel configuration * Allow HTTP functions in firebase rules to specify audience (envoyproxy#244) * Allow HTTP functions in firebase rules to specify audience * Allow GetAuthToken to ignore cache and fix style checks. * Fix GetAuthToken * Address Wayne's comment * Check for empty response body * Remove .bazelrc.jenkins file not present in the master branch. * Remove forward_attribute_filter.cc not present in master.
Signed-off-by: Antonio Vicente <avd@google.com>
zh-translation:docs/root/install/install.rst
Adding sphinx extensions for sha references and issues Signed-off-by: Alan Chiu <achiu@lyft.com> For an explanation of how to fill out the fields, please see the relevant section in [PULL_REQUESTS.md](https://github.com/envoyproxy/envoy/blob/master/PULL_REQUESTS.md) Description: add sphinx extensions for tree and issues Risk Level: low Testing: local Docs Changes: /docs Release Notes: [Optional Fixes #Issue] [Optional Deprecated:] Signed-off-by: JP Simard <jp@jpsim.com>
Adding sphinx extensions for sha references and issues Signed-off-by: Alan Chiu <achiu@lyft.com> For an explanation of how to fill out the fields, please see the relevant section in [PULL_REQUESTS.md](https://github.com/envoyproxy/envoy/blob/master/PULL_REQUESTS.md) Description: add sphinx extensions for tree and issues Risk Level: low Testing: local Docs Changes: /docs Release Notes: [Optional Fixes #Issue] [Optional Deprecated:] Signed-off-by: JP Simard <jp@jpsim.com>
Adding changes for ppc64le.
This is a first take providing support for systematic failure injection using fault injection filters (issue #198 ). The filter supports two types of faults:
aborts
anddelays
. Several common failure scenarios manifest at the application layer as either delayed responses to requests or failure codes from upstream clusters.The semantics of abort vary from protocol to protocol. With HTTP, abort is expected to return a standard HTTP error code. With gRPC, abort would return one of the generic gRPC error codes. With TCP, abort would reset the tcp connection.
Similarly, the semantics of delay is different between TCP and HTTP. For HTTP, the delay is a one time delay before propagating the request upstream. For TCP, delay would be implemented as a bandwidth restriction on the TCP pipe between the downstream request and upstream cluster. [Similar principle might apply to a streaming gRPC request.]
The fault filter treats the aborts and delays as independent events and allows the user to inject either a delay or an abort or both based on a percentage of requests. This decoupling enables the user to model an overloaded service (e.g., delay response by 5s and return a HTTP 503).
This PR is a work in progress. The current PR has been tested only with HTTP (with Front proxy example, using curl client). I would like to get some feedback on the current approach. With the current code base, it is possible to inject delays and HTTP error codes, with optional ability to restrict the faults to requests containing a specific set of headers.
Some example configuration blocks (note: fault filter should be inserted before the routing filter).
delay_enabled
andabort_enabled
range from 1 to 100.delay_duration
is in milliseconds.abort_code
corresponds to HTTP or gRPC return codes (gRPC return code has not been tested).A simple fault filter injecting a 5s delay
A fault filter injecting a 5s delay with 50% probability, followed by an abort (code HTTP 503) with a 50% probability
Simple delay fault with header match
TODO: