Add annotations to configure retries, max connections to a service #241
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cmaloney
force-pushed
the
ingress-annotation-retries
branch
2 times, most recently
from
826c2ba to
1ae96f9
Compare
|
Looks pretty good to me. Sadly there will be a merge conflict after #240. Hopefully this will be the last big change to the grpc types in the 0.4 cycle. |
cmaloney
force-pushed
the
ingress-annotation-retries
branch
from
1ae96f9 to
a1f9384
Compare
cmaloney
changed the title
cmaloney
force-pushed
the
ingress-annotation-retries
branch
from
a1f9384 to
c1d1f3f
Compare
|
@davecheney rebased on master. Has unit tests around the Uint32 annotation parsing. If there's anything else needed let me know |
These are needed in order to scale a service up as well as help cope with when a pod for a kubernetes service disappears unexpectedly. By default Envoy has a limit of 1024 simultaneous connections to a cluster and doesn't do any retries. A single http server backend of a service can handle tens of thousands of requests per second however, and in environments where that happens the low limit of envoy causes envoy to circuit break those services rather than "flood" an unsuspecting backend with requests. By default, envoy doesn't retry requests, which means if a request gets made to a pod which is the process of exiting / just stopped responding to connections (Say it segfaulted) the request gets dropped and the client sees an error rather than a retry. The retry annotations allow changing this policy as appropriate in the environment Signed-off-by: Cody Maloney <cody@emeraldcloudlab.com>
cmaloney
force-pushed
the
ingress-annotation-retries
branch
from
c1d1f3f to
c6f1df4
Compare
|
Any updates here? |
davecheney
added a commit
to davecheney/contour
that referenced
this pull request
Mar 12, 2018
davecheney
added a commit
to davecheney/contour
that referenced
this pull request
Mar 12, 2018
Signed-off-by: Dave Cheney <dave@cheney.net>
davecheney
added a commit
to davecheney/contour
that referenced
this pull request
Mar 12, 2018
Updates projectcontour#241 Signed-off-by: Dave Cheney <dave@cheney.net>
This was referenced Mar 12, 2018
davecheney
added a commit
to davecheney/contour
that referenced
this pull request
Mar 12, 2018
Signed-off-by: Dave Cheney <dave@cheney.net>
alvaroaleman
pushed a commit
to alvaroaleman/contour
that referenced
this pull request
Mar 12, 2018
Updates projectcontour#241 Signed-off-by: Dave Cheney <dave@cheney.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These are needed in order to scale a service up as well as help cope
with when a pod for a kubernetes service disappears unexpectedly. By
default Envoy has a limit of 1024 simultaneous connections to a cluster
and doesn't do any retries. A single http server backend of a service
can handle tens of thousands of requests per second however, and in
environments where that happens the low limit of envoy causes envoy to
circuit break those services rather than "flood" an unsuspecting backend
with requests.
By default, envoy doesn't retry requests, which means if a request gets
made to a pod which is the process of exiting / just stopped responding
to connections (Say it segfaulted) the request gets dropped and the
client sees an error rather than a retry. The retry annotations allow
changing this policy as appropriate in the environment
Signed-off-by: Cody Maloney cody@emeraldcloudlab.com
TODO:
If wanted, I can add new tests around the service + ingress specifically that the annotations get handled / passed thorugh correctly