-
Notifications
You must be signed in to change notification settings - Fork 674
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make configurable Envoy listener socketOptions to allow DSCP marking for QoS purposes #4605
Comments
Contour currently configures two types of listeners, the basic HTTP one, that does routing based on HTTP header Just to be clear, you're looking for Envoy to set the TCP Socket options across all listeners, or do you want this configurable at the per-vhost level? Because of the above structure, we'd only be able to do per-vhost for HTTPS listeners. The TCP socket options are to be set on the downstream, towards-the-client socket, correct? We may be able to implement this, but I would like to ask: Is Contour's Envoys the right place to do this? It will set the DiffServ info for the hop between Envoy and whatever downstream, closer-to-the-internet loadbalancer you have, and I'm not sure if cloud load balancers will respect the DiffServ bits. If you're running on a bare-metal loadbalancer that you control, then I understand a bit more. |
Hi @youngnick ! Many thanks for your reply!
No. We are not interested in configuring the listeners of admin or metrics interfaces for instance. Just towards clients that send requests to microservices via Contour.
This "HTTPS" word confuses me a little. Pure HTTP listeners - belonging for vhost defined in HTTPProxy - cannot be updated with these socket options?
Yes
Good point and was studied for a very long time. The conclusion was this is the preferred solution. |
since this is a socket level option and not something that can be configured per filter chain/vhost, and we only have the two ingress listen sockets (one for HTTP and one for HTTPS+TLS passthrough), this is an all or nothing configuration per listen socket |
Oh, of course, thanks @sunjayBhatia.
I wasn't clear here. What I'm trying to get at is that we need to figure out where we will put this configuration, and since Contour has two listeners, HTTP and HTTPS, the config will most likely need to live in the configuration file or CRD. The other thing to determine is whether we need to specify different DiffServ details for HTTP and HTTPS. It doesn't seem like this would be useful to me. What that leaves us with is a configuration setup added into our CRD and configuration file, that will let you configure these parameters, and have them applied to all sockets opened on the Contour Envoys. Does that describe what you want @egerkke? |
Hi Gents, Sorry for the late response, I was on short vacation. Many thanks for your replies!
I need to check if such use case cannot happen at our customer and will come back.
Yes |
Thanks @egerkke, if you could just confirm that you don't need different DiffServ details between HTTP and HTTPS, we'll get this prioritized. |
Hi @youngnick, It is confirmed that we don't need different DiffServ details between HTTP and HTTPS, as this is not valid use case for us. BR. |
The Contour project currently lacks enough contributors to adequately respond to all Issues. This bot triages Issues according to the following rules:
You can:
Please send feedback to the #contour channel in the Kubernetes Slack |
The Contour project currently lacks enough contributors to adequately respond to all Issues. This bot triages Issues according to the following rules:
You can:
Please send feedback to the #contour channel in the Kubernetes Slack |
Don't close the issue please, we are trying to get resource to implement it. |
F.Y.I. The last PR for listener was just merged: listener: enable socket_options for multiple addresses#24210 |
The Contour project currently lacks enough contributors to adequately respond to all Issues. This bot triages Issues according to the following rules:
You can:
Please send feedback to the #contour channel in the Kubernetes Slack |
Trying to get help in implementing the feature. |
The Contour project currently lacks enough contributors to adequately respond to all Issues. This bot triages Issues according to the following rules:
You can:
Please send feedback to the #contour channel in the Kubernetes Slack |
The Contour project currently lacks enough contributors to adequately respond to all Issues. This bot triages Issues according to the following rules:
You can:
Please send feedback to the #contour channel in the Kubernetes Slack |
Hi Ladies and Gents,
DSCP (differentiated services code point) is a mechanism used for classifying/prioritizing network traffic on IP networks DSCP_wiki. Since Contour is used as O&M external interface provider for microservices at our customer, request arrived to support DSCP marking. Therefore all microservice O&M HTTP interface DSCP marking can be done in a unified place.
DiffServ uses 6-bit differentiated services code point (DSCP) in the 8-bit differentiated services field (DS field) in the IP header for packet classification purposes (last 2 bits are reserved - CU). Please see picture with details:
Example Envoy IPv4 listener configuration to set CS2 selection via Envoy Socket Option
int_value 64 will be converted into 8 bits: 0 1 0 0 0 0 0 0 and used to fill the given IP header section of DS according to the picture.
Since the last 2 bits are reserved, this will end up in configuring 0 1 0 0 0 0 as Class Selector that is CS2 according to Class Selector mapping
This Class Selector can also be captured in IP packets received from Envoy:
Captured packets:
dscp_CS2_settings.pcapng.tar.gz
Example Envoy IPv6 listener configuration:
Example to configure both IPv4 and IPv6 with different DSCP values:
Reproduction steps (for Ubuntu):
For reproduction purposes the following configuration of Envoy can be used:
base_envoy_config_socket_options_CS2.txt
Create a kind network:
Start Envoy container:
Copy go, myserver.go.tar.gz (simple echo application) into container, install tcpdump and start myserver.go:
Start tcpdump in another shell:
Create direct entry in /etc/hosts file that targets the Envoy container:
In another shell execute curl command:
Results can be copied and analyzed:
Other notes:
The text was updated successfully, but these errors were encountered: