Cost saving: Feature request to enable switch to toggle Envoy compression filter

[chaosbox]

Details:
Currently, Contour attaches a compression filter to envoy httpConnectionManager by default, as a result compression is enabled everywhere. While this feature is beneficial in certain scenarios, we have observed that the associated CPU cost can be prohibitively high compared to network costs. Specifically, we've found that CPU is more expensive than network bandwidth in our use case.

Rationale:
Compression is particularly useful for last-mile traffic. However, when CDNs front the ingress layer, which has dedicated bandwidth to origin servers, the network impact is negligible. In such cases, the return on investment for CPU cycles used in compression is significantly reduced. Therefore, the cost savings of disabling compression outweigh the benefits when not dealing with last-mile traffic or ingress layers meant for internal traffic.

Proposal:
We propose adding a control-plane configuration option, such as --disable-compression=true, to allow users to skip compression from the default filters. This change would provide significant cost savings in scenarios where compression is not beneficial.

Reduces CPU usage and allows for more efficient use of resources by running lean Envoy instances. We believe it will add significant value for users who, like us, need to optimize for cost and resource efficiency.

Observations from Our Cluster Testing:

• Enabling compression increased CPU utilization by nearly three times.
  • Without compression, the same workload utilized approximately 0.6 CPU cores.
  • With compression enabled, CPU utilization spiked to 2 cores.
  • The payload per request during our test was 310KB (uncompressed) and 85KB (compressed).
    Given these results, we would prefer to run lean instances of Envoy and scale out as needed.

[github-actions]

Hey @chaosbox! Thanks for opening your first issue. We appreciate your contribution and welcome you to our community! We are glad to have you here and to have your input on Contour. You can also join us on our mailing list and in our channel in the Kubernetes Slack Workspace

[aecay]

Additional context: way back in #310, the possibility was discussed to have a switch to disable the compression logic; this didn't wind up being implemented in the end. The cost savings we are discussing here (I am a colleague of @chaosbox 's) would effectively give another justification for adding the toggle (in addition to the security-related justification also mentioned on the linked ticket).

[github-actions]

The Contour project currently lacks enough contributors to adequately respond to all Issues.

This bot triages Issues according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, the Issue is closed

You can:

• Mark this Issue as fresh by commenting
• Close this Issue
• Offer to help out with triage

Please send feedback to the #contour channel in the Kubernetes Slack