Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emissary 404's request with host:port in Host Header #3625

Open
jmeza-xyz opened this issue Jul 21, 2021 · 2 comments
Open

Emissary 404's request with host:port in Host Header #3625

jmeza-xyz opened this issue Jul 21, 2021 · 2 comments
Labels
stale Issue is stale and will be closed t:bug Something isn't working v2 Related to Emissary-Ingress 2 (2.y.z)

Comments

@jmeza-xyz
Copy link

jmeza-xyz commented Jul 21, 2021

Describe the bug
Prometheus scrape GET request uses host with port to grab metrics from an endpoint (example-server.com:443/metrics). Emissary does not match the host/authority with port to Mapping with host set to (example-server.com). Same issue as #2276

To Reproduce
Steps to reproduce the behavior:

  1. Install Emissary v2.0.0ea with helm
  2. Deploy AmbassadorListener, AmbassadorHost, AmbassadorMapping, TLSContext
  3. Enable scrape in Prometheus config to host behind Emissary like prometheus-internal.example.com
  4. Curl with Host curl https://prometheus-internal.example.com -H'host:prometheus-internal.example.com:443'
  5. Emissary responds with 404
> GET / HTTP/1.1
> Host:prometheus-internal.example.com:443
> User-Agent: curl/7.77.0
> Accept: */*
< HTTP/1.1 404 Not Found
< date: Wed, 21 Jul 2021 18:40:55 GMT
< server: envoy
  1. Emissary logs confirm
Example scrape from Prometheus:
ACCESS [2021-07-21T18:21:43.936Z] "GET / HTTP/1.1" 404 NR 0 0 0 - "10.11.10.18" "Prometheus/2.25.2" "5d11d9bc-2242-427d-9b57-a032a8beda16" "prometheus-internal.example.com:443" "-"
Example curl:
ACCESS [2021-07-21T19:14:05.737Z] "GET / HTTP/1.1" 404 NR 0 0 0 - "10.11.91.181" "curl/7.77.0" "b94a32ee-d5bd-4940-8e4a-cb3476acd3f5" "prometheus-internal.example.com:443" "-"

Expected behavior
Emissary to route correctly by handling host:port in host/authority header

Versions (please complete the following information):

  • Ambassador: v2.0ea
  • Kubernetes environment: k3s (1.20.7) , EKS (1.19)
  • Version: 2.0.0-dev.383+g00c1c766b

Additional context
Attempted work around mentioned in #2276, none worked except for the Lua script mentioned here. This is fine for testing, but not a solution for multi-cluster deployment.

@efunk efunk added the v2 Related to Emissary-Ingress 2 (2.y.z) label Jul 27, 2021
@efunk efunk added this to the 2021 Cycle 6 milestone Aug 20, 2021
@khussey khussey modified the milestones: 2021 Cycle 6, 2021 Cycle 7 Oct 13, 2021
@khussey khussey removed this from the 2021 Cycle 7 milestone Jan 4, 2022
@sumit-salunke
Copy link

sumit-salunke commented Dec 2, 2022

Is there any update on it ?

@OmegaVVeapon
Copy link
Contributor

OmegaVVeapon commented Dec 6, 2022

We're stuck on this as well.
As mentioned in the description, the only thing that has worked is the Lua script.
Running emissary:2.3.2.

I'm running Ambassador on debug and the strip_matching_host_port config documented in https://www.getambassador.io/docs/edge-stack/latest/howtos/grpc#mappings-with-hosts doesn't seem to do absolutely anything.

I see the :authority header has the 443 port regardless of whether the config is true or false in the conn_manager_impl request headers.

I imagine that it has to do with the part that says:

This only applies if the port matches the underlying Envoy listener port.

but it's extremely unclear to what this is referring to.

Do you have an example of how to configure that so that the port is actually stripped from the request?

Update: Been checking inside the Ambassador container and I see that this line is most likely referring related to the Listener CRD but can't make it bind on 443 so I'm suspecting that that's the reason the strip_matching_host_port setting isn't doing anything for us...

@cindymullins-dw cindymullins-dw added the t:bug Something isn't working label Nov 3, 2023
@dosubot dosubot bot added the stale Issue is stale and will be closed label Jul 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Issue is stale and will be closed t:bug Something isn't working v2 Related to Emissary-Ingress 2 (2.y.z)
Projects
None yet
Development

No branches or pull requests

6 participants