Host header may include port #2276
Comments
|
We ran into the same issue and worked around it by using |
|
@MattSurabian how exactly you made it work? I am having the same problem |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
not stale |
|
@yokiworks Sorry for the delayed response here, you've probably figured it out or set it down by now. But if you're still wondering here's the mapping object spec as an example of the regex workaround I mentioned above: The UNESCAPED It should be noted that this work around would also allow host header matching for mutations of the TLD. Specifically, the above example will also host map to |
|
@MattSurabian which version of ambassador are you using? We tried it with 1.4.2 but could not get it to work. It feels like the regex is only applied to the hostname without the port. |
|
Been putting off the upgrade to 1.X.X, we're the .8X.0 minor version family still. Was looking to do the migration in a few weeks, sounds like I may be in for at least one surprise when I do. I'd assume the dropped support for this workaround is through Envoy, right? I'll let you know if I end up with a different work around on the latest version. Sorry for misleading anyone that's running the newer version :-( |
|
Found this problem in 1.4.2 too. With regex it works with only 1 host. If you have multiple host and configure all of them using regex, only 1st host works (Ascending orders). |
|
Same issue with us, exactly same problem. It regular expression it only works with the first host. |
|
We upgraded recently to 1.5.3 and hit this. Somewhere between 0.86 and 1.5.3, Ambassador moved from a wildcard tl;dr(Some of the
The path toward a fix?Ideally, the result is that Ambassador implements functionally equivalent logic to envoy#10960, so routing decisions treat an explicit external port as equivalent to implicit-port requests. Then, explicit port requests will work, and do so without a separate It seems the shortest path to that is by:
Figuring out the external port number might be tricky. It looks like some of the k8s service envvars have the external port number (I'm using the Mapping comparison between versionsFor example, with these mappings: Ambassador 0.86 generates this config: but the following one with 1.5.3 ( Looking to the Envoy upstream for a fixI found Envoy's troubleshooting guide for cases like this, which backs up the conclusion that At first, I thought envoy#10960 (#2818 datawire/envoy#3) would help with the
Beyond that, every Envoy binary I built segfaulted on startup. I thought Envoy's walk of its protobuf messages on startup caused this because of the protobuf field numbering discontinuity in my backport. Even after fixing that, Envoy still segfaulted with the same backtrace. Kludging additional values into
|
|
@jwm Thanks for the detailed info! What I'm thinking of here is
Is that something you could test quickly if we toss a build your way? |
|
@kflynn totally. Thanks for looking into this! |
|
@jwm I am also facing this issue. I am getting 404's when an external Prometheus stack is trying to scrape a /metrics endpoint that I have behind a mapping. I would also be happy to test. |
|
I am facing similar issue, i tried workaround with regex it didn't work and getting 404's (working browser and curl). Is there any update on issue |
|
upgrading ambassador on our production cluster completely broke the monitoring of all our microservices due to this issue, has there been any progress? it seems as if it hasn't been figured out, which means we will probably have to migrate to another solution unfortunately |
|
Note, this has been added as a config on Envoy envoyproxy/envoy#10960. @kflynn the better approach would be to allow users to specify enabling this flag in the |
|
As is probably obvious, since my last comment here I got pulled in (many) different directions. 🙁 @matdehaast, many thanks! As of 1.7.0, we have that Envoy fix, so let me see if there's a quick way to enable that flag. |
|
Hi, is there an update on this? We are currently facing an issue on integrating a third party solution because of this. Im wondering if i should wait for this to get solved or to go around it another way 😊 |
|
Hi, I think you can also do a literal match on Hope that helps! |
|
Cool, i tried your Lua suggestion and it seems to be working Will this however be suported by ambassador so i dont need to have a Lua script run on each request? I did not get the host_regex to work last time i tried sadly |
|
Hm, did this behavior end up changing in 1.7.x? I upgraded today, and noticed that our local kludge wasn't necessary any more. In fact, it was breaking the Envoy config, which wound up looking like: which caused: Removing the kludge (i.e., going back to a stock Ambassador) yields: |
|
Oops, ignore that previous comment. There was a |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
not stale |
|
This this issue was fixed in the 1.12 release, which is now available. |
|
@khussey Are you sure this issue is (still) fixed? Running works fine. But using: (which is close to what Prometheus is doing internally) results in a 404. As far as I understand, Prometheus' behaviour is that of every Go-application and does comply with the relevant RFCs. We have also tried adding every thinkable variant of wildcards in the route, with no success. Has this bug just resurfaced, or is there anything else we need to do to make it work? |
|
This can be worked around with the below Lua script. It effectively strips the port from the host header prior to evaluating the routing behavior. This will work both with HTTP/1.1 and gRPC. ---
apiVersion: getambassador.io/v2
kind: Module
metadata:
name: ambassador
namespace: ambassador
spec:
config:
# Use the items below for config fields
lua_scripts: |
function envoy_on_request(request_handle)
local authority = request_handle:headers():get(":authority")
if(string.find(authority, ":") ~= nil)
then
local authority_index = string.find(authority, ":")
local stripped_authority = string.sub(authority, 1, authority_index - 1)
request_handle:headers():replace(":authority", stripped_authority)
end
endedit: Also, to be clear, the issue you're seeing is slightly different from the one fixed in the above issue. In your case, the problem is that the server name being requested by the application during SNI is mismatched with the Host header. If you were to run |
|
While a workaround is certainly nice, isn't this something that should be fixed "properly"? As far as I understand it, any Go-Application trying to reach something behind ambassador should be having these issues. And the behaviour of adding :443 to the Host-header may be strange, but absolutely valid. Are you sure this is an SNI-Issue? The returned certificate is correct, just the request-routing is borked. People above in this thread were describing the exact same issue I had. "Prometheus gets a 404 when scraping", so I don't understand how my issue differs from the original one. It just seems like the original issue was never really fixed at all. |
|
So if I understood correctly you can't expose ambassador on a port different than the default (either 80 or 443). Otherwise every request's header will be filled with the port in the 'Host' field and Ambassador is not able to handle the mapping correctly. I can only make it working if I use curl with the option I tried to use the This only applies if the port matches the underlying Envoy listener port. Unfortunately I can't understand what this means in practice. The only thing that worked in my case was to add the |
|
In addition to the issue described here, I found that Ambassador's host mathing is case-sensitive i.e. it matches a request with |
and others
Describe the bug
If the
Hostheader provided in a request includes the port, Ambassador responds with a 404 where there is an otherwise valid route.To Reproduce
Steps to reproduce the behavior:
curl https://myhostname.comwill work because curl sendsHost: myhostname.comcurl https://myhostname.com -H "Host: myhostname.com:443"will return a 404 with an empty bodyExpected behavior
Ambassador should ignore the port portion of the Host header, or at least treat the default ports as equivalent to the absence of a port.
Versions (please complete the following information):
Additional context
Our client is using Qt's
QNetworkAccessManagerto send requests, but upon testing Ambassador I have found that requests were failing from our client (but working from browsers, curl, etc.) Initially I assumed this was a bug in Qt, but the RFC states that the Host header may include a port.The text was updated successfully, but these errors were encountered: