Drop default port from request

[mazhack]

2016/04/26 15:30:40 [INFO] consul: Using tag prefix "fabio_prod-"
2016/04/26 15:30:40 [INFO] consul: Watching KV path "/fabio_prod/config"
2016/04/26 15:30:40 [INFO] consul: Manual config changed to #1
2016/04/26 15:30:40 [INFO] HTTPS proxy listening on :443 with certificate /etc/fabio/ssl/wildcard_lostaxis_com.ca-bundle
2016/04/26 15:30:40 [INFO] Updated config to
2016/04/26 15:30:40 [INFO] consul: Health changed to #2181
2016/04/26 15:30:40 [INFO] Updated config to
route add losmovilesapi_lostaxis losmovilesapi.lostaxis.com/ http://192.168.100.123:32774/ tags "losmovilesapi.lostaxis.com,fabio_prod-losmovilesapi.lostaxis.com/"
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:41 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:42 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835
2016/04/26 15:30:42 [WARN] No route for losmovilesapi.lostaxis.com:443/v1/driver/13543547/835

¿the port 443 is not removed from route?

losmovilesapi.lostaxis.com:443/ != losmovilesapi.lostaxis.com/

[mazhack]

i change the tag, and now do not work without 443

2016/04/26 15:43:45 [INFO] consul: Health changed to #2208
2016/04/26 15:43:45 [INFO] Updated config to
route add losmovilesapi_lostaxis losmovilesapi.lostaxis.com:443/ http://192.168.100.123:32775/ tags "losmovilesapi.lostaxis.com,fabio_prod-losmovilesapi.lostaxis.com:443/"
2016/04/26 15:43:52 [WARN] No route for losmovilesapi.lostaxis.com/losmoviles/api/driver/1095923341/993
2016/04/26 15:43:52 [WARN] No route for losmovilesapi.lostaxis.com/losmoviles/api/driverphoto/1095923341
2016/04/26 15:43:52 [WARN] No route for losmovilesapi.lostaxis.com/losmoviles/api/driver/1095923341/993
2016/04/26 15:43:52 [WARN] No route for losmovilesapi.lostaxis.com/losmoviles/api/driver/1095923341/993

[mazhack]

finally it works but i need define 2 routes

2016/04/26 15:46:29 [INFO] Updated config to
route add losmovilesapi_lostaxis losmovilesapi.lostaxis.com/ http://192.168.100.123:32776/ tags "losmovilesapi.lostaxis.com,fabio_prod-losmovilesapi.lostaxis.com:443/,fabio_prod-losmovilesapi.lostaxis.com/"
route add losmovilesapi_lostaxis losmovilesapi.lostaxis.com:443/ http://192.168.100.123:32776/ tags "losmovilesapi.lostaxis.com,fabio_prod-losmovilesapi.lostaxis.com:443/,fabio_prod-losmovilesapi.lostaxis.com/"

all traffic is https, only 443 port is open, see the first comment. it is a bug or a feature?

[magiconair]

Hmm, that looks like a bug. I'll have a look.

[magiconair]

My guess is the same thing happens when you add a route for urlprefix-foo.com/ and call fabio with http://foo.com:80/. When I look at the table below then the fix would be to drop the default port from the incoming request. That means http://foo.com:80/ would be translated to http://foo.com/ and https://foo.com:443/ to https://foo.com/ before lookup.

In both cases the urlprefix-foo.com/ route would match but neither urlprefix-foo.com:80/ nor urlprefix-foo.com:443/ would although they would be an exact match with the incoming request. They would only match if :80 (or :443) is not the default port, .e.g http://foo.com:443/ or https://foo.com:80/.

This would only become a problem if you want to match on certain protocols I guess which I'm going to address when introducing the new prefix syntax as described in #42.

I guess dropping the default port from the incoming request is what you would expect to happen. But there is a small chance that this will break someones setup if they have a prefix urlprefix-foo.com:80/ with requests for http://foo.com:80/ which will then no longer match. I'll probably have to hide this behind a switch so that you can revert to the old behavior.

url                     urlprefix       match
http://foo.com/         foo.com/        yes
http://foo.com:80/      foo.com/        yes
http://foo.com:443/     foo.com/        no

http://foo.com/         foo.com:80/     yes
http://foo.com:80/      foo.com:80/     yes
http://foo.com:443/     foo.com:80/     no
http://foo.com:443/     foo.com:443/    no

https://foo.com/        foo.com/        yes
https://foo.com:443/    foo.com/        yes
https://foo.com:80/     foo.com/        no

https://foo.com/        foo.com:443/    yes
https://foo.com:443/    foo.com:443/    yes
https://foo.com:80/     foo.com:443/    no
https://foo.com:80/     foo.com:80/     yes

[magiconair]

@mazhack I've pushed a fix for this into a branch. I still need to add a switch to the configuration but could you check whether that works for you?

[mazhack]

Perfect, this fix my problems in ticket #87 too. Now i can redirect traffic using the protocol default port. Thank you. I am using fabio in production and is amazing!!!

[ksoftirqd]

Would it make any sense to strip away fabio's own port entirely from Host header for http services?

It seems like it's hard to run fabio as a service in e.g. mesos/marathon and use a host+url based routing.

[magiconair]

@ksoftirqd Can you give a concrete example?

[ksoftirqd]

If fabio gets assigned a dynamic port on which it is available for other services, services would need to connect to http://host:<random_port>/ and routing and tags in consul would need to have that port as well.
E.g. even for default port 9999, every rule has to have :9999 in the end if one wants to use host based routing.

[jamesrwu]

I think the issue is similar with http clients adding the port to the host header when connecting via a non-standard port:
curl --verbose 'http://blah/' sets the host header in the request to "Host: blah"
curl --verbose 'http://blah:80/' sets the host header to "Host: blah"
but
curl --verbose 'http://blah:1234/' sets the host header to "Host: blah:1234"

IMHO, I am ok with the way fabio treats this, as it is explicit in its urlprefix host registration. However, I can see the usefulness of adding a config option to have fabio match the host header while ignoring the port. IE: for a service with tag urlprefix-blah/, fabio would route a http request with Host: blah or Host: blah:1234 to that service.

[ksoftirqd]

What i tried to say is that fabio's own port should not probably be taken into account in routing rules.

Just to illustrate my point, here is an analogy with nginx:

server {
    listen 1234;
    server_name qqq;
    return 200 "qqq\n";
}
server {
    listen 1234;
    server_name zzz;
    return 200 "zzz\n";
}

No matter what port is used by nginx to listen on, second server stanza will be used for all requests where Host header contains "zzz". Regardless of port portion:

$ curl -H 'Host: zzz' localhost:1234
zzz
$ curl -H 'Host: zzz:1234' localhost:1234
zzz
$ curl -H 'Host: zzz:8888' localhost:1234
zzz

As of now, if we want to use host based routing in fabio, we have to append fabio's port to all services' tags (prefix-url-myhostname:<fabios_port>/).
Now if we run fabio in e.g. mesos/marathon and it is accessible on randomly assigned port 12345, all routing rules need to have that port.
If fabio instance gets restarted -- it'll become available under a new port and all services' rules have to be adjusted.

[magiconair]

@ksoftirqd Why are you running fabio on a random port? Its main use case is to route incoming traffic from the internet to frontend services (FE-BE or N-S) and not between services (BE-BE or E-W). For that you use service discovery. I'm assuming that you are trying to use fabio for BE-BE routing because you try to put it into the urlprefix- tag.

For FE-BE routing to work you usually run fabio on a well known port since you need to configure your inbound LB (which listens to 80/443) to forward traffic to the fabio instances. If fabio is your internet facing LB then you need to run it on 80/443 anyway. Please have a look at https://github.com/ebay/fabio/wiki/Deployment.

But maybe I'm missing something here and if so please explain.

[ksoftirqd]

Yes, we want to use it for BE-BE routing/load balancing, only because service discovery by itself doesn't do a very good job of spreading a load between available instances.

If you run fabio on port 80/443 it works as expected (change 23a1aca). It would be really great if fabio could be used in the same way on any port (without a need to append fabio's current non standard port to any of the routing rules).

My question is why not just strip away a port portion from every request's hostname before looking it up in the routing table? :-)

Thanks

[magiconair]

@ksoftirqd I need to think about that. You may be right that this isn't really necessary.

For now you can work around that by registering the route as urlprefix-fabio:*/foo. This should work out of the box.

[magiconair]

That was added with #163