Subtle issue with normalizing URIs

[benlovell]

Since 81509bc we've had issues with using this gem to communicate with elasticsearch. It seems that prior to this change, the request URI was being normalized correctly and thus not including the scheme and host segment. As is evidenced below:

And after 81509bc:

It seems some servers are strict in this regard. Elasticsearch apparently being one of them. 🙈

This is easily resolved by uri.normalize.request_uri instead of just normalizing the URI, but this causes some issues with proxying that I don't really understand.

[ixti]

Thanks for report. My bad. Will fix today.

[ixti]

It should not send schema://hostname inside METHOD ... line.

[ixti]

Actually I have misunderstood original code and wiped out things that were essential. Will provide a fix ASAP.

[ixti]

Released as 0.8.3.

[benlovell]

❤️

[britishtea]

I ran into an issue today due to this commit.

I often use Charles Proxy to inspect HTTP(S) requests. When proxying HTTPS request to, for example, https://www.google.com I see requests being made to https://www.google.comhttps://www.google.com/. I suspect this is due to the headline, which is "GET https://google.com/ HTTP/1.1".

The following code causes the problem:

require "http"

HTTP.via("0.0.0.0", 8888).get "https://www.google.com", :ssl => { :verify_mode => OpenSSL::SSL::VERIFY_NONE }

When doing the same thing with a persistent connection to http://www.google.com (plain HTTP) everything works as expected. The headline is "GET / HTTP/1.1" in that case.

Is http doing the correct thing here? I've not had such issues with other ruby libraries when proxying HTTPS requests.

[tarcieri]

@britishtea definitely seems like a bug, possibly related to Addressable?

[ixti]

@Connorhd fixed issue reported by @britishtea in 2625436