Keep Alive and HTTP Pipelining

em-http supports both keep-alive and HTTP pipelining. By default, all em-http requests add a Connection: close header to indicate that the connection should be closed immediately after servicing the request. However, many servers allow you reuse the same connection to fetch multiple resources (setup and teardown of a connection is not free, after all).

Keep-Alive and Persistent Connections

You can reuse the same persistent connection by issuing nested requests from within the callback function:

EventMachine.run do
  conn = EventMachine::HttpRequest.new('http://www.igvita.com/')
  
  req1 = conn.get :keepalive => true
  req1.callback {
    p [:success, 1, req1]

    req2 = conn.get :path => '/about/'
    req2.callback {
      p [:success, 2, req2]
    }
  }
end

In the example above, we re-use the same connection, but the requests are made serially – we wait for the first one to complete before we fire a subsequent request.

Pipelining

HTTP pipelining is a technique in which multiple HTTP requests are written out to a single socket without waiting for the corresponding responses. Pipelining is only supported in HTTP/1.1, not in 1.0. The pipelining of requests results in a dramatic improvement in page loading times, especially over high latency connections such as satellite Internet connections.

To pipeline your requests in em-http, simply fire multiple requests from within the same connection:

EventMachine.run do
  conn = EventMachine::HttpRequest.new('http://www.igvita.com/')
  
  pipe1 = conn.get :keepalive => true # request "/" root path
  pipe2 = conn.get :path => '/about/'

  pipe1.callback { }
  pipe2.callback { }
end

In the above case, a single connection is opened to igvita.com and multiple requests are dispatched simultaneously to the server. Note that when we make the first request to root path, we specify :keepalive => true to indicate to the server that the connection should not be closed following this request. When we dispatch the second request, we omit the :keepalive flag and em-http inserts a Connection:close header, which tells the upstream server that it can now close the persistent connection once it sends us all the outstanding data.

Note that not all web-servers support HTTP pipelining, so be careful.