aiohttp.Client can't parse Trailer headers and throws exception

[yurifedoseev]

Long story short

I work with some storage service via http. It sends me chunked response and several trailing headers after the body. All of this trailers are specified in a Trailer header according to https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Trailer

First problem: the client completely ignores this headers and doesn't parse them. Such behaviour violates HTTP 1.1

However, there is a bigger problem. If you use a single session the parsers would start the next parsing from the previous (!) response - beginning from these trailing headers. So the parser thinks it's the first line of new response and throws exception because it didn't find HTTP verb in it.
Therefore I can't reuse session in my app which leads to performance problems :(

Expected behaviour

1. The client has to parse trailing headers in chunked response
2. The next response should not be affected by any data from previous

Actual behaviour

1. Client ignores trailer headers.
2. The next response is affected by trailer headers from previous. The parser throws BadStatusLine exception.

Steps to reproduce

1. Create server which respond which send chunked response with trailing headers after the body. A special Trailer header should be placed before the body and contains all trailing headers names.
2. Create a new ClientSession. All next responses do via this session
3. Send the first request to your server. You won't find any trailer headers in the response
4. Send the second request. The client session will throw exception BadStatusLine

Checked on macOs 10.12.3 and Ubuntu Trusty 14.04

[fafhrd91]

seems example from https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Trailer page
is wrong. am i right?

it should be like this

HTTP/1.1 200 OK 
Content-Type: text/plain 
Transfer-Encoding: chunked
Trailer: Expires

7\r\n 
Mozilla\r\n 
9\r\n 
Developer\r\n 
7\r\n 
Network\r\n 
0\r\n 
Expires: Wed, 21 Oct 2015 07:28:00 GMT\r\n
\r\n

[yurifedoseev]

Yes, you're right. I used it just to illustrate headers and trailers order

[fafhrd91]

ok, should be easy to fix

[fafhrd91]

so new parser should strip trailers properly. but should we provide some api to access trailers data?

[fafhrd91]

fixed in master

[fafhrd91]

how to access trailer is in #1652

[yurifedoseev]

In my case I don't need these headers, however, it'd a nice feature to have (access to trailer)
Thanks a lot for your fix

[yurifedoseev]

Hi, can you say when do you plan to release a new version with this fix?

[fafhrd91]

I am planing to release 2.0dev next Tuesday.
2.0 final release, probably week after

[yurifedoseev]

Hi, I've just updated to version 2.0.5
Unfortunately, the fix didn't solve the problem with Trailers. I have the same error when use single session for requests

[fafhrd91]

Could you create test case

[asvetlov]

Actually the fix works as expected.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new