aiohttp.client_exceptions.ClientPayloadError on feeds using chunked encoding

[fluffy-critter]

Comment feeds from phpBB are delivered using chunked encoding, in a way which makes aiohttp barf. Fetching via curl works:

$ curl -i https://songfight.net/forums/app.php/feed/posts
HTTP/1.1 200 OK
Date: Sun, 21 Jun 2020 07:30:08 GMT
Server: Apache
Cache-Control: private, must-revalidate
Set-Cookie: phpbb3_fhvs1_u=1; expires=Mon, 21-Jun-2021 07:30:08 GMT; path=/; domain=.songfight.net; HttpOnly
Set-Cookie: phpbb3_fhvs1_k=; expires=Mon, 21-Jun-2021 07:30:08 GMT; path=/; domain=.songfight.net; HttpOnly
Set-Cookie: phpbb3_fhvs1_sid=da219fd5cb7c6839f6f2ecaae0ce9dd7; expires=Mon, 21-Jun-2021 07:30:08 GMT; path=/; domain=.songfight.net; HttpOnly
Upgrade: h2
Connection: Upgrade
Last-Modified: Sun, 21 Jun 2020 07:21:51 GMT
Cache-Control: max-age=172800
Expires: Tue, 23 Jun 2020 07:30:08 GMT
Vary: User-Agent
Transfer-Encoding: chunked
Content-Type: application/atom+xml

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en-gb">
...

but in pushl it fails:

$ pipenv run pushl -rvvvk --rel-exclude '' https://songfight.net/forums/app.php/feed/posts
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:pushl:++WAIT: https://songfight.net/forums/app.php/feed/posts: get feed
DEBUG:pushl.feeds:++WAIT: cache get feed https://songfight.net/forums/app.php/feed/posts
DEBUG:pushl.feeds:++DONE: cache get feed https://songfight.net/forums/app.php/feed/posts
DEBUG:pushl.feeds:++WAIT: request get https://songfight.net/forums/app.php/feed/posts None)
DEBUG:utils:https://songfight.net/forums/app.php/feed/posts: got error <class 'aiohttp.client_exceptions.ClientPayloadError'> Response payload is not completed (retry=0)
DEBUG:utils:https://songfight.net/forums/app.php/feed/posts: got error <class 'aiohttp.client_exceptions.ClientPayloadError'> Response payload is not completed (retry=1)
DEBUG:utils:https://songfight.net/forums/app.php/feed/posts: got error <class 'aiohttp.client_exceptions.ClientPayloadError'> Response payload is not completed (retry=2)
DEBUG:utils:https://songfight.net/forums/app.php/feed/posts: got error <class 'aiohttp.client_exceptions.ClientPayloadError'> Response payload is not completed (retry=3)
DEBUG:utils:https://songfight.net/forums/app.php/feed/posts: got error <class 'aiohttp.client_exceptions.ClientPayloadError'> Response payload is not completed (retry=4)
WARNING:utils:https://songfight.net/forums/app.php/feed/posts: Exceeded maximum retries; errors: {'Response payload is not completed'}
DEBUG:pushl.feeds:++DONE: request get https://songfight.net/forums/app.php/feed/posts
ERROR:pushl.feeds:Could not get feed https://songfight.net/forums/app.php/feed/posts: -1
DEBUG:pushl:++DONE: https://songfight.net/forums/app.php/feed/posts: get feed
INFO:pushl.main:Completed all tasks

There is probably some configuration that needs to be sent to aiohttp to make it more tolerant of chunked encoding weirdness.

[fluffy-critter]

Weirdly enough, it works fine from macOS, but not from Linux... argh.