Graceful shutdown for client sessions

[frederikaalund]

Long story short

We have a great guide for graceful shutdown of server connections. We need the same guide for client connections.

Expected behaviour

I except that when I pack everything into async with, I don't have to worry about closing underlying connections.

Actual behaviour

Even the simplest client connection produces a ResourceWarning on exit. This was first reported in #1115.

Steps to reproduce

import asyncio
import aiohttp

async def main(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            await response.read()

url = 'https://example.org/'
loop = asyncio.get_event_loop()
loop.run_until_complete(main(url))
loop.close()

When run, this produces a ResourceWarning. This is quite unexpected given that everything is wrapped in async with.

Resolution

The resolution in #1115 was:

So if you add await asyncio.sleep(0) just before loop closing -- warning disappears.

I'd really prefer another way to go about it, but apparently this can't be done.

In my own tests, I had to use await asyncio.sleep(0.25). I also tried sleep(0.2), which failed approximately 1 out of 10 times.

Do we really have to rely on this mechanic? Is it completely impossible to have an await aiohttp.wait_closed() call or similar? Maybe said function could internally just call await asyncio.sleep(x), where x is found empirically through a comprehensive test.

In any case, at least add this to the documentation. That is, please add a Graceful Shutdown guide for client connections (as is already done for server connections).

Your environment

Python 3.6.1. Aiohttp 2.2.0.

[frederikaalund]

Or maybe even a busy loop:

async def wait_closed():
    while underlying_connections_are_still_open:
        await asyncio.sleep(0.01)

Really, anything but "just call await asyncio.sleep(x) before closing the loop" because x will vary from system to system.

[asvetlov]

Thank you for raising a question.
We have couple related issues: #2036 and #1925
Unfortunately the fix is relative long trip but it will be finished eventually.

If you want to make documentation upgrade PR -- please don't hesitate :)

[frederikaalund]

Closed by #2045.

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.