Can not run a subprocess in a test: "the child watcher does not have a loop attached"

[cecton]

Long story short

The loop created for the testing purpose doesn't allow you to spawn subprocesses using asyncio.create_subprocess_*.

Expected behaviour

You can spawn subprocesses without problem like you would do with a default loop.

Actual behaviour

When you try to spawn a subprocess, you automatically get:

self = <asyncio.unix_events.SafeChildWatcher object at 0x7fe80eb15e10>, pid = 9399
callback = <bound method _UnixSelectorEventLoop._child_watcher_callback of <_UnixSelectorEventLoop running=False closed=False debug=False>>
args = (<_UnixSubprocessTransport pid=9399 running>,)

    def add_child_handler(self, pid, callback, *args):
        if self._loop is None:
            raise RuntimeError(
>               "Cannot add child handler, "
                "the child watcher does not have a loop attached")
E           RuntimeError: Cannot add child handler, the child watcher does not have a loop attached

/usr/lib64/python3.6/asyncio/unix_events.py:858: RuntimeError

Steps to reproduce

import asyncio


# tokio event loop does not allow to override attributes
def skip_if_no_dict(loop):
    if not hasattr(loop, '__dict__'):
        pytest.skip("can not override loop attributes")


def stopper(loop):
    def f(*args):
        loop.call_later(0.001, loop.stop)
    return f


async def test_subprocess_co(loop):
    proc = await asyncio.create_subprocess_shell("exit 0", loop=loop)
    await proc.wait()
    assert proc.retcode == 0


def test_subprocess(loop, mocker):
    skip_if_no_dict(loop)
    loop.run_until_complete(test_subprocess_co(loop))

Your environment

Arch Linux, Python 3.6

[cecton]

@asvetlov is it possible that it is because the current loop is set to None when running the test?

https://github.com/aio-libs/aiohttp/blob/master/aiohttp/test_utils.py#L442

I have no idea why we put that to None but setting it to loop fixes the issue. Should I make a PR?

[kblin]

The PR fixes an issue I was seeing trying to use a pytest-aiohttp event loop to test a function using asyncio.subprocess. Thanks!

[cecton]

@kblin yw \o/

As I explained there: #1946 (comment) You really don't need asyncio.create_subprocess_* in most cases. If you do not intend to actually use the event loop to run multiple things in parallel, I really don't see why you would introduce such complexity. But this is my opinion.

I'm using ayncio's subprocesses in my application so when I test it it runs under the test loop and that's how I ran into that problem.

[kblin]

Same here, I'm calling an external tool in the function that I wanted to test.

[cecton]

Fixed by #2075

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.