Trio loop

[mehaase]

This PR adds support for running a long-lived Trio event loop on the foreground thread and pushes the ZMQ/other kernel stuff into a background thread. When Trio runs on the main thread, all cells are executed as tasks inside a global nursery. The global nursery is also injected into builtins as GLOBAL_NURSERY so that notebook users can run tasks in the background even after a cell finishes executing. Exceptions in background tasks are caught and displayed in the current cell.

This requires a matching PR on IPython itself!!

The loop is activated by passing a kernel flag, so by default this PR doesn't affect any current users—not even existing %autoawait trio users. It's completely opt-in. The following example configuration shows how to start a kernel in Trio loop mode:

{
 "argv": [
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}",
  "--IPKernelApp.trio_loop=True"
 ],
 "display_name": "Python 3 Trio",
 "language": "python"
}

(Save this file as .../share/jupyter/python-trio/kernel.json. The leading ... part will vary depending on whether you're installing system-wide, per-user, or within a Python venv.)

In Trio loop mode, the kernel implicitly runs %autoawait trio at startup. It also disables the %autoawait magic to prevent users from trying to switch loops, which would certainly fail.

Here's an example notebooks session running in this mode:

We know the code isn't pretty, and we're seeking feedback on some of the implementation decisions, including (but not limited to):

• How to make a global nursery available to notebook users? Injecting into builtins is pretty hacky, as is hard-coding the name GLOBAL_NURSERY.
• How to inject the TrioRunner created in kernelapp.py into the shell instance, where it is needed to run cells on the main Trio loop.
• Whether to disable %autoawait and how best to do so. If disabled, what's the best way to warn the user?

[Carreau]

Matching PR is in IPython and has been released as IPython 7.12.

[auphofBSF]

Firstly thank you for this work, I am trying to use trio and derivative classes in a ipython/ipykernel running within Hydrogen in Atom IDE. (version details below) Your Untitled17 example above works but I am getting some issues. I will write the 2nd issue in a followup comment. Important to note the issue does not appear when I run in Jupyter notebook

1)In AtomIDE with Hydrogen on the first run of a kernel previously stopped. Executing import trio produces the following error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~\Anaconda3\envs\BSFE20W07PY37dataAnalysis\lib\site-packages\trio\_threads.py in _run_fn_as_system_task(cb, fn, trio_token, *args)
    324         try:
--> 325             trio_token = TOKEN_LOCAL.token
    326         except AttributeError:

AttributeError: '_thread._local' object has no attribute 'token'

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
c:\users\bsfau\cloudstation\bsfsoftdev\axternal\ipykernel\ipykernel\trio_runner.py in __call__(self, async_fn)
     58             self._cell_cancel_scope = None
     59 
---> 60         return trio.from_thread.run(loc, async_fn, trio_token=self._trio_token)

~\Anaconda3\envs\BSFE20W07PY37dataAnalysis\lib\site-packages\trio\_threads.py in from_thread_run(afn, trio_token, *args)
    389         trio.hazmat.spawn_system_task(await_in_trio_thread_task, name=afn)
    390 
--> 391     return _run_fn_as_system_task(callback, afn, *args, trio_token=trio_token)
    392 
    393 

~\Anaconda3\envs\BSFE20W07PY37dataAnalysis\lib\site-packages\trio\_threads.py in _run_fn_as_system_task(cb, fn, trio_token, *args)
    326         except AttributeError:
    327             raise RuntimeError(
--> 328                 "this thread wasn't created by Trio, pass kwarg trio_token=..."
    329             )
    330 

RuntimeError: this thread wasn't created by Trio, pass kwarg trio_token=...

Retrying the import trio executes successfully and then all lines as per your example execute successfully. If I restart the kernel there is no error, only after shutdown of kernel does it appear

Versions:

ipykernel : origin https://github.com/HyperionGray/ipykernel.git -branch: trio-loop commit 3bb1bf5 31/1/2020
ipython 7.12.0
trio 0.13.0
python 3.7.6
atom 1.43.0
electron 4.2.7
chrome 69.0.3497.128
node v10.11.0
hydrogen 2.13.1
windows 10
conda version : 4.7.10
conda-build version : 3.18.9
python version : 3.6.9.final.0

[auphofBSF]

Warnings from %autoawait ??? are only shown on 1st execution, see below

This is a 2nd Issue to #479 (comment) ,same version details apply

[blink1073]

Thanks!

[mehaase]

For people who read this thread in the future, I made a new package trio-jupyter that installs a Jupyter kernelspec with Trio-mode enabled. This simplifies the process of setting it up. https://github.com/mehaase/trio-jupyter

[blink1073]

Nice, thanks for sharing @mehaase!