Exceptions when spawning process(es) with `multiprocessing` #1198

pentschev · 2023-02-03T12:30:40Z

Description

Running code that relies on spawning new processes with the multiprocessing library raises exceptions during runpy.run_path. This was initially discovered when running a Dask-CUDA cluster that spawns workers as multiple processes and is commonly used in Jupyter. Such errors are also minimally reproducible using multiprocessing only.

Rolling back several versions, I could not reproduce the error on jupyter_server=1.18.1, but jupyter_server=1.19.1 and above would already reproduce variations of the same error. For example, with jupyter_server=2.0.0:

Reproduce

Start a docker miniconda3 docker container (optional, baremetal will reproduce the error too): docker run -t -i -p 8888:8888 continuumio/miniconda3 /bin/bash
Create a new conda environment with Jupyter only: conda create -n jlab-test --override-channels --strict-channel-priority -c conda-forge -c nodefaults jupyterlab;
Activate environment: conda activate jlab-test;
Start JupyterLab: jupyter-lab --ip=0.0.0.0 --port=8888 --no-browser --allow-root (--allow-root should be skipped if not running on docker);
Connect to JupyterLab;
Create new notebook;
Paste the snippet (see below) in first cell;
Save;
Run cell (Shift+Enter).

import multiprocessing as mp
import time

p = mp.get_context("spawn").Process(target=time.sleep, args=(10,))
p.start()

Expected behavior

Spawning processes with multiprocessing should pass successfully without exceptions in runpy.run_path(...).

Context

Operating System and version: Debian Bullseye
Browser and version: Chromium 108.0.5359.124
Jupyter Server version: 1.19.1 - 2.2.1

Troubleshoot Output

$PATH: /opt/conda/envs/jlab-test/bin /opt/conda/condabin /opt/conda/bin /usr/local/sbin /usr/local/bin /usr/sbin /usr/bin /sbin /bin

sys.path:
/opt/conda/envs/jlab-test/bin
/opt/conda/envs/jlab-test/lib/python311.zip
/opt/conda/envs/jlab-test/lib/python3.11
/opt/conda/envs/jlab-test/lib/python3.11/lib-dynload
/opt/conda/envs/jlab-test/lib/python3.11/site-packages

sys.executable:
/opt/conda/envs/jlab-test/bin/python

sys.version:
3.11.0 | packaged by conda-forge | (main, Jan 14 2023, 12:27:40) [GCC 11.3.0]

platform.platform():
Linux-4.15.0-189-generic-x86_64-with-glibc2.31

which -a jupyter:
/opt/conda/envs/jlab-test/bin/jupyter

pip list:
Package Version
----------------------------- -----------
aiofiles 22.1.0
aiosqlite 0.18.0
anyio 3.6.2
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
asttokens 2.2.1
attrs 22.2.0
Babel 2.11.0
backcall 0.2.0
backports.functools-lru-cache 1.6.4
beautifulsoup4 4.11.2
bleach 6.0.0
brotlipy 0.7.0
certifi 2022.12.7
cffi 1.15.1
charset-normalizer 2.1.1
comm 0.1.2
cryptography 39.0.0
debugpy 1.6.6
decorator 5.1.1
defusedxml 0.7.1
entrypoints 0.4
executing 1.2.0
fastjsonschema 2.16.2
flit_core 3.8.0
idna 3.4
importlib-metadata 6.0.0
importlib-resources 5.10.2
ipykernel 6.21.1
ipython 8.9.0
ipython-genutils 0.2.0
jedi 0.18.2
Jinja2 3.1.2
json5 0.9.5
jsonschema 4.17.3
jupyter_client 8.0.2
jupyter_core 5.2.0
jupyter-events 0.5.0
jupyter_server 2.2.1
jupyter_server_fileid 0.6.0
jupyter_server_terminals 0.4.4
jupyter_server_ydoc 0.6.1
jupyter-ydoc 0.2.2
jupyterlab 3.6.0
jupyterlab-pygments 0.2.2
jupyterlab_server 2.19.0
MarkupSafe 2.1.2
matplotlib-inline 0.1.6
mistune 2.0.4
nbclassic 0.5.1
nbclient 0.7.2
nbconvert 7.2.9
nbformat 5.7.3
nest-asyncio 1.5.6
notebook 6.5.2
notebook_shim 0.2.2
packaging 23.0
pandocfilters 1.5.0
parso 0.8.3
pexpect 4.8.0
pickleshare 0.7.5
pip 23.0
pkgutil_resolve_name 1.3.10
platformdirs 2.6.2
prometheus-client 0.16.0
prompt-toolkit 3.0.36
psutil 5.9.4
ptyprocess 0.7.0
pure-eval 0.2.2
pycparser 2.21
Pygments 2.14.0
pyOpenSSL 23.0.0
pyrsistent 0.19.3
PySocks 1.7.1
python-dateutil 2.8.2
python-json-logger 2.0.4
pytz 2022.7.1
PyYAML 6.0
pyzmq 25.0.0
requests 2.28.2
Send2Trash 1.8.0
setuptools 67.1.0
six 1.16.0
sniffio 1.3.0
soupsieve 2.3.2.post1
stack-data 0.6.2
terminado 0.17.1
tinycss2 1.2.1
tomli 2.0.1
tornado 6.2
traitlets 5.9.0
typing_extensions 4.4.0
urllib3 1.26.14
wcwidth 0.2.6
webencodings 0.5.1
websocket-client 1.5.0
wheel 0.38.4
y-py 0.5.5
ypy-websocket 0.8.2
zipp 3.12.0

conda list:
# packages in environment at /opt/conda/envs/jlab-test:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
aiofiles 22.1.0 pyhd8ed1ab_0 conda-forge
aiosqlite 0.18.0 pyhd8ed1ab_0 conda-forge
anyio 3.6.2 pyhd8ed1ab_0 conda-forge
argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py311hd4cff14_3 conda-forge
asttokens 2.2.1 pyhd8ed1ab_0 conda-forge
attrs 22.2.0 pyh71513ae_0 conda-forge
babel 2.11.0 pyhd8ed1ab_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.11.2 pyha770c72_0 conda-forge
bleach 6.0.0 pyhd8ed1ab_0 conda-forge
brotlipy 0.7.0 py311hd4cff14_1005 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
ca-certificates 2022.12.7 ha878542_0 conda-forge
certifi 2022.12.7 pyhd8ed1ab_0 conda-forge
cffi 1.15.1 py311h409f033_3 conda-forge
charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge
comm 0.1.2 pyhd8ed1ab_0 conda-forge
cryptography 39.0.0 py311h9b4c7bb_0 conda-forge
debugpy 1.6.6 py311hcafe171_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
flit-core 3.8.0 pyhd8ed1ab_0 conda-forge
idna 3.4 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.0.0 pyha770c72_0 conda-forge
importlib_metadata 6.0.0 hd8ed1ab_0 conda-forge
importlib_resources 5.10.2 pyhd8ed1ab_0 conda-forge
ipykernel 6.21.1 pyh210e3f2_0 conda-forge
ipython 8.9.0 pyh41d4057_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
jedi 0.18.2 pyhd8ed1ab_0 conda-forge
jinja2 3.1.2 pyhd8ed1ab_1 conda-forge
json5 0.9.5 pyh9f0ad1d_0 conda-forge
jsonschema 4.17.3 pyhd8ed1ab_0 conda-forge
jupyter_client 8.0.2 pyhd8ed1ab_0 conda-forge
jupyter_core 5.2.0 py311h38be061_0 conda-forge
jupyter_events 0.5.0 pyhd8ed1ab_1 conda-forge
jupyter_server 2.2.1 pyhd8ed1ab_0 conda-forge
jupyter_server_fileid 0.6.0 pyhd8ed1ab_0 conda-forge
jupyter_server_terminals 0.4.4 pyhd8ed1ab_1 conda-forge
jupyter_server_ydoc 0.6.1 pyhd8ed1ab_0 conda-forge
jupyter_ydoc 0.2.2 pyhd8ed1ab_0 conda-forge
jupyterlab 3.6.0 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge
jupyterlab_server 2.19.0 pyhd8ed1ab_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 12.2.0 h65d4601_19 conda-forge
libgomp 12.2.0 h65d4601_19 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libsqlite 3.40.0 h753d276_0 conda-forge
libstdcxx-ng 12.2.0 h46fd767_19 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libzlib 1.2.13 h166bdaf_4 conda-forge
markupsafe 2.1.2 py311h2582759_0 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mistune 2.0.4 pyhd8ed1ab_0 conda-forge
nbclassic 0.5.1 pyhd8ed1ab_0 conda-forge
nbclient 0.7.2 pyhd8ed1ab_0 conda-forge
nbconvert 7.2.9 pyhd8ed1ab_0 conda-forge
nbconvert-core 7.2.9 pyhd8ed1ab_0 conda-forge
nbconvert-pandoc 7.2.9 pyhd8ed1ab_0 conda-forge
nbformat 5.7.3 pyhd8ed1ab_0 conda-forge
ncurses 6.3 h27087fc_1 conda-forge
nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge
notebook 6.5.2 pyha770c72_1 conda-forge
notebook-shim 0.2.2 pyhd8ed1ab_0 conda-forge
openssl 3.0.7 h0b41bf4_2 conda-forge
packaging 23.0 pyhd8ed1ab_0 conda-forge
pandoc 2.19.2 h32600fe_1 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pexpect 4.8.0 pyh1a96a4e_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pip 23.0 pyhd8ed1ab_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_0 conda-forge
platformdirs 2.6.2 pyhd8ed1ab_0 conda-forge
prometheus_client 0.16.0 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.36 pyha770c72_0 conda-forge
psutil 5.9.4 py311hd4cff14_0 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.14.0 pyhd8ed1ab_0 conda-forge
pyopenssl 23.0.0 pyhd8ed1ab_0 conda-forge
pyrsistent 0.19.3 py311h2582759_0 conda-forge
pysocks 1.7.1 pyha2e5f31_6 conda-forge
python 3.11.0 he550d4f_1_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.16.2 pyhd8ed1ab_0 conda-forge
python-json-logger 2.0.4 pyhd8ed1ab_0 conda-forge
python_abi 3.11 3_cp311 conda-forge
pytz 2022.7.1 pyhd8ed1ab_0 conda-forge
pyyaml 6.0 py311hd4cff14_5 conda-forge
pyzmq 25.0.0 py311hd6ccaeb_0 conda-forge
readline 8.1.2 h0f457ee_0 conda-forge
requests 2.28.2 pyhd8ed1ab_0 conda-forge
send2trash 1.8.0 pyhd8ed1ab_0 conda-forge
setuptools 67.1.0 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sniffio 1.3.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
terminado 0.17.1 pyh41d4057_0 conda-forge
tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
tornado 6.2 py311hd4cff14_1 conda-forge
traitlets 5.9.0 pyhd8ed1ab_0 conda-forge
typing-extensions 4.4.0 hd8ed1ab_0 conda-forge
typing_extensions 4.4.0 pyha770c72_0 conda-forge
tzdata 2022g h191b570_0 conda-forge
urllib3 1.26.14 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
webencodings 0.5.1 py_1 conda-forge
websocket-client 1.5.0 pyhd8ed1ab_0 conda-forge
wheel 0.38.4 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
y-py 0.5.5 py311hfe55011_0 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
ypy-websocket 0.8.2 pyhd8ed1ab_0 conda-forge
zeromq 4.3.4 h9c3ff4c_1 conda-forge
zipp 3.12.0 pyhd8ed1ab_0 conda-forge

conda env:
name: jlab-test
channels:
- conda-forge
- defaults
dependencies:
- _libgcc_mutex=0.1=conda_forge
- _openmp_mutex=4.5=2_gnu
- aiofiles=22.1.0=pyhd8ed1ab_0
- aiosqlite=0.18.0=pyhd8ed1ab_0
- anyio=3.6.2=pyhd8ed1ab_0
- argon2-cffi=21.3.0=pyhd8ed1ab_0
- argon2-cffi-bindings=21.2.0=py311hd4cff14_3
- asttokens=2.2.1=pyhd8ed1ab_0
- attrs=22.2.0=pyh71513ae_0
- babel=2.11.0=pyhd8ed1ab_0
- backcall=0.2.0=pyh9f0ad1d_0
- backports=1.0=pyhd8ed1ab_3
- backports.functools_lru_cache=1.6.4=pyhd8ed1ab_0
- beautifulsoup4=4.11.2=pyha770c72_0
- bleach=6.0.0=pyhd8ed1ab_0
- brotlipy=0.7.0=py311hd4cff14_1005
- bzip2=1.0.8=h7f98852_4
- ca-certificates=2022.12.7=ha878542_0
- certifi=2022.12.7=pyhd8ed1ab_0
- cffi=1.15.1=py311h409f033_3
- charset-normalizer=2.1.1=pyhd8ed1ab_0
- comm=0.1.2=pyhd8ed1ab_0
- cryptography=39.0.0=py311h9b4c7bb_0
- debugpy=1.6.6=py311hcafe171_0
- decorator=5.1.1=pyhd8ed1ab_0
- defusedxml=0.7.1=pyhd8ed1ab_0
- entrypoints=0.4=pyhd8ed1ab_0
- executing=1.2.0=pyhd8ed1ab_0
- flit-core=3.8.0=pyhd8ed1ab_0
- idna=3.4=pyhd8ed1ab_0
- importlib-metadata=6.0.0=pyha770c72_0
- importlib_metadata=6.0.0=hd8ed1ab_0
- importlib_resources=5.10.2=pyhd8ed1ab_0
- ipykernel=6.21.1=pyh210e3f2_0
- ipython=8.9.0=pyh41d4057_0
- ipython_genutils=0.2.0=py_1
- jedi=0.18.2=pyhd8ed1ab_0
- jinja2=3.1.2=pyhd8ed1ab_1
- json5=0.9.5=pyh9f0ad1d_0
- jsonschema=4.17.3=pyhd8ed1ab_0
- jupyter_client=8.0.2=pyhd8ed1ab_0
- jupyter_core=5.2.0=py311h38be061_0
- jupyter_events=0.5.0=pyhd8ed1ab_1
- jupyter_server=2.2.1=pyhd8ed1ab_0
- jupyter_server_fileid=0.6.0=pyhd8ed1ab_0
- jupyter_server_terminals=0.4.4=pyhd8ed1ab_1
- jupyter_server_ydoc=0.6.1=pyhd8ed1ab_0
- jupyter_ydoc=0.2.2=pyhd8ed1ab_0
- jupyterlab=3.6.0=pyhd8ed1ab_0
- jupyterlab_pygments=0.2.2=pyhd8ed1ab_0
- jupyterlab_server=2.19.0=pyhd8ed1ab_0
- ld_impl_linux-64=2.40=h41732ed_0
- libffi=3.4.2=h7f98852_5
- libgcc-ng=12.2.0=h65d4601_19
- libgomp=12.2.0=h65d4601_19
- libnsl=2.0.0=h7f98852_0
- libsodium=1.0.18=h36c2ea0_1
- libsqlite=3.40.0=h753d276_0
- libstdcxx-ng=12.2.0=h46fd767_19
- libuuid=2.32.1=h7f98852_1000
- libzlib=1.2.13=h166bdaf_4
- markupsafe=2.1.2=py311h2582759_0
- matplotlib-inline=0.1.6=pyhd8ed1ab_0
- mistune=2.0.4=pyhd8ed1ab_0
- nbclassic=0.5.1=pyhd8ed1ab_0
- nbclient=0.7.2=pyhd8ed1ab_0
- nbconvert=7.2.9=pyhd8ed1ab_0
- nbconvert-core=7.2.9=pyhd8ed1ab_0
- nbconvert-pandoc=7.2.9=pyhd8ed1ab_0
- nbformat=5.7.3=pyhd8ed1ab_0
- ncurses=6.3=h27087fc_1
- nest-asyncio=1.5.6=pyhd8ed1ab_0
- notebook=6.5.2=pyha770c72_1
- notebook-shim=0.2.2=pyhd8ed1ab_0
- openssl=3.0.7=h0b41bf4_2
- packaging=23.0=pyhd8ed1ab_0
- pandoc=2.19.2=h32600fe_1
- pandocfilters=1.5.0=pyhd8ed1ab_0
- parso=0.8.3=pyhd8ed1ab_0
- pexpect=4.8.0=pyh1a96a4e_2
- pickleshare=0.7.5=py_1003
- pip=23.0=pyhd8ed1ab_0
- pkgutil-resolve-name=1.3.10=pyhd8ed1ab_0
- platformdirs=2.6.2=pyhd8ed1ab_0
- prometheus_client=0.16.0=pyhd8ed1ab_0
- prompt-toolkit=3.0.36=pyha770c72_0
- psutil=5.9.4=py311hd4cff14_0
- ptyprocess=0.7.0=pyhd3deb0d_0
- pure_eval=0.2.2=pyhd8ed1ab_0
- pycparser=2.21=pyhd8ed1ab_0
- pygments=2.14.0=pyhd8ed1ab_0
- pyopenssl=23.0.0=pyhd8ed1ab_0
- pyrsistent=0.19.3=py311h2582759_0
- pysocks=1.7.1=pyha2e5f31_6
- python=3.11.0=he550d4f_1_cpython
- python-dateutil=2.8.2=pyhd8ed1ab_0
- python-fastjsonschema=2.16.2=pyhd8ed1ab_0
- python-json-logger=2.0.4=pyhd8ed1ab_0
- python_abi=3.11=3_cp311
- pytz=2022.7.1=pyhd8ed1ab_0
- pyyaml=6.0=py311hd4cff14_5
- pyzmq=25.0.0=py311hd6ccaeb_0
- readline=8.1.2=h0f457ee_0
- requests=2.28.2=pyhd8ed1ab_0
- send2trash=1.8.0=pyhd8ed1ab_0
- setuptools=67.1.0=pyhd8ed1ab_0
- six=1.16.0=pyh6c4a22f_0
- sniffio=1.3.0=pyhd8ed1ab_0
- soupsieve=2.3.2.post1=pyhd8ed1ab_0
- stack_data=0.6.2=pyhd8ed1ab_0
- terminado=0.17.1=pyh41d4057_0
- tinycss2=1.2.1=pyhd8ed1ab_0
- tk=8.6.12=h27826a3_0
- tomli=2.0.1=pyhd8ed1ab_0
- tornado=6.2=py311hd4cff14_1
- traitlets=5.9.0=pyhd8ed1ab_0
- typing-extensions=4.4.0=hd8ed1ab_0
- typing_extensions=4.4.0=pyha770c72_0
- tzdata=2022g=h191b570_0
- urllib3=1.26.14=pyhd8ed1ab_0
- wcwidth=0.2.6=pyhd8ed1ab_0
- webencodings=0.5.1=py_1
- websocket-client=1.5.0=pyhd8ed1ab_0
- wheel=0.38.4=pyhd8ed1ab_0
- xz=5.2.6=h166bdaf_0
- y-py=0.5.5=py311hfe55011_0
- yaml=0.2.5=h7f98852_2
- ypy-websocket=0.8.2=pyhd8ed1ab_0
- zeromq=4.3.4=h9c3ff4c_1
- zipp=3.12.0=pyhd8ed1ab_0
prefix: /opt/conda/envs/jlab-test

Command Line Output

# jupyter-lab --ip=0.0.0.0 --port=8888 --no-browser --allow-root --debugcd root/              
[W 2023-02-03 12:12:04.948 ServerApp] Unrecognized alias: 'debugcd', it will have no effect.
[I 2023-02-03 12:12:04.956 ServerApp] Package jupyterlab took 0.0000s to import
[I 2023-02-03 12:12:04.959 ServerApp] Package jupyter_server_fileid took 0.0026s to import
[I 2023-02-03 12:12:04.964 ServerApp] Package jupyter_server_terminals took 0.0045s to import
[I 2023-02-03 12:12:04.980 ServerApp] Package jupyter_server_ydoc took 0.0152s to import
[I 2023-02-03 12:12:04.981 ServerApp] Package nbclassic took 0.0000s to import
[W 2023-02-03 12:12:04.982 ServerApp] A `_jupyter_server_extension_points` function was not found in nbclassic. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-02-03 12:12:04.983 ServerApp] Package notebook_shim took 0.0000s to import
[W 2023-02-03 12:12:04.983 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-02-03 12:12:04.988 ServerApp] jupyter_server_fileid | extension was successfully linked.
[I 2023-02-03 12:12:04.992 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-02-03 12:12:04.996 ServerApp] jupyter_server_ydoc | extension was successfully linked.
[I 2023-02-03 12:12:05.001 ServerApp] jupyterlab | extension was successfully linked.
[I 2023-02-03 12:12:05.005 ServerApp] nbclassic | extension was successfully linked.
[I 2023-02-03 12:12:05.195 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-02-03 12:12:05.319 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-02-03 12:12:05.319 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager
[I 2023-02-03 12:12:05.320 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /root
[I 2023-02-03 12:12:05.320 FileIdExtension] ArbitraryFileIdManager : Configured database path: /root/.local/share/jupyter/file_id_manager.db
[I 2023-02-03 12:12:05.320 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file.
[I 2023-02-03 12:12:05.320 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices.
[I 2023-02-03 12:12:05.321 FileIdExtension] Attached event listeners.
[I 2023-02-03 12:12:05.321 ServerApp] jupyter_server_fileid | extension was successfully loaded.
[I 2023-02-03 12:12:05.322 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2023-02-03 12:12:05.322 ServerApp] jupyter_server_ydoc | extension was successfully loaded.
[I 2023-02-03 12:12:05.323 LabApp] JupyterLab extension loaded from /opt/conda/envs/jlab-test/lib/python3.11/site-packages/jupyterlab
[I 2023-02-03 12:12:05.323 LabApp] JupyterLab application directory is /opt/conda/envs/jlab-test/share/jupyter/lab
[I 2023-02-03 12:12:05.327 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-02-03 12:12:05.330 ServerApp] nbclassic | extension was successfully loaded.
[I 2023-02-03 12:12:05.331 ServerApp] Serving notebooks from local directory: /root
[I 2023-02-03 12:12:05.331 ServerApp] Jupyter Server 2.2.1 is running at:
[I 2023-02-03 12:12:05.331 ServerApp] http://fe43115f9dec:8888/lab?token=207d217beeafe09934d652ccc60ba3e5a6033b404823554a
[I 2023-02-03 12:12:05.331 ServerApp]     http://127.0.0.1:8888/lab?token=207d217beeafe09934d652ccc60ba3e5a6033b404823554a
[I 2023-02-03 12:12:05.331 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2023-02-03 12:12:05.335 ServerApp]
To access the server, open this file in a browser:
    file:///root/.local/share/jupyter/runtime/jpserver-1150-open.html
Or copy and paste one of these URLs:
    http://fe43115f9dec:8888/lab?token=207d217beeafe09934d652ccc60ba3e5a6033b404823554a
    http://127.0.0.1:8888/lab?token=207d217beeafe09934d652ccc60ba3e5a6033b404823554a

[W 2023-02-03 12:12:10.643 ServerApp] 404 GET /api/contents/root?content=1&1675426357835 (10.2.60.137): No such file or directory: root

[W 2023-02-03 12:12:10.643 ServerApp] wrote error: 'No such file or directory: root'

Traceback (most recent call last):

File "/opt/conda/envs/jlab-test/lib/python3.11/site-packages/tornado/web.py", line 1713, in _execute

result = await result

^^^^^^^^^^^^

File "/opt/conda/envs/jlab-test/lib/python3.11/site-packages/jupyter_server/services/contents/handlers.py", line 121, in get

model = await ensure_async(

^^^^^^^^^^^^^^^^^^^

File "/opt/conda/envs/jlab-test/lib/python3.11/site-packages/jupyter_core/utils/init.py", line 182, in ensure_async

result = await obj

^^^^^^^^^

File "/opt/conda/envs/jlab-test/lib/python3.11/site-packages/jupyter_server/services/contents/filemanager.py", line 755, in get

raise web.HTTPError(404, "No such file or directory: %s" % path)

tornado.web.HTTPError: HTTP 404: Not Found (No such file or directory: root)

[W 2023-02-03 12:12:10.645 ServerApp] 404 GET /api/contents/root?content=1&1675426357835 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 3.30ms referer=http://10.33.227.163:8888/lab/workspaces/auto-z/tree/root/Untitled.ipynb

0.00s - Debugger warning: It seems that frozen modules are being used, which may

0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off

0.00s - to python to disable frozen modules.

0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

[W 2023-02-03 12:12:30.653 LabApp] Could not determine jupyterlab build status without nodejs

[I 2023-02-03 12:12:34.830 ServerApp] Creating new notebook in

[I 2023-02-03 12:12:35.927 ServerApp] Kernel started: ca113042-ad4c-45d0-bb61-d0aee266fcaa

0.00s - Debugger warning: It seems that frozen modules are being used, which may

0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off

0.00s - to python to disable frozen modules.

0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

[I 2023-02-03 12:12:36.482 ServerApp] Connecting to kernel ca113042-ad4c-45d0-bb61-d0aee266fcaa.

[I 2023-02-03 12:12:36.857 ServerApp] Connecting to kernel ca113042-ad4c-45d0-bb61-d0aee266fcaa.

[I 2023-02-03 12:12:37.228 ServerApp] Connecting to kernel ca113042-ad4c-45d0-bb61-d0aee266fcaa.

[I 2023-02-03 12:12:44.163 ServerApp] Saving file at /Untitled.ipynb

[W 2023-02-03 12:12:46.210 ServerApp] 404 GET /static/lab/8284.54c936b3303933d2da84.js.map?v=54c936b3303933d2da84 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 33.41ms referer=None

[W 2023-02-03 12:12:46.213 ServerApp] 404 GET /static/lab/4657.8562f70714e36dc33542.js.map?v=8562f70714e36dc33542 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.93ms referer=None

[W 2023-02-03 12:12:46.215 ServerApp] 404 GET /static/lab/4631.96a143e70f005fef7b59.js.map?v=96a143e70f005fef7b59 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.83ms referer=None

[W 2023-02-03 12:12:46.218 ServerApp] 404 GET /static/lab/807.96166378e2efe232d81e.js.map?v=96166378e2efe232d81e (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 2.41ms referer=None

[W 2023-02-03 12:12:46.220 ServerApp] 404 GET /static/lab/6700.c96344a466d1896657bf.js.map?v=c96344a466d1896657bf (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 3.59ms referer=None

[W 2023-02-03 12:12:46.221 ServerApp] 404 GET /static/lab/9473.2e52d2ba788beec303be.js.map?v=2e52d2ba788beec303be (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 4.75ms referer=None

[W 2023-02-03 12:12:46.397 ServerApp] 404 GET /static/lab/1036.a055f7326513fc5a04f7.js.map?v=a055f7326513fc5a04f7 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.02ms referer=None

[W 2023-02-03 12:12:46.401 ServerApp] 404 GET /static/lab/4155.784ca1752696680bf373.js.map?v=784ca1752696680bf373 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.04ms referer=None

[W 2023-02-03 12:12:46.403 ServerApp] 404 GET /static/lab/1033.890eeae47460e241b1e9.js.map?v=890eeae47460e241b1e9 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 0.98ms referer=None

[W 2023-02-03 12:12:46.405 ServerApp] 404 GET /static/lab/4570.53adcb6f69939da383ff.js.map?v=53adcb6f69939da383ff (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 0.92ms referer=None

[W 2023-02-03 12:12:46.584 ServerApp] 404 GET /static/lab/7294.f71c2889fedcd71bd1ee.js.map?v=f71c2889fedcd71bd1ee (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.31ms referer=None

[W 2023-02-03 12:12:46.585 ServerApp] 404 GET /static/lab/3935.4159b022aa6d82e44127.js.map?v=4159b022aa6d82e44127 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 2.15ms referer=None

[W 2023-02-03 12:12:46.587 ServerApp] 404 GET /static/lab/5096.69752c98bd72a17134d0.js.map?v=69752c98bd72a17134d0 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.09ms referer=None

[W 2023-02-03 12:12:46.588 ServerApp] 404 GET /static/lab/1142.074d125bb59f5a332666.js.map?v=074d125bb59f5a332666 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 0.93ms referer=None

[W 2023-02-03 12:12:46.590 ServerApp] 404 GET /static/lab/7755.d506a1d9dadf30b1e490.js.map?v=d506a1d9dadf30b1e490 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 0.95ms referer=None

[W 2023-02-03 12:12:46.591 ServerApp] 404 GET /static/lab/3308.3a94151c0d57440646ab.js.map?v=3a94151c0d57440646ab (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 0.93ms referer=None

[W 2023-02-03 12:12:46.781 ServerApp] 404 GET /static/lab/6080.a73b0910165a6a06f05c.js.map?v=a73b0910165a6a06f05c (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.37ms referer=None

[W 2023-02-03 12:12:46.782 ServerApp] 404 GET /static/lab/1057.11d5284f6d720f77a44d.js.map?v=11d5284f6d720f77a44d (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 2.16ms referer=None

[W 2023-02-03 12:12:46.784 ServerApp] 404 GET /static/lab/3029.fcc4588f76d34cbb8886.js.map?v=fcc4588f76d34cbb8886 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.28ms referer=None

[W 2023-02-03 12:12:46.785 ServerApp] 404 GET /static/lab/1255.6af28fc1253a813ef553.js.map?v=6af28fc1253a813ef553 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 2.08ms referer=None

[W 2023-02-03 12:12:46.786 ServerApp] 404 GET /static/lab/6655.0bf8ef55b16a78705ab9.js.map?v=0bf8ef55b16a78705ab9 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 2.94ms referer=None

[W 2023-02-03 12:12:46.787 ServerApp] 404 GET /static/lab/2970.6f82b3c05eff68cc5f16.js.map?v=6f82b3c05eff68cc5f16 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 3.71ms referer=None

[W 2023-02-03 12:12:46.977 ServerApp] 404 GET /static/lab/4151.e8bd2b5893531345189c.js.map?v=e8bd2b5893531345189c (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.47ms referer=None

[W 2023-02-03 12:12:46.978 ServerApp] 404 GET /static/lab/3496.ecb0e7fcc54191234ae6.js.map?v=ecb0e7fcc54191234ae6 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 2.30ms referer=None

[W 2023-02-03 12:12:46.980 ServerApp] 404 GET /static/lab/4429.c4f083ef6b6e29345fd4.js.map?v=c4f083ef6b6e29345fd4 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.16ms referer=None

[W 2023-02-03 12:12:46.981 ServerApp] 404 GET /static/lab/466.239a0c009f33f1cb7cab.js.map?v=239a0c009f33f1cb7cab (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.97ms referer=None

[W 2023-02-03 12:12:46.982 ServerApp] 404 GET /static/lab/126.1e2fb998804b27c72a3e.js.map?v=1e2fb998804b27c72a3e (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 2.77ms referer=None

[W 2023-02-03 12:12:46.984 ServerApp] 404 GET /static/lab/8524.48062ecfe543b6d76860.js.map?v=48062ecfe543b6d76860 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 0.88ms referer=None

[W 2023-02-03 12:12:47.169 ServerApp] 404 GET /static/lab/911.0c08f040896753efc653.js.map?v=0c08f040896753efc653 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.86ms referer=None

[W 2023-02-03 12:12:47.172 ServerApp] 404 GET /static/lab/1249.047c76b5ea96a41605db.js.map?v=047c76b5ea96a41605db (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.64ms referer=None

[W 2023-02-03 12:12:47.174 ServerApp] 404 GET /static/lab/1358.9ab4f57e8227ce75f427.js.map?v=9ab4f57e8227ce75f427 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 1.65ms referer=None

[W 2023-02-03 12:12:47.177 ServerApp] 404 GET /static/lab/jlab_core.550273ade1d543ef6833.js.map?v=550273ade1d543ef6833 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 2.29ms referer=None

[W 2023-02-03 12:12:47.178 ServerApp] 404 GET /static/lab/7796.f4b9945bc5724417b449.js.map?v=f4b9945bc5724417b449 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 3.09ms referer=None

[W 2023-02-03 12:12:47.179 ServerApp] 404 GET /static/lab/main.fd01f15990abbce283d4.js.map?v=fd01f15990abbce283d4 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 3.61ms referer=None

[W 2023-02-03 12:12:47.362 ServerApp] 404 GET /static/lab/6443.04025a1d063425902d56.js.map?v=04025a1d063425902d56 (e84ac1b193ce48c5ab7f141d125d8c3e@10.2.60.137) 0.92ms referer=None

Traceback (most recent call last):

File "", line 1, in 

File "/opt/conda/envs/jlab-test/lib/python3.11/multiprocessing/spawn.py", line 120, in spawn_main

exitcode = _main(fd, parent_sentinel)

^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/conda/envs/jlab-test/lib/python3.11/multiprocessing/spawn.py", line 129, in _main

prepare(preparation_data)

File "/opt/conda/envs/jlab-test/lib/python3.11/multiprocessing/spawn.py", line 240, in prepare

_fixup_main_from_path(data['init_main_from_path'])

File "/opt/conda/envs/jlab-test/lib/python3.11/multiprocessing/spawn.py", line 291, in _fixup_main_from_path

main_content = runpy.run_path(main_path,

^^^^^^^^^^^^^^^^^^^^^^^^^

File "", line 291, in run_path

File "", line 98, in _run_module_code

File "", line 88, in _run_code

File "/root/Untitled.ipynb", line 5, in 

"execution_count": null,

^^^^

NameError: name 'null' is not defined

^C[I 2023-02-03 12:13:05.791 ServerApp] interrupted

[I 2023-02-03 12:13:05.791 ServerApp] Serving notebooks from local directory: /root

1 active kernel

Jupyter Server 2.2.1 is running at:

http://fe43115f9dec:8888/lab?token=207d217beeafe09934d652ccc60ba3e5a6033b404823554a

http://127.0.0.1:8888/lab?token=207d217beeafe09934d652ccc60ba3e5a6033b404823554a

Shutdown this Jupyter server (y/[n])? y

[C 2023-02-03 12:13:06.540 ServerApp] Shutdown confirmed

[I 2023-02-03 12:13:06.540 ServerApp] Shutting down 6 extensions

[I 2023-02-03 12:13:06.541 ServerApp] Shutting down 1 kernel

[I 2023-02-03 12:13:06.541 ServerApp] Kernel shutdown: ca113042-ad4c-45d0-bb61-d0aee266fcaa

Browser Output

jlab_core.550273ade1d543ef6833.js?v=550273ade1d543ef6833:2 Language pack 'en_US' not installed!
fetch @ jlab_core.550273ade1d543ef6833.js?v=550273ade1d543ef6833:2
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/8284.54c936b3303933d2da84.js.map?v=54c936b3303933d2da84: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/4657.8562f70714e36dc33542.js.map?v=8562f70714e36dc33542: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/4631.96a143e70f005fef7b59.js.map?v=96a143e70f005fef7b59: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/807.96166378e2efe232d81e.js.map?v=96166378e2efe232d81e: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/6700.c96344a466d1896657bf.js.map?v=c96344a466d1896657bf: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/9473.2e52d2ba788beec303be.js.map?v=2e52d2ba788beec303be: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/1036.a055f7326513fc5a04f7.js.map?v=a055f7326513fc5a04f7: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/4155.784ca1752696680bf373.js.map?v=784ca1752696680bf373: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/1033.890eeae47460e241b1e9.js.map?v=890eeae47460e241b1e9: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/4570.53adcb6f69939da383ff.js.map?v=53adcb6f69939da383ff: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/5096.69752c98bd72a17134d0.js.map?v=69752c98bd72a17134d0: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/3935.4159b022aa6d82e44127.js.map?v=4159b022aa6d82e44127: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/1142.074d125bb59f5a332666.js.map?v=074d125bb59f5a332666: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/7755.d506a1d9dadf30b1e490.js.map?v=d506a1d9dadf30b1e490: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/7294.f71c2889fedcd71bd1ee.js.map?v=f71c2889fedcd71bd1ee: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/3308.3a94151c0d57440646ab.js.map?v=3a94151c0d57440646ab: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/6080.a73b0910165a6a06f05c.js.map?v=a73b0910165a6a06f05c: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/1057.11d5284f6d720f77a44d.js.map?v=11d5284f6d720f77a44d: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/2970.6f82b3c05eff68cc5f16.js.map?v=6f82b3c05eff68cc5f16: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/6655.0bf8ef55b16a78705ab9.js.map?v=0bf8ef55b16a78705ab9: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/3029.fcc4588f76d34cbb8886.js.map?v=fcc4588f76d34cbb8886: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/1255.6af28fc1253a813ef553.js.map?v=6af28fc1253a813ef553: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/3496.ecb0e7fcc54191234ae6.js.map?v=ecb0e7fcc54191234ae6: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/4151.e8bd2b5893531345189c.js.map?v=e8bd2b5893531345189c: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/466.239a0c009f33f1cb7cab.js.map?v=239a0c009f33f1cb7cab: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/4429.c4f083ef6b6e29345fd4.js.map?v=c4f083ef6b6e29345fd4: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/126.1e2fb998804b27c72a3e.js.map?v=1e2fb998804b27c72a3e: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/8524.48062ecfe543b6d76860.js.map?v=48062ecfe543b6d76860: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/911.0c08f040896753efc653.js.map?v=0c08f040896753efc653: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/1249.047c76b5ea96a41605db.js.map?v=047c76b5ea96a41605db: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/1358.9ab4f57e8227ce75f427.js.map?v=9ab4f57e8227ce75f427: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/jlab_core.550273ade1d543ef6833.js.map?v=550273ade1d543ef6833: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/7796.f4b9945bc5724417b449.js.map?v=f4b9945bc5724417b449: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/main.fd01f15990abbce283d4.js.map?v=fd01f15990abbce283d4: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://10.33.227.163:8888/static/lab/6443.04025a1d063425902d56.js.map?v=04025a1d063425902d56: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE

The text was updated successfully, but these errors were encountered:

welcome · 2023-02-03T12:30:43Z

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

quasiben · 2023-02-03T13:24:51Z

@jacobtomlinson or @ian-r-rose have you seen errors like this before with jupyter server ?

Temporary solution for jupyter-server/jupyter_server#1198

pentschev · 2023-02-03T19:14:41Z

To add one more data point, it seems that some change in ipykernel=6.21.0 is what triggered this more recently. I can still confirm that either downgrading to jupyter_server=1.18.1 (and keeping ipykernel=6.21.0) OR downgrading to ipykernel=6.20.1 (and keeping jupyter_server=2.2.1) makes the issue from both the minimal reproducer from the original post and the Dask-CUDA cluster launching go away.

krassowski · 2023-02-06T03:20:50Z

This could be related to ipython/ipykernel#1078.

jacobtomlinson · 2023-02-06T14:40:10Z

I just ran through the reproducer with pykernel=6.21.1 and things seem to be fixed now. Probably can go ahead and close this.

blink1073 · 2023-02-06T15:20:30Z

Great, thanks for letting us know!

pentschev · 2023-02-06T16:54:51Z

I can still reproduce with ipykernel=6.21.1:

@jacobtomlinson could you check if you follow the same steps as described in the reproducer, particularly the "save" part? When I was debugging that I noticed that without saving the notebook it wouldn't reproduce, thus why I added that step specifically.

I'm reopening this for now, but please let me know if I'm missing something or if I should provide additional information.

pentschev · 2023-02-06T16:55:36Z

Seems I can't reopen the issue, could you reopen it @blink1073 ?

jacobtomlinson · 2023-02-06T17:02:15Z

Yup sorry @pentschev is right it turns out I can reproduce it still. @blink1073 could you reopen?

I think the difference was clicking the File -> Save rather than just hitting Command+S. Equally I could've just hit the wrong button the first time around.

jingxuanlim · 2023-02-11T07:51:10Z

I'm also running into a similar error when creating a dask distributed client.

Setting processes=False is fine, but when processes=True, the following error is thrown.

I think distributed attempts to restart the worker and this happens repeatedly for a while...

ipykernel                 6.21.1
jupyter_server            2.2.1
jupyterlab_server         2.19.0

blink1073 · 2023-02-11T16:12:20Z

Okay, it looks like there are two problems. I tried the original repro with the latest versions and tried reverting recent commits in ipykernel. With no reversions I get the No such file or directory error.

If I revert ipython/ipykernel#1055 I get the 'null' is not defined error. If I also revert ipython/ipykernel#1078 then the repro works.

It sounds like we need to add a test in ipykernel to catch this problem somehow...

jingxuanlim · 2023-02-12T03:37:24Z

To add one more data point, it seems that some change in ipykernel=6.21.0 is what triggered this more recently. I can still confirm that either downgrading to jupyter_server=1.18.1 (and keeping ipykernel=6.21.0) OR downgrading to ipykernel=6.20.1 (and keeping jupyter_server=2.2.1) makes the issue from both the minimal reproducer from the original post and the Dask-CUDA cluster launching go away.

Can confirm that downgrading jupyter_server to 1.18.1 solves the issue.

At the same time, jupyterlab_server was also downgraded.

The following packages will be DOWNGRADED:

  jupyter_server                         2.2.1-pyhd8ed1ab_0 --> 1.18.1-pyhd8ed1ab_0
  jupyterlab_server                     2.19.0-pyhd8ed1ab_0 --> 2.16.6-pyhd8ed1ab_0

emmanuel-ch · 2023-02-12T09:14:39Z

To add one more data point, it seems that some change in ipykernel=6.21.0 is what triggered this more recently. I can still confirm that either downgrading to jupyter_server=1.18.1 (and keeping ipykernel=6.21.0) OR downgrading to ipykernel=6.20.1 (and keeping jupyter_server=2.2.1) makes the issue from both the minimal reproducer from the original post and the Dask-CUDA cluster launching go away.

Can confirm that downgrading jupyter_server to 1.18.1 solves the issue.

At the same time, jupyterlab_server was also downgraded.
The following packages will be DOWNGRADED:

  jupyter_server                         2.2.1-pyhd8ed1ab_0 --> 1.18.1-pyhd8ed1ab_0
  jupyterlab_server                     2.19.0-pyhd8ed1ab_0 --> 2.16.6-pyhd8ed1ab_0

I confirm downgrading ipykernel<6.21 solved my problem.

The following packages will be DOWNGRADED:

  ipykernel                             6.21.1-pyh025b116_0 --> 6.15.0-pyh025b116_0

krassowski · 2023-02-12T20:46:23Z

CC @Carreau as author of ipython/ipykernel#1078 (see #1198 (comment) and in jupyterlab/jupyterlab#13970 (comment) where we got a confirmation that adding del __file__ works as a workaround).

I wonder if we should use __notebook__ instead to avoid conflicts with multiprocessing internals?

blink1073 · 2023-02-12T22:09:03Z

I'm trying to make a test that fails on Ubuntu in ipython/ipykernel#1094, so far no luck.

blink1073 · 2023-02-13T03:14:29Z

Okay, so ipython/ipykernel#1078 is the only real issue, if I revert that then it works in the docker image. I still don't know how to properly recreate the error from ipykernel's tests though.

blink1073 · 2023-02-13T03:15:44Z

@meeseeksdev please migrate to ipython/ipykernel

lumberbot-app · 2023-02-13T03:15:46Z

I'm afraid I can't do that. Maybe I need to be installed on target repository ?
Click here to do that.

Carreau · 2023-02-13T08:37:33Z

I wonder if we should use __notebook__ instead to avoid conflicts with multiprocessing internals?

I'd like to avoid __notebook__, as if it's used from within vs code or anything else then that will be factually incorrect.

We can maybe take a peak at the filename , and set __notebook__ only when it ends with ipynb ? Otherwise set __file__ ? But let's revert setting file for now.

See jupyter-server/jupyter_server#1198, setting __file__ will break multiprocessing.

pentschev · 2023-02-13T10:48:13Z

Thanks all for the discussion here and @Carreau for the fix, I've manually applied ipython/ipykernel#1095 and tested locally both the minimal multiprocessing reproducer and Dask and both seem to be working fine with that.

blink1073 · 2023-02-13T15:40:41Z

Fixed in https://github.com/ipython/ipykernel/releases/tag/v6.21.2

As per jupyter-server/jupyter_server#1198 ipykernel version 6.21.0, 6.21.1 were broken when running with multiprocessing module. As of 6.21.2 this is now fixed by ipython/ipykernel#1095 .

pentschev added the bug label Feb 3, 2023

pentschev mentioned this issue Feb 3, 2023

[BUG] Starting a LocalCUDACluster causes "no space left on device" error in CUDA 11.8 on ARM and x86 in Jupyterlab rapidsai/dask-cuda#1104

Closed

raydouglass mentioned this issue Feb 3, 2023

Pin two notebook-env packages rapidsai/integration#609

Merged

raydouglass added a commit to raydouglass/integration that referenced this issue Feb 3, 2023

Pin two notebook-env packages

424ada6

Temporary solution for jupyter-server/jupyter_server#1198

blink1073 closed this as completed Feb 6, 2023

blink1073 reopened this Feb 6, 2023

anstadnik mentioned this issue Feb 7, 2023

[CLI]: SystemMonitor thread starting new thread, None as the result wandb/wandb#4903

Closed

wence- mentioned this issue Feb 8, 2023

read_parquet raises an error with config {scheduler : processes} dask/dask#9931

Closed

keewis mentioned this issue Feb 10, 2023

dask.distributed.Client() keeps spawning new workers with FileNotFoundError dask/distributed#7515

Closed

blink1073 mentioned this issue Feb 11, 2023

Add multiprocessing test ipython/ipykernel#1094

Closed

keewis mentioned this issue Feb 11, 2023

Starting a Local cluster in an existing jupyter notebook crashes dask/distributed#7536

Closed

krassowski mentioned this issue Feb 12, 2023

Multiprocessing attempts to run .ipynb on process launch jupyterlab/jupyterlab#13970

Closed

Carreau added a commit to Carreau/ipykernel that referenced this issue Feb 13, 2023

Un-expose __file__ and expose __session__ instead.

ed78ee5

See jupyter-server/jupyter_server#1198, setting __file__ will break multiprocessing.

Carreau mentioned this issue Feb 13, 2023

Un-expose __file__ and expose __session__ instead. ipython/ipykernel#1095

Merged

blink1073 closed this as completed Feb 13, 2023

pentschev mentioned this issue Feb 13, 2023

Update ipykernel pin to reject broken multiprocessing releases rapidsai/integration#623

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exceptions when spawning process(es) with `multiprocessing` #1198

Exceptions when spawning process(es) with `multiprocessing` #1198

pentschev commented Feb 3, 2023

welcome bot commented Feb 3, 2023

quasiben commented Feb 3, 2023 •

edited

Loading

pentschev commented Feb 3, 2023

krassowski commented Feb 6, 2023

jacobtomlinson commented Feb 6, 2023

blink1073 commented Feb 6, 2023

pentschev commented Feb 6, 2023

pentschev commented Feb 6, 2023

jacobtomlinson commented Feb 6, 2023

jingxuanlim commented Feb 11, 2023 •

edited

Loading

blink1073 commented Feb 11, 2023

jingxuanlim commented Feb 12, 2023 •

edited

Loading

emmanuel-ch commented Feb 12, 2023

krassowski commented Feb 12, 2023

blink1073 commented Feb 12, 2023

blink1073 commented Feb 13, 2023

blink1073 commented Feb 13, 2023

lumberbot-app bot commented Feb 13, 2023

Carreau commented Feb 13, 2023

pentschev commented Feb 13, 2023

blink1073 commented Feb 13, 2023

Exceptions when spawning process(es) with multiprocessing #1198

Exceptions when spawning process(es) with multiprocessing #1198

Comments

pentschev commented Feb 3, 2023

Description

Reproduce

Expected behavior

Context

welcome bot commented Feb 3, 2023

quasiben commented Feb 3, 2023 • edited Loading

pentschev commented Feb 3, 2023

krassowski commented Feb 6, 2023

jacobtomlinson commented Feb 6, 2023

blink1073 commented Feb 6, 2023

pentschev commented Feb 6, 2023

pentschev commented Feb 6, 2023

jacobtomlinson commented Feb 6, 2023

jingxuanlim commented Feb 11, 2023 • edited Loading

blink1073 commented Feb 11, 2023

jingxuanlim commented Feb 12, 2023 • edited Loading

emmanuel-ch commented Feb 12, 2023

krassowski commented Feb 12, 2023

blink1073 commented Feb 12, 2023

blink1073 commented Feb 13, 2023

blink1073 commented Feb 13, 2023

lumberbot-app bot commented Feb 13, 2023

Carreau commented Feb 13, 2023

pentschev commented Feb 13, 2023

blink1073 commented Feb 13, 2023

Exceptions when spawning process(es) with `multiprocessing` #1198

Exceptions when spawning process(es) with `multiprocessing` #1198

quasiben commented Feb 3, 2023 •

edited

Loading

jingxuanlim commented Feb 11, 2023 •

edited

Loading

jingxuanlim commented Feb 12, 2023 •

edited

Loading