No space left on device - Unable to make pip running within tox to point to another temporary folder

[b1quint]

Dear tox folks,

I am using Tox in a Jenkins CI server with several partitions, and I am facing the issue described pypa/pip#5816. The problem is that, since I am using Tox to run my tests, I need to apply the solutions described in that Issue via Tox.

My tox.ini file is running with isolated_build = true, I set $TMPDIR to point to a local directory inside a large partition, and I am overriding the install_command with python -m pip install --cache-dir=.pip_cache/ {opts} {packages}.

However, the (shortened) traceback below shows that the pip command is still using my /tmp folder somehow. Would you have any other ideas or suggestions of how I could force pip to use the other partition via tox?

[2021-04-27T18:57:18.491Z] [424911] /rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5$ /rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5/.tox/py37-gmosls/bin/python -m pip install --cache-dir=.pip_cache/ --exists-action w '/rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5/.tox/.tmp/package/1/dragons-3.1.0.dev0.tar.gz[test]' >.tox/py37-gmosls/log/py37-gmosls-4.log
[2021-04-27T18:58:25.998Z] ERROR: invocation failed (exit code 1), logfile: /rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5/.tox/py37-gmosls/log/py37-gmosls-4.log
[2021-04-27T18:58:25.998Z] ================================== log start ===================================
[2021-04-27T18:58:25.998Z] Processing ./.tox/.tmp/package/1/dragons-3.1.0.dev0.tar.gz
[2021-04-27T18:58:25.998Z]   Installing build dependencies: started
[2021-04-27T18:58:25.998Z]   Installing build dependencies: finished with status 'done'
[2021-04-27T18:58:25.998Z]   Getting requirements to build wheel: started
[2021-04-27T18:58:25.998Z]   Getting requirements to build wheel: finished with status 'done'
[2021-04-27T18:58:25.998Z]     Preparing wheel metadata: started
[2021-04-27T18:58:25.998Z]     Preparing wheel metadata: finished with status 'done'
[2021-04-27T18:58:25.998Z] Requirement already satisfied: sqlalchemy in ./.tox/py37-gmosls/lib/python3.7/site-packages (from dragons==3.1.0.dev0) (1.4.7)
[2021-04-27T18:58:25.998Z] Requirement already satisfied: astropy>=4.2 in ./.tox/py37-gmosls/lib/python3.7/site-packages (from dragons==3.1.0.dev0) (4.2.1)

... 

error: could not write to 'build/bdist.linux-x86_64/wheel/geminidr/gmos/lookups/BPM/gmos-n_bpm_HAM_12_full_12amp_v1.fits': No space left on device
[2021-04-27T18:26:43.807Z]   ----------------------------------------
[2021-04-27T18:26:43.807Z]   ERROR: Failed building wheel for dragons
[2021-04-27T18:26:43.807Z]   Building wheel for astroscrappy (setup.py): started
[2021-04-27T18:26:43.807Z]   Building wheel for astroscrappy (setup.py): finished with status 'done'
[2021-04-27T18:26:43.807Z]   Created wheel for astroscrappy: filename=astroscrappy-1.0.8-cp37-cp37m-linux_x86_64.whl size=1135176 sha256=a6cf22e7e4f5eabe962f0adf63512dc2128f047771e95aefafc3b12f01cf64e3
[2021-04-27T18:26:43.807Z]   Stored in directory: /rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5/.pip_cache/wheels/50/45/57/80103bcbdc1e72e8c43e58cb10020646654264950f3576e84c
[2021-04-27T18:26:43.807Z]   Building wheel for bokeh (setup.py): started
[2021-04-27T18:26:43.807Z]   Building wheel for bokeh (setup.py): finished with status 'error'
[2021-04-27T18:26:43.807Z]   ERROR: Command errored out with exit status 1:
[2021-04-27T18:26:43.807Z]    command: /rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5/.tox/py37-gmosls/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8yqeo2c3/bokeh_673c1021360d41438a92e10fe00492c5/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8yqeo2c3/bokeh_673c1021360d41438a92e10fe00492c5/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-fnumf7rg
[2021-04-27T18:26:43.807Z]        cwd: /tmp/pip-install-8yqeo2c3/bokeh_673c1021360d41438a92e10fe00492c5/
[2021-04-27T18:26:43.807Z]   Complete output (5 lines):
[2021-04-27T18:26:43.807Z]   running bdist_wheel
[2021-04-27T18:26:43.807Z]   running build
[2021-04-27T18:26:43.807Z]   running build_py
[2021-04-27T18:26:43.807Z]   creating build
[2021-04-27T18:26:43.807Z]   error: could not create 'build': No space left on device
[2021-04-27T18:26:43.807Z]   ----------------------------------------
[2021-04-27T18:26:43.807Z]   ERROR: Failed building wheel for bokeh
[2021-04-27T18:26:43.807Z]   Running setup.py clean for bokeh
[2021-04-27T18:26:43.807Z]   Building wheel for bottleneck (PEP 517): started
[2021-04-27T18:26:43.807Z]   Building wheel for bottleneck (PEP 517): finished with status 'done'
[2021-04-27T18:26:43.807Z]   Created wheel for bottleneck: filename=Bottleneck-1.3.2-cp37-cp37m-linux_x86_64.whl size=319652 sha256=ce7bc1c8f83964ed90e7be9485bf2ccf2a3a893adc0fb74bf1deb4d91ab88f21
[2021-04-27T18:26:43.807Z]   Stored in directory: /rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5/.pip_cache/wheels/87/85/9c/a325c89ff0498660ef8a335fb4b3912939c273ea4f094af29f
[2021-04-27T18:26:43.807Z] Successfully built astroscrappy bottleneck
[2021-04-27T18:26:43.807Z] Failed to build dragons bokeh
[2021-04-27T18:26:43.807Z] ERROR: Could not build wheels for dragons which use PEP 517 and cannot be installed directly
[2021-04-27T18:26:43.807Z] 
[2021-04-27T18:26:43.807Z] =================================== log end ====================================
[2021-04-27T18:26:43.807Z] ___________________________________ summary ____________________________________
[2021-04-27T18:26:43.807Z] ERROR:   py37-gmosls: InvocationError for command /rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5/.tox/py37-gmosls/bin/python -m pip install --cache-dir=.pip_cache/ --exists-action w '/rtfproc/jenkins/workspace/dragons_py3_tox_temp_folder@5/.tox/.tmp/package/1/dragons-3.1.0.dev0.tar.gz[test]' (exited with code 1)

[b1quint]

It seems that Tox has nothing to do with this issue.

From what I understand, PIP is deprecating the use of -b/--build command-line option in favor of setting the temporary folders via $TMPDIR et al. This issue is expected to be fixed on PIP 21.3. The same issue has been reported in pypa/pip#8333 in this comment.

[jugmac00]

Thanks for the heads up! ❤️