Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ResourceWarnings when running in Python Development Mode #2158

Closed
domdfcoding opened this issue Jul 27, 2021 · 3 comments
Closed

ResourceWarnings when running in Python Development Mode #2158

domdfcoding opened this issue Jul 27, 2021 · 3 comments
Labels
bug help-wanted

Comments

@domdfcoding
Copy link

Issue

When running with PYTHONDEVMODE=1 and PYTHONTRACEMALLOC=1 a number of ResourceWarnings are emitted:

/usr/lib/python3.8/subprocess.py:946: ResourceWarning: subprocess 20 is still running
  _warn("subprocess %s is still running" % self.pid,
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/periodic_update.py", lineno 193
    process = Popen(cmd, **kwargs)
/usr/lib/python3.8/subprocess.py:946: ResourceWarning: subprocess 18 is still running
  _warn("subprocess %s is still running" % self.pid,
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/periodic_update.py", lineno 193
    process = Popen(cmd, **kwargs)
/usr/lib/python3.8/subprocess.py:946: ResourceWarning: subprocess 19 is still running
  _warn("subprocess %s is still running" % self.pid,
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/periodic_update.py", lineno 193
    process = Popen(cmd, **kwargs)
created virtual environment CPython3.8.10.final.0-64 in 3890ms
  creator CPython3Posix(dest=/buildbot/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/buildbot/.local/share/virtualenv)
    added seed packages: pip==21.1.3, setuptools==57.1.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=6>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=5>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=11>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=15>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=3>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=9>

Environment

Provide at least:

  • OS: Ubuntu 20.04.2 LTS, aarch64
  • pip list of the host python where virtualenv is installed:
Package                           Version             
--------------------------------- --------------------
attrs                             21.2.0              
Automat                           20.2.0              
backports.entry-points-selectable 1.1.0               
buildbot-worker                   2021.7.27           
certifi                           2019.11.28          
cffi                              1.14.6              
chardet                           3.0.4               
constantly                        15.1.0              
cryptography                      3.4.7               
dbus-python                       1.2.16              
distlib                           0.3.2               
distro-info                       0.23ubuntu1         
filelock                          3.0.12              
future                            0.18.2              
hyperlink                         21.0.0              
idna                              3.2                 
incremental                       21.3.0              
pip                               20.0.2              
platformdirs                      2.1.0               
pyasn1                            0.4.8               
pyasn1-modules                    0.2.8               
pycparser                         2.20                
PyGObject                         3.36.0              
pyOpenSSL                         20.0.1              
python-apt                        2.0.0+ubuntu0.20.4.5
requests                          2.22.0              
requests-unixsocket               0.2.0               
service-identity                  21.1.0              
setuptools                        45.2.0              
six                               1.16.0              
Twisted                           21.2.0              
unattended-upgrades               0.1                 
urllib3                           1.25.8              
virtualenv                        20.6.0              
wheel                             0.34.2              
zope.interface                    5.4.0

I can't reproduce this on AMD64, so this might be aarch64 specific.

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

$ PYTHONDEVMODE=1 PYTHONTRACEMALLOC=1 python3 -m virtualenv venv -vvv --with-traceback
599 setup logging to NOTSET [DEBUG report:43]
602 created app data folder /home/buildbot/.local/share/virtualenv [DEBUG __init__:41]
824 find interpreter for spec PythonSpec(path=/usr/bin/python3) [INFO builtin:62]
825 proposed PythonInfo(spec=CPython3.8.10.final.0-64, exe=/usr/bin/python3, platform=linux, version='3.8.10 (default, Jun  2 2021, 10:49:15) \n[GCC 9.4.0]', encoding_fs_io=utf-8-utf-8) [INFO builtin:69]
827 accepted PythonInfo(spec=CPython3.8.10.final.0-64, exe=/usr/bin/python3, platform=linux, version='3.8.10 (default, Jun  2 2021, 10:49:15) \n[GCC 9.4.0]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:71]
839 filesystem is case-sensitive [DEBUG info:28]
1080 create virtual environment via CPython3Posix(dest=/buildbot/venv, clear=False, no_vcs_ignore=False, global=False) [INFO session:52]
1082 create folder /buildbot/venv/bin [DEBUG _sync:25]
1084 create folder /buildbot/venv/lib/python3.8/site-packages [DEBUG _sync:25]
1085 write /buildbot/venv/pyvenv.cfg [DEBUG pyenv_cfg:34]
1085 	home = /usr [DEBUG pyenv_cfg:38]
1086 	implementation = CPython [DEBUG pyenv_cfg:38]
1086 	version_info = 3.8.10.final.0 [DEBUG pyenv_cfg:38]
1086 	virtualenv = 20.6.0 [DEBUG pyenv_cfg:38]
1086 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
1087 	base-prefix = /usr [DEBUG pyenv_cfg:38]
1087 	base-exec-prefix = /usr [DEBUG pyenv_cfg:38]
1088 	base-executable = /usr/bin/python3 [DEBUG pyenv_cfg:38]
1089 symlink /usr/bin/python3 to /buildbot/venv/bin/python [DEBUG _sync:44]
1092 create virtualenv import hook file /buildbot/venv/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:95]
1094 create /buildbot/venv/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:98]
1096 ============================== target debug ============================== [DEBUG session:54]
1098 debug via /buildbot/venv/bin/python /usr/local/lib/python3.8/dist-packages/virtualenv/create/debug.py [DEBUG creator:224]
1097 {
  "sys": {
    "executable": "/buildbot/venv/bin/python",
    "_base_executable": "/buildbot/venv/bin/python",
    "prefix": "/buildbot/venv",
    "base_prefix": "/usr",
    "real_prefix": null,
    "exec_prefix": "/buildbot/venv",
    "base_exec_prefix": "/usr",
    "path": [
      "/usr/lib/python38.zip",
      "/usr/lib/python3.8",
      "/usr/lib/python3.8/lib-dynload",
      "/buildbot/venv/lib/python3.8/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.8.10 (default, Jun  2 2021, 10:49:15) \n[GCC 9.4.0]",
  "makefile_filename": "/usr/lib/python3.8/config-3.8-aarch64-linux-gnu/Makefile",
  "os": "<module 'os' from '/usr/lib/python3.8/os.py'>",
  "site": "<module 'site' from '/usr/lib/python3.8/site.py'>",
  "datetime": "<module 'datetime' from '/usr/lib/python3.8/datetime.py'>",
  "math": "<module 'math' (built-in)>",
  "json": "<module 'json' from '/usr/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
1325 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/buildbot/.local/share/virtualenv) [INFO session:59]
1339 wrote embed update of distribution setuptools at /home/buildbot/.local/share/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:161]
1341 wrote embed update of distribution wheel at /home/buildbot/.local/share/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:161]
1343 wrote embed update of distribution pip at /home/buildbot/.local/share/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:161]
1351 triggered periodic upgrade of pip==21.1.3 (for python 3.8) via background process having PID 18 [INFO periodic_update:194]
1360 triggered periodic upgrade of wheel==0.36.2 (for python 3.8) via background process having PID 19 [INFO periodic_update:194]
1370 triggered periodic upgrade of setuptools==57.1.0 (for python 3.8) via background process having PID 20 [INFO periodic_update:194]
/usr/lib/python3.8/subprocess.py:946: ResourceWarning: subprocess 20 is still running
  _warn("subprocess %s is still running" % self.pid,
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/periodic_update.py", lineno 193
    process = Popen(cmd, **kwargs)
/usr/lib/python3.8/subprocess.py:946: ResourceWarning: subprocess 18 is still running
  _warn("subprocess %s is still running" % self.pid,
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/periodic_update.py", lineno 193
    process = Popen(cmd, **kwargs)
/usr/lib/python3.8/subprocess.py:946: ResourceWarning: subprocess 19 is still running
  _warn("subprocess %s is still running" % self.pid,
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/periodic_update.py", lineno 193
    process = Popen(cmd, **kwargs)
1413 got embed update of distribution setuptools from /home/buildbot/.local/share/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:135]
1417 got embed update of distribution wheel from /home/buildbot/.local/share/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:135]
1430 got embed update of distribution pip from /home/buildbot/.local/share/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:135]
1455 install wheel from wheel /usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/embed/wheel-0.36.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
1467 Attempting to acquire lock 281473597575808 on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any.lock [DEBUG filelock:270]
1468 install pip from wheel /usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/embed/pip-21.1.3-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
1470 Lock 281473597575808 acquired on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any.lock [INFO filelock:274]
1472 Attempting to acquire lock 281473597578608 on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-21.1.3-py3-none-any.lock [DEBUG filelock:270]
1481 build install image for wheel-0.36.2-py2.py3-none-any.whl to /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any [DEBUG base:52]
1481 install setuptools from wheel /usr/local/lib/python3.8/dist-packages/virtualenv/seed/wheels/embed/setuptools-57.1.0-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
1482 Lock 281473597578608 acquired on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-21.1.3-py3-none-any.lock [INFO filelock:274]
1486 build install image for pip-21.1.3-py3-none-any.whl to /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-21.1.3-py3-none-any [DEBUG base:52]
1491 Attempting to acquire lock 281473597582864 on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any.lock [DEBUG filelock:270]
1495 Lock 281473597582864 acquired on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any.lock [INFO filelock:274]
1496 build install image for setuptools-57.1.0-py3-none-any.whl to /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any [DEBUG base:52]
1771 Attempting to release lock 281473597575808 on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any.lock [DEBUG filelock:315]
1792 Lock 281473597575808 released on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any.lock [INFO filelock:318]
1800 copy directory /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any/wheel to /buildbot/venv/lib/python3.8/site-packages/wheel [DEBUG _sync:52]
1877 copy /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any/wheel-0.36.2.virtualenv to /buildbot/venv/lib/python3.8/site-packages/wheel-0.36.2.virtualenv [DEBUG _sync:52]
1898 copy directory /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.36.2-py2.py3-none-any/wheel-0.36.2.dist-info to /buildbot/venv/lib/python3.8/site-packages/wheel-0.36.2.dist-info [DEBUG _sync:52]
1946 generated console scripts wheel wheel-3.8 wheel3 wheel3.8 [DEBUG base:48]
2540 Attempting to release lock 281473597582864 on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any.lock [DEBUG filelock:315]
2542 Lock 281473597582864 released on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any.lock [INFO filelock:318]
2544 copy directory /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any/setuptools to /buildbot/venv/lib/python3.8/site-packages/setuptools [DEBUG _sync:52]
3129 copy /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any/distutils-precedence.pth to /buildbot/venv/lib/python3.8/site-packages/distutils-precedence.pth [DEBUG _sync:52]
3146 copy /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any/setuptools-57.1.0.virtualenv to /buildbot/venv/lib/python3.8/site-packages/setuptools-57.1.0.virtualenv [DEBUG _sync:52]
3149 copy directory /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any/pkg_resources to /buildbot/venv/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:52]
3255 copy directory /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any/_distutils_hack to /buildbot/venv/lib/python3.8/site-packages/_distutils_hack [DEBUG _sync:52]
3261 copy directory /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-57.1.0-py3-none-any/setuptools-57.1.0.dist-info to /buildbot/venv/lib/python3.8/site-packages/setuptools-57.1.0.dist-info [DEBUG _sync:52]
3298 generated console scripts  [DEBUG base:48]
3618 Attempting to release lock 281473597578608 on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-21.1.3-py3-none-any.lock [DEBUG filelock:315]
3621 Lock 281473597578608 released on /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-21.1.3-py3-none-any.lock [INFO filelock:318]
3624 copy directory /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-21.1.3-py3-none-any/pip-21.1.3.dist-info to /buildbot/venv/lib/python3.8/site-packages/pip-21.1.3.dist-info [DEBUG _sync:52]
3640 copy /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-21.1.3-py3-none-any/pip-21.1.3.virtualenv to /buildbot/venv/lib/python3.8/site-packages/pip-21.1.3.virtualenv [DEBUG _sync:52]
3646 copy directory /home/buildbot/.local/share/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-21.1.3-py3-none-any/pip to /buildbot/venv/lib/python3.8/site-packages/pip [DEBUG _sync:52]
4922 generated console scripts pip3 pip pip3.8 pip-3.8 [DEBUG base:48]
4937 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
4952 write /buildbot/venv/pyvenv.cfg [DEBUG pyenv_cfg:34]
4954 	home = /usr [DEBUG pyenv_cfg:38]
4958 	implementation = CPython [DEBUG pyenv_cfg:38]
4962 	version_info = 3.8.10.final.0 [DEBUG pyenv_cfg:38]
4968 	virtualenv = 20.6.0 [DEBUG pyenv_cfg:38]
4972 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
4977 	base-prefix = /usr [DEBUG pyenv_cfg:38]
4982 	base-exec-prefix = /usr [DEBUG pyenv_cfg:38]
4986 	base-executable = /usr/bin/python3 [DEBUG pyenv_cfg:38]
4993 created virtual environment CPython3.8.10.final.0-64 in 4405ms
  creator CPython3Posix(dest=/buildbot/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/buildbot/.local/share/virtualenv)
    added seed packages: pip==21.1.3, setuptools==57.1.0, wheel==0.36.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:19]
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=13>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=15>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=8>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=5>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=3>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=4>
@gaborbernat
Copy link
Contributor

A PR fixing this would be welcomed 👍

@gaborbernat
Copy link
Contributor

As this is 2 years old without updates closing it. If still valid we can reopen.

@sigma67
Copy link
Contributor

sigma67 commented Aug 4, 2023

This is the same issue as #2472

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug help-wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gaborbernat @domdfcoding @sigma67