Do not download all wheels of pip when the release history cannot be acquired #1883
Labels
Comments
|
This is expected, see https://virtualenv.pypa.io/en/latest/user_guide.html#wheels. though it shouldn't use much CPU, can you post their logs? 🤔 |
|
Where would I look for the relevant logs? I can't find anything under ~/Library/Logs or ~/Library/Application Support/virtualenv (or .../pip). If it's of any use: it just happened to me again, and this time I let the processes run for a short while. Here are the relevant lines from Interestingly, if I wait for a few seconds and run And after a while, the processes even finish naturally: It looks like for some reason virtualenv is downloading every available version of pip, setuptools and wheel, by calling |
|
That's the expected behaviour here. The only question remain to be seen is that why it keeps retrying all the time and not just once every two weeks. Can you post the content of For what it's worth you can disable this behaviour by using the https://virtualenv.pypa.io/en/latest/cli_interface.html#no-periodic-update flag, that can be also enabled globally by setting Thanks! |
|
pip.json just contains this: {
"completed": null,
"periodic": true,
"started": "2020-07-03T16:10:39.723035Z",
"versions": []
}I am able to access https://pypi.org/pypi/pip/json, both in the browser and from Python using
To clarify - is it expected that virtualenv downloads and keeps wheels for every version of pip/setuptools/wheel that was ever released? The total size of all of those wheels is about 250 MB, which is not huge, but I don't think virtualenv should use that much bandwidth/disk space without asking. I'm not sure what the old versions are useful for either - shouldn't you only need the latest version that supports the environment's Python version? |
|
That's a problem, seems we start acquiring the information but never finish. If we'd do the versions would not be empty and completed would have a value. Can you delete that pip.json file and then run: This should now put the update logs on the output, and do the update inline rather than background process.
We go back in history until we find one that has been released 28 days ago, or previous minor and is compatible with the current Python version... So should not take all, just the last few versions. |
|
I didn't even have to delete the pip.json, the periodic update ran again automatically for all three packages. Here's the output: Output$ env _VIRTUALENV_PERIODIC_UPDATE_INLINE=1 virtualenv --clear env --with-traceback -vvv
144 setup logging to NOTSET [DEBUG report:42]
177 find interpreter for spec PythonSpec(path=/Library/Frameworks/Python.framework/Versions/3.8/bin/python3) [INFO builtin:44]
178 proposed PythonInfo(spec=CPython3.8.3.final.0-64, exe=/Library/Frameworks/Python.framework/Versions/3.8/bin/python3, platform=darwin, version='3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) \n[Clang 6.0 (clang-600.0.57)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:50]
178 accepted PythonInfo(spec=CPython3.8.3.final.0-64, exe=/Library/Frameworks/Python.framework/Versions/3.8/bin/python3, platform=darwin, version='3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) \n[Clang 6.0 (clang-600.0.57)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:52]
182 filesystem is not case-sensitive [DEBUG info:28]
333 create virtual environment via CPython3Posix(dest=/Users/david/Downloads/work/env, clear=True, global=False) [INFO session:52]
333 create folder /Users/david/Downloads/work/env/bin [DEBUG _sync:25]
334 create folder /Users/david/Downloads/work/env/lib/python3.8/site-packages [DEBUG _sync:25]
334 write /Users/david/Downloads/work/env/pyvenv.cfg [DEBUG pyenv_cfg:34]
334 home = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
335 implementation = CPython [DEBUG pyenv_cfg:38]
335 version_info = 3.8.3.final.0 [DEBUG pyenv_cfg:38]
335 virtualenv = 20.0.25 [DEBUG pyenv_cfg:38]
335 include-system-site-packages = false [DEBUG pyenv_cfg:38]
335 base-prefix = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
335 base-exec-prefix = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
335 base-executable = /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 [DEBUG pyenv_cfg:38]
336 symlink /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 to /Users/david/Downloads/work/env/bin/python [DEBUG _sync:44]
337 create virtualenv import hook file /Users/david/Downloads/work/env/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:94]
337 create /Users/david/Downloads/work/env/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:97]
338 ============================== target debug ============================== [DEBUG session:54]
338 debug via /Users/david/Downloads/work/env/bin/python /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:209]
338 {
"sys": {
"executable": "/Users/david/Downloads/work/env/bin/python",
"_base_executable": "/Users/david/Downloads/work/env/bin/python",
"prefix": "/Users/david/Downloads/work/env",
"base_prefix": "/Library/Frameworks/Python.framework/Versions/3.8",
"real_prefix": null,
"exec_prefix": "/Users/david/Downloads/work/env",
"base_exec_prefix": "/Library/Frameworks/Python.framework/Versions/3.8",
"path": [
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip",
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8",
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload",
"/Users/david/Downloads/work/env/lib/python3.8/site-packages"
],
"meta_path": [
"<class '_virtualenv._Finder'>",
"<class '_frozen_importlib.BuiltinImporter'>",
"<class '_frozen_importlib.FrozenImporter'>",
"<class '_frozen_importlib_external.PathFinder'>"
],
"fs_encoding": "utf-8",
"io_encoding": "utf-8"
},
"version": "3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) \n[Clang 6.0 (clang-600.0.57)]",
"makefile_filename": "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/config-3.8-darwin/Makefile",
"os": "<module 'os' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py'>",
"site": "<module 'site' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site.py'>",
"datetime": "<module 'datetime' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/datetime.py'>",
"math": "<module 'math' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/math.cpython-38-darwin.so'>",
"json": "<module 'json' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
401 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/david/Library/Application Support/virtualenv) [INFO session:59]
406 got embed update of distribution setuptools from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:133]
407 got embed update of distribution wheel from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:133]
407 got embed update of distribution pip from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:133]
411 wrote embed update of distribution setuptools at /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:159]
413 wrote embed update of distribution wheel at /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:159]
414 wrote embed update of distribution pip at /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:159]
414 triggered periodic upgrade of setuptools==47.3.1 (for python 3.8) via background process having PID 59590 [INFO periodic_update:182]
417 triggered periodic upgrade of wheel==0.34.2 (for python 3.8) via background process having PID 59591 [INFO periodic_update:182]
422 triggered periodic upgrade of pip==20.1.1 (for python 3.8) via background process having PID 59592 [INFO periodic_update:182]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/periodic_update.py", line 214, in do_update
dest = acquire.download_wheel(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/acquire.py", line 67, in download_wheel
raise subprocess.CalledProcessError(process.returncode, cmd, **kwargs)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3', '-m', 'pip', 'download', '--disable-pip-version-check', '--only-binary=:all:', '--no-deps', '--python-version', '3.8', '-d', '/Users/david/Library/Application Support/virtualenv/wheel/house', 'wheel<0.9']' returned non-zero exit status 1.
155299 got embed update of distribution wheel from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:133]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/periodic_update.py", line 214, in do_update
dest = acquire.download_wheel(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/acquire.py", line 67, in download_wheel
raise subprocess.CalledProcessError(process.returncode, cmd, **kwargs)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3', '-m', 'pip', 'download', '--disable-pip-version-check', '--only-binary=:all:', '--no-deps', '--python-version', '3.8', '-d', '/Users/david/Library/Application Support/virtualenv/wheel/house', 'pip<1.5.1']' returned non-zero exit status 1.
216355 got embed update of distribution pip from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:133]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/periodic_update.py", line 214, in do_update
dest = acquire.download_wheel(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/acquire.py", line 67, in download_wheel
raise subprocess.CalledProcessError(process.returncode, cmd, **kwargs)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3', '-m', 'pip', 'download', '--disable-pip-version-check', '--only-binary=:all:', '--no-deps', '--python-version', '3.8', '-d', '/Users/david/Library/Application Support/virtualenv/wheel/house', 'setuptools<0.9.8']' returned non-zero exit status 1.
1184058 got embed update of distribution setuptools from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:133]
1184058 install wheel from wheel /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/wheel-0.34.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:53]
1184059 Attempting to acquire lock 4476672128 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any.lock [DEBUG filelock:270]
1184059 install pip from wheel /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/pip-20.1.1-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:53]
1184059 Lock 4476672128 acquired on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any.lock [INFO filelock:274]
1184060 Attempting to acquire lock 4476671696 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any.lock [DEBUG filelock:270]
1184060 install setuptools from wheel /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/setuptools-47.3.1-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:53]
1184060 Lock 4476671696 acquired on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any.lock [INFO filelock:274]
1184061 Attempting to acquire lock 4476671360 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any.lock [DEBUG filelock:270]
1184061 Attempting to release lock 4476672128 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any.lock [DEBUG filelock:315]
1184061 Attempting to release lock 4476671696 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any.lock [DEBUG filelock:315]
1184061 Lock 4476671360 acquired on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any.lock [INFO filelock:274]
1184061 Lock 4476672128 released on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any.lock [INFO filelock:318]
1184061 Lock 4476671696 released on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any.lock [INFO filelock:318]
1184062 Attempting to release lock 4476671360 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any.lock [DEBUG filelock:315]
1184062 Lock 4476671360 released on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any.lock [INFO filelock:318]
1184063 copy /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/easy_install.py to /Users/david/Downloads/work/env/lib/python3.8/site-packages/easy_install.py [DEBUG _sync:52]
1184065 copy /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.virtualenv to /Users/david/Downloads/work/env/lib/python3.8/site-packages/wheel-0.34.2.virtualenv [DEBUG _sync:52]
1184066 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any/pip to /Users/david/Downloads/work/env/lib/python3.8/site-packages/pip [DEBUG _sync:52]
1184067 copy /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/setuptools-47.3.1.virtualenv to /Users/david/Downloads/work/env/lib/python3.8/site-packages/setuptools-47.3.1.virtualenv [DEBUG _sync:52]
1184068 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel to /Users/david/Downloads/work/env/lib/python3.8/site-packages/wheel [DEBUG _sync:52]
1184071 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/setuptools to /Users/david/Downloads/work/env/lib/python3.8/site-packages/setuptools [DEBUG _sync:52]
1184093 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.dist-info to /Users/david/Downloads/work/env/lib/python3.8/site-packages/wheel-0.34.2.dist-info [DEBUG _sync:52]
1184137 generated console scripts wheel wheel3 wheel-3.8 [DEBUG base:53]
1184267 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/pkg_resources to /Users/david/Downloads/work/env/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:52]
1184301 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/setuptools-47.3.1.dist-info to /Users/david/Downloads/work/env/lib/python3.8/site-packages/setuptools-47.3.1.dist-info [DEBUG _sync:52]
1184326 generated console scripts easy_install3 easy_install easy_install-3.8 [DEBUG base:53]
1184772 copy /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any/pip-20.1.1.virtualenv to /Users/david/Downloads/work/env/lib/python3.8/site-packages/pip-20.1.1.virtualenv [DEBUG _sync:52]
1184773 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any/pip-20.1.1.dist-info to /Users/david/Downloads/work/env/lib/python3.8/site-packages/pip-20.1.1.dist-info [DEBUG _sync:52]
1184791 generated console scripts pip3.8 pip-3.8 pip3 pip [DEBUG base:53]
1184791 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
1184801 write /Users/david/Downloads/work/env/pyvenv.cfg [DEBUG pyenv_cfg:34]
1184801 home = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1184801 implementation = CPython [DEBUG pyenv_cfg:38]
1184801 version_info = 3.8.3.final.0 [DEBUG pyenv_cfg:38]
1184801 virtualenv = 20.0.25 [DEBUG pyenv_cfg:38]
1184801 include-system-site-packages = false [DEBUG pyenv_cfg:38]
1184801 base-prefix = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1184802 base-exec-prefix = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1184802 base-executable = /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 [DEBUG pyenv_cfg:38]
1184802 created virtual environment CPython3.8.3.final.0-64 in 1184661ms
creator CPython3Posix(dest=/Users/david/Downloads/work/env, clear=True, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/david/Library/Application Support/virtualenv)
added seed packages: pip==20.1.1, setuptools==47.3.1, wheel==0.34.2
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:18]
-> exit 0 |
|
Can you delete those three please, and run it again? For what I see now is that we ran out of wheels to download... but that would imply the content of this file should be something else than you posted. |
|
Just to be sure, I renamed away the entire ~/Library/Application Support/virtualenv directory and re-ran the command from your previous comment (#1883 (comment)). The results are the same as before - every version of pip/setuptools/wheel is downloaded, but after everthing has finished running, .../virtualenv/wheel/3.8/embed/1/pip.json still looks like this: {
"completed": null,
"periodic": true,
"started": "2020-07-03T18:35:05.539654Z",
"versions": []
}setuptools.json and wheel.json are identical to pip.json, except for the Full verbose output$ env _VIRTUALENV_PERIODIC_UPDATE_INLINE=1 virtualenv --clear env --with-traceback -vvv
752 setup logging to NOTSET [DEBUG report:42]
753 created app data folder /Users/david/Library/Application Support/virtualenv [DEBUG __init__:30]
840 find interpreter for spec PythonSpec(path=/Library/Frameworks/Python.framework/Versions/3.8/bin/python3) [INFO builtin:44]
840 proposed PythonInfo(spec=CPython3.8.3.final.0-64, exe=/Library/Frameworks/Python.framework/Versions/3.8/bin/python3, platform=darwin, version='3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) \n[Clang 6.0 (clang-600.0.57)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:50]
841 accepted PythonInfo(spec=CPython3.8.3.final.0-64, exe=/Library/Frameworks/Python.framework/Versions/3.8/bin/python3, platform=darwin, version='3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) \n[Clang 6.0 (clang-600.0.57)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:52]
845 filesystem is not case-sensitive [DEBUG info:28]
1021 create virtual environment via CPython3Posix(dest=/Users/david/Downloads/work/env, clear=True, global=False) [INFO session:52]
1021 create folder /Users/david/Downloads/work/env/bin [DEBUG _sync:25]
1022 create folder /Users/david/Downloads/work/env/lib/python3.8/site-packages [DEBUG _sync:25]
1022 write /Users/david/Downloads/work/env/pyvenv.cfg [DEBUG pyenv_cfg:34]
1022 home = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1022 implementation = CPython [DEBUG pyenv_cfg:38]
1022 version_info = 3.8.3.final.0 [DEBUG pyenv_cfg:38]
1022 virtualenv = 20.0.25 [DEBUG pyenv_cfg:38]
1022 include-system-site-packages = false [DEBUG pyenv_cfg:38]
1022 base-prefix = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1022 base-exec-prefix = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1023 base-executable = /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 [DEBUG pyenv_cfg:38]
1023 symlink /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 to /Users/david/Downloads/work/env/bin/python [DEBUG _sync:44]
1024 create virtualenv import hook file /Users/david/Downloads/work/env/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:94]
1025 create /Users/david/Downloads/work/env/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:97]
1026 ============================== target debug ============================== [DEBUG session:54]
1026 debug via /Users/david/Downloads/work/env/bin/python /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:209]
1026 {
"sys": {
"executable": "/Users/david/Downloads/work/env/bin/python",
"_base_executable": "/Users/david/Downloads/work/env/bin/python",
"prefix": "/Users/david/Downloads/work/env",
"base_prefix": "/Library/Frameworks/Python.framework/Versions/3.8",
"real_prefix": null,
"exec_prefix": "/Users/david/Downloads/work/env",
"base_exec_prefix": "/Library/Frameworks/Python.framework/Versions/3.8",
"path": [
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip",
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8",
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload",
"/Users/david/Downloads/work/env/lib/python3.8/site-packages"
],
"meta_path": [
"<class '_virtualenv._Finder'>",
"<class '_frozen_importlib.BuiltinImporter'>",
"<class '_frozen_importlib.FrozenImporter'>",
"<class '_frozen_importlib_external.PathFinder'>"
],
"fs_encoding": "utf-8",
"io_encoding": "utf-8"
},
"version": "3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) \n[Clang 6.0 (clang-600.0.57)]",
"makefile_filename": "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/config-3.8-darwin/Makefile",
"os": "<module 'os' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py'>",
"site": "<module 'site' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site.py'>",
"datetime": "<module 'datetime' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/datetime.py'>",
"math": "<module 'math' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/math.cpython-38-darwin.so'>",
"json": "<module 'json' from '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
1095 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/david/Library/Application Support/virtualenv) [INFO session:59]
1099 wrote embed update of distribution pip at /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:159]
1099 wrote embed update of distribution wheel at /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:159]
1099 wrote embed update of distribution setuptools at /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:159]
1105 triggered periodic upgrade of pip==20.1.1 (for python 3.8) via background process having PID 61078 [INFO periodic_update:182]
1110 triggered periodic upgrade of wheel==0.34.2 (for python 3.8) via background process having PID 61079 [INFO periodic_update:182]
1111 triggered periodic upgrade of setuptools==47.3.1 (for python 3.8) via background process having PID 61080 [INFO periodic_update:182]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/periodic_update.py", line 214, in do_update
dest = acquire.download_wheel(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/acquire.py", line 67, in download_wheel
raise subprocess.CalledProcessError(process.returncode, cmd, **kwargs)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3', '-m', 'pip', 'download', '--disable-pip-version-check', '--only-binary=:all:', '--no-deps', '--python-version', '3.8', '-d', '/Users/david/Library/Application Support/virtualenv/wheel/house', 'wheel<0.9']' returned non-zero exit status 1.
126431 got embed update of distribution wheel from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/wheel.json [DEBUG via_disk_folder:133]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/periodic_update.py", line 214, in do_update
dest = acquire.download_wheel(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/acquire.py", line 67, in download_wheel
raise subprocess.CalledProcessError(process.returncode, cmd, **kwargs)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3', '-m', 'pip', 'download', '--disable-pip-version-check', '--only-binary=:all:', '--no-deps', '--python-version', '3.8', '-d', '/Users/david/Library/Application Support/virtualenv/wheel/house', 'pip<1.5.1']' returned non-zero exit status 1.
183000 got embed update of distribution pip from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/pip.json [DEBUG via_disk_folder:133]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/periodic_update.py", line 214, in do_update
dest = acquire.download_wheel(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/acquire.py", line 67, in download_wheel
raise subprocess.CalledProcessError(process.returncode, cmd, **kwargs)
subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.8/bin/python3', '-m', 'pip', 'download', '--disable-pip-version-check', '--only-binary=:all:', '--no-deps', '--python-version', '3.8', '-d', '/Users/david/Library/Application Support/virtualenv/wheel/house', 'setuptools<0.9.8']' returned non-zero exit status 1.
1127421 got embed update of distribution setuptools from /Users/david/Library/Application Support/virtualenv/wheel/3.8/embed/1/setuptools.json [DEBUG via_disk_folder:133]
1127422 install wheel from wheel /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/wheel-0.34.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:53]
1127422 install pip from wheel /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/pip-20.1.1-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:53]
1127423 install setuptools from wheel /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/setuptools-47.3.1-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:53]
1127423 Attempting to acquire lock 4511023696 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any.lock [DEBUG filelock:270]
1127424 Attempting to acquire lock 4511023216 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any.lock [DEBUG filelock:270]
1127424 Attempting to acquire lock 4511023504 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any.lock [DEBUG filelock:270]
1127424 Lock 4511023696 acquired on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any.lock [INFO filelock:274]
1127424 Lock 4511023216 acquired on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any.lock [INFO filelock:274]
1127424 Lock 4511023504 acquired on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any.lock [INFO filelock:274]
1127424 build install image for wheel-0.34.2-py2.py3-none-any.whl to /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any [DEBUG base:57]
1127424 build install image for setuptools-47.3.1-py3-none-any.whl to /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any [DEBUG base:57]
1127425 build install image for pip-20.1.1-py2.py3-none-any.whl to /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any [DEBUG base:57]
1127483 Attempting to release lock 4511023696 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any.lock [DEBUG filelock:315]
1127483 Lock 4511023696 released on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any.lock [INFO filelock:318]
1127485 copy /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.virtualenv to /Users/david/Downloads/work/env/lib/python3.8/site-packages/wheel-0.34.2.virtualenv [DEBUG _sync:52]
1127488 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel to /Users/david/Downloads/work/env/lib/python3.8/site-packages/wheel [DEBUG _sync:52]
1127547 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.dist-info to /Users/david/Downloads/work/env/lib/python3.8/site-packages/wheel-0.34.2.dist-info [DEBUG _sync:52]
1127586 generated console scripts wheel-3.8 wheel3 wheel [DEBUG base:53]
1127742 Attempting to release lock 4511023216 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any.lock [DEBUG filelock:315]
1127743 Lock 4511023216 released on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any.lock [INFO filelock:318]
1127743 copy /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/easy_install.py to /Users/david/Downloads/work/env/lib/python3.8/site-packages/easy_install.py [DEBUG _sync:52]
1127745 copy /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/setuptools-47.3.1.virtualenv to /Users/david/Downloads/work/env/lib/python3.8/site-packages/setuptools-47.3.1.virtualenv [DEBUG _sync:52]
1127746 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/setuptools to /Users/david/Downloads/work/env/lib/python3.8/site-packages/setuptools [DEBUG _sync:52]
1127870 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/pkg_resources to /Users/david/Downloads/work/env/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:52]
1127891 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/setuptools-47.3.1-py3-none-any/setuptools-47.3.1.dist-info to /Users/david/Downloads/work/env/lib/python3.8/site-packages/setuptools-47.3.1.dist-info [DEBUG _sync:52]
1127909 generated console scripts easy_install-3.8 easy_install3 easy_install [DEBUG base:53]
1127984 Attempting to release lock 4511023504 on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any.lock [DEBUG filelock:315]
1127984 Lock 4511023504 released on /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any.lock [INFO filelock:318]
1127985 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any/pip to /Users/david/Downloads/work/env/lib/python3.8/site-packages/pip [DEBUG _sync:52]
1128249 copy /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any/pip-20.1.1.virtualenv to /Users/david/Downloads/work/env/lib/python3.8/site-packages/pip-20.1.1.virtualenv [DEBUG _sync:52]
1128250 copy directory /Users/david/Library/Application Support/virtualenv/wheel/3.8/image/1/CopyPipInstall/pip-20.1.1-py2.py3-none-any/pip-20.1.1.dist-info to /Users/david/Downloads/work/env/lib/python3.8/site-packages/pip-20.1.1.dist-info [DEBUG _sync:52]
1128258 generated console scripts pip-3.8 pip3.8 pip3 pip [DEBUG base:53]
1128258 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
1128265 write /Users/david/Downloads/work/env/pyvenv.cfg [DEBUG pyenv_cfg:34]
1128265 home = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1128265 implementation = CPython [DEBUG pyenv_cfg:38]
1128265 version_info = 3.8.3.final.0 [DEBUG pyenv_cfg:38]
1128266 virtualenv = 20.0.25 [DEBUG pyenv_cfg:38]
1128266 include-system-site-packages = false [DEBUG pyenv_cfg:38]
1128266 base-prefix = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1128266 base-exec-prefix = /Library/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
1128266 base-executable = /Library/Frameworks/Python.framework/Versions/3.8/bin/python3 [DEBUG pyenv_cfg:38]
1128267 created virtual environment CPython3.8.3.final.0-64 in 1127517ms
creator CPython3Posix(dest=/Users/david/Downloads/work/env, clear=True, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/david/Library/Application Support/virtualenv)
added seed packages: pip==20.1.1, setuptools==47.3.1, wheel==0.34.2
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:18]
-> exit 0 |
|
ok, I think I found the underlying problem: Python 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> resp = requests.get("https://pypi.org/pypi/pip/json")
>>> resp.status_code, resp.reason
(200, 'OK')
>>> from six.moves.urllib.request import urlopen
>>> stream = urlopen("https://pypi.org/pypi/pip/json")
Traceback (most recent call last):
# ...
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)>
virtualenv/src/virtualenv/seed/wheels/periodic_update.py Lines 237 to 246 in 455afbc And if the actual release date is unknown, the release date checking code falls back to using the download time: virtualenv/src/virtualenv/seed/wheels/periodic_update.py Lines 99 to 101 in 455afbc And that will cause it to download every available version if none of the wheels have been downloaded before, because then every version's download time will be too new. |
|
To conclude the fix here would be to stop downloading wheels once we reach the embedded wheels version. |
gaborbernat
changed the title
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Issue
Occasionally when I create a virtual environment,
virtualenvspawns three background Python processes that continue running even aftervirtualenvhas exited. These background processes consume about 60-80 % CPU each and don't seem to exit on their own. Each process also has another Python subprocess that seems to be restarted every few seconds.This doesn't happen on every
virtualenvrun - in particular, after it happens once (and I terminate all of the background Python processes), it doesn't happen again when I re-runvirtualenv. But if I wait a while and then runvirtualenvagain, the problem reappears.Here is a part of the command lines of these processes. The command lines are incomplete because I copied them from the output of
htopand my terminal window wasn't wide enough. Because the issue is difficult to reproduce I haven't been able to get a better copy of the command lines yet.It looks like this has something to do with the new "periodic update" feature, which would explain why the problem only reappears after waiting for a while. Although according to
virtualenv --helpthe periodic update should only happen every 14 days, and I'm having this problem much more frequently than that (it reappears after a day at most).Environment
OS: macOS 10.14.6
Python: CPython 3.8.3 (macOS installer from Python.org)
`pip list` of the host python where `virtualenv` is installed
Output of the virtual environment creation
The text was updated successfully, but these errors were encountered: