Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Editable installs seem to be broken in tox 4 #2638

Closed
icemac opened this issue Dec 8, 2022 · 6 comments
Closed

Editable installs seem to be broken in tox 4 #2638

icemac opened this issue Dec 8, 2022 · 6 comments

Comments

@icemac
Copy link

icemac commented Dec 8, 2022

Issue

Using an editable install (-e git+https://github.com/user/repos.git@master#egg=repos) no longer works, #egg=repos seems to be handled as a comment.

Using quotes around the URL (-e "git+https://github.com/user/repos.git@master#egg=repos") leads to No closing quotation for tox env py within deps.

Environment

Provide at least:

  • OS: MacOS 12.6.1
  • pip list of the host Python where tox is installed:
$ bin/python -m pip list
Package                           Version
--------------------------------- -------
backports.entry-points-selectable 1.1.1
cachetools                        5.2.0
chardet                           5.1.0
colorama                          0.4.6
distlib                           0.3.6
filelock                          3.8.2
packaging                         22.0
pip                               22.3.1
platformdirs                      2.6.0
pluggy                            1.0.0
py                                1.11.0
pyparsing                         3.0.6
pyproject_api                     1.2.1
setuptools                        65.6.3
six                               1.16.0
toml                              0.10.2
tomli                             2.0.1
tox                               4.0.2
virtualenv                        20.17.1
wheel                             0.38.4

Output of running tox

Provide the output of tox -rvv:

py39: 345 W remove tox env folder /...examle/.tox/py39 [tox/tox_env/api.py:292]
py39: 540 I find interpreter for spec PythonSpec(major=3, minor=9) [virtualenv/discovery/builtin.py:56]
py39: 540 D discover exe for PythonInfo(spec=CPython3.9.16.final.0-64, exe=/.../.local/pipx/venvs/tox/bin/python, platform=darwin, version='3.9.16 (main, Dec  7 2022, 02:41:07) \n[Clang 13.0.0 (clang-1300.0.29.30)]', encoding_fs_io=utf-8-utf-8) in /.../3.9 [virtualenv/discovery/py_info.py:437]
py39: 544 D filesystem is not case-sensitive [virtualenv/info.py:24]
py39: 545 D Attempting to acquire lock 4502976688 on /.../Library/Application Support/virtualenv/py_info/1/0e1b9ad537d5944f5f117e551c80dd4430441b66b9eb1d1db813364c160abf4b.lock [filelock/_api.py:172]
py39: 545 D Lock 4502976688 acquired on /.../Library/Application Support/virtualenv/py_info/1/0e1b9ad537d5944f5f117e551c80dd4430441b66b9eb1d1db813364c160abf4b.lock [filelock/_api.py:176]
py39: 546 D got python info of /.../3.9/bin/python3.9 from /.../Library/Application Support/virtualenv/py_info/1/0e1b9ad537d5944f5f117e551c80dd4430441b66b9eb1d1db813364c160abf4b.json [virtualenv/app_data/via_disk_folder.py:129]
py39: 547 D Attempting to release lock 4502976688 on /.../Library/Application Support/virtualenv/py_info/1/0e1b9ad537d5944f5f117e551c80dd4430441b66b9eb1d1db813364c160abf4b.lock [filelock/_api.py:209]
py39: 547 D Lock 4502976688 released on /.../Library/Application Support/virtualenv/py_info/1/0e1b9ad537d5944f5f117e551c80dd4430441b66b9eb1d1db813364c160abf4b.lock [filelock/_api.py:212]
py39: 547 I proposed PythonInfo(spec=CPython3.9.16.final.0-64, system=/.../3.9/bin/python3.9, exe=/.../.local/pipx/venvs/tox/bin/python, platform=darwin, version='3.9.16 (main, Dec  7 2022, 02:41:07) \n[Clang 13.0.0 (clang-1300.0.29.30)]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:63]
py39: 547 D accepted PythonInfo(spec=CPython3.9.16.final.0-64, system=/.../3.9/bin/python3.9, exe=/.../.local/pipx/venvs/tox/bin/python, platform=darwin, version='3.9.16 (main, Dec  7 2022, 02:41:07) \n[Clang 13.0.0 (clang-1300.0.29.30)]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:65]
py39: 604 I create virtual environment via CPython3Posix(dest=/...examle/.tox/py39, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:48]
py39: 605 D create folder /...examle/.tox/py39/bin [virtualenv/util/path/_sync.py:9]
py39: 605 D create folder /...examle/.tox/py39/lib/python3.9/site-packages [virtualenv/util/path/_sync.py:9]
py39: 605 D write /...examle/.tox/py39/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
py39: 605 D 	home = /.../3.9/bin [virtualenv/create/pyenv_cfg.py:34]
py39: 605 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
py39: 605 D 	version_info = 3.9.16.final.0 [virtualenv/create/pyenv_cfg.py:34]
py39: 606 D 	virtualenv = 20.17.1 [virtualenv/create/pyenv_cfg.py:34]
py39: 606 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
py39: 606 D 	base-prefix = /.../3.9 [virtualenv/create/pyenv_cfg.py:34]
py39: 606 D 	base-exec-prefix = /.../3.9 [virtualenv/create/pyenv_cfg.py:34]
py39: 606 D 	base-executable = /.../3.9/bin/python3.9 [virtualenv/create/pyenv_cfg.py:34]
py39: 606 D symlink /.../3.9/bin/python3.9 to /...examle/.tox/py39/bin/python [virtualenv/util/path/_sync.py:28]
py39: 607 D create virtualenv import hook file /...examle/.tox/py39/lib/python3.9/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:89]
py39: 607 D create /...examle/.tox/py39/lib/python3.9/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:92]
py39: 608 D ============================== target debug ============================== [virtualenv/run/session.py:50]
py39: 608 D debug via /...examle/.tox/py39/bin/python /.../.local/pipx/venvs/tox/lib/python3.9/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:197]
py39: 608 D {
  "sys": {
    "executable": "/...examle/.tox/py39/bin/python",
    "_base_executable": "/...examle/.tox/py39/bin/python",
    "prefix": "/...examle/.tox/py39",
    "base_prefix": "/.../3.9",
    "real_prefix": null,
    "exec_prefix": "/...examle/.tox/py39",
    "base_exec_prefix": "/.../3.9",
    "path": [
      "/.../3.9/lib/python39.zip",
      "/.../3.9/lib/python3.9",
      "/.../3.9/lib/python3.9/lib-dynload",
      "/...examle/.tox/py39/lib/python3.9/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.9.16 (main, Dec  7 2022, 02:41:07) \n[Clang 13.0.0 (clang-1300.0.29.30)]",
  "makefile_filename": "/.../3.9/lib/python3.9/config-3.9-darwin/Makefile",
  "os": "<module 'os' from '/.../3.9/lib/python3.9/os.py'>",
  "site": "<module 'site' from '/.../3.9/lib/python3.9/site.py'>",
  "datetime": "<module 'datetime' from '/.../3.9/lib/python3.9/datetime.py'>",
  "math": "<module 'math' from '/.../3.9/lib/python3.9/lib-dynload/math.cpython-39-darwin.so'>",
  "json": "<module 'json' from '/.../3.9/lib/python3.9/json/__init__.py'>"
} [virtualenv/run/session.py:51]
py39: 679 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/.../Library/Application Support/virtualenv) [virtualenv/run/session.py:55]
py39: 683 D got embed update of distribution pip from /.../Library/Application Support/virtualenv/wheel/3.9/embed/3/pip.json [virtualenv/app_data/via_disk_folder.py:129]
py39: 683 D got embed update of distribution wheel from /.../Library/Application Support/virtualenv/wheel/3.9/embed/3/wheel.json [virtualenv/app_data/via_disk_folder.py:129]
py39: 683 D got embed update of distribution setuptools from /.../Library/Application Support/virtualenv/wheel/3.9/embed/3/setuptools.json [virtualenv/app_data/via_disk_folder.py:129]
py39: 688 D install pip from wheel /.../.local/pipx/venvs/tox/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.3.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py39: 689 D install wheel from wheel /.../.local/pipx/venvs/tox/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/wheel-0.38.4-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py39: 689 D Attempting to acquire lock 4505143472 on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any.lock [filelock/_api.py:172]
py39: 690 D install setuptools from wheel /.../.local/pipx/venvs/tox/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/setuptools-65.6.3-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py39: 690 D Attempting to acquire lock 4505168336 on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any.lock [filelock/_api.py:172]
py39: 690 D Attempting to acquire lock 4505168720 on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any.lock [filelock/_api.py:172]
py39: 690 D Lock 4505143472 acquired on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any.lock [filelock/_api.py:176]
py39: 691 D Lock 4505168720 acquired on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any.lock [filelock/_api.py:176]
py39: 691 D Lock 4505168336 acquired on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any.lock [filelock/_api.py:176]
py39: 691 D Attempting to release lock 4505168720 on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any.lock [filelock/_api.py:209]
py39: 691 D Attempting to release lock 4505143472 on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any.lock [filelock/_api.py:209]
py39: 692 D Lock 4505143472 released on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any.lock [filelock/_api.py:212]
py39: 692 D Lock 4505168720 released on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any.lock [filelock/_api.py:212]
py39: 692 D Attempting to release lock 4505168336 on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any.lock [filelock/_api.py:209]
py39: 692 D Lock 4505168336 released on /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any.lock [filelock/_api.py:212]
py39: 693 D copy directory /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel to /...examle/.tox/py39/lib/python3.9/site-packages/wheel [virtualenv/util/path/_sync.py:36]
py39: 693 D copy /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools-65.6.3.virtualenv to /...examle/.tox/py39/lib/python3.9/site-packages/setuptools-65.6.3.virtualenv [virtualenv/util/path/_sync.py:36]
py39: 693 D copy /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip-22.3.1.virtualenv to /...examle/.tox/py39/lib/python3.9/site-packages/pip-22.3.1.virtualenv [virtualenv/util/path/_sync.py:36]
py39: 694 D copy /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/distutils-precedence.pth to /...examle/.tox/py39/lib/python3.9/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:36]
py39: 695 D copy directory /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip to /...examle/.tox/py39/lib/python3.9/site-packages/pip [virtualenv/util/path/_sync.py:36]
py39: 696 D copy directory /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools-65.6.3.dist-info to /...examle/.tox/py39/lib/python3.9/site-packages/setuptools-65.6.3.dist-info [virtualenv/util/path/_sync.py:36]
py39: 704 D copy directory /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools to /...examle/.tox/py39/lib/python3.9/site-packages/setuptools [virtualenv/util/path/_sync.py:36]
py39: 714 D copy /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.virtualenv to /...examle/.tox/py39/lib/python3.9/site-packages/wheel-0.38.4.virtualenv [virtualenv/util/path/_sync.py:36]
py39: 715 D copy directory /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.dist-info to /...examle/.tox/py39/lib/python3.9/site-packages/wheel-0.38.4.dist-info [virtualenv/util/path/_sync.py:36]
py39: 725 D changing mode of /...examle/.tox/py39/bin/wheel-3.9 to 755 [distlib/util.py:572]
py39: 725 D changing mode of /...examle/.tox/py39/bin/wheel3 to 755 [distlib/util.py:572]
py39: 725 D changing mode of /...examle/.tox/py39/bin/wheel3.9 to 755 [distlib/util.py:572]
py39: 726 D changing mode of /...examle/.tox/py39/bin/wheel to 755 [distlib/util.py:572]
py39: 726 D generated console scripts wheel3.9 wheel wheel-3.9 wheel3 [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py39: 888 D copy directory /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/pkg_resources to /...examle/.tox/py39/lib/python3.9/site-packages/pkg_resources [virtualenv/util/path/_sync.py:36]
py39: 930 D copy directory /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/_distutils_hack to /...examle/.tox/py39/lib/python3.9/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:36]
py39: 933 D generated console scripts  [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py39: 1154 D copy directory /.../Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip-22.3.1.dist-info to /...examle/.tox/py39/lib/python3.9/site-packages/pip-22.3.1.dist-info [virtualenv/util/path/_sync.py:36]
py39: 1159 D changing mode of /...examle/.tox/py39/bin/pip-3.9 to 755 [distlib/util.py:572]
py39: 1160 D changing mode of /...examle/.tox/py39/bin/pip3.9 to 755 [distlib/util.py:572]
py39: 1160 D changing mode of /...examle/.tox/py39/bin/pip3 to 755 [distlib/util.py:572]
py39: 1161 D changing mode of /...examle/.tox/py39/bin/pip to 755 [distlib/util.py:572]
py39: 1161 D generated console scripts pip pip3 pip3.9 pip-3.9 [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py39: 1161 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:61]
py39: 1165 D write /...examle/.tox/py39/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
py39: 1165 D 	home = /.../3.9/bin [virtualenv/create/pyenv_cfg.py:34]
py39: 1165 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
py39: 1165 D 	version_info = 3.9.16.final.0 [virtualenv/create/pyenv_cfg.py:34]
py39: 1165 D 	virtualenv = 20.17.1 [virtualenv/create/pyenv_cfg.py:34]
py39: 1165 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
py39: 1166 D 	base-prefix = /.../3.9 [virtualenv/create/pyenv_cfg.py:34]
py39: 1166 D 	base-exec-prefix = /.../3.9 [virtualenv/create/pyenv_cfg.py:34]
py39: 1166 D 	base-executable = /.../3.9/bin/python3.9 [virtualenv/create/pyenv_cfg.py:34]
py39: 1169 W install_deps> python -I -m pip install -e git+https://github.com/zopefoundation/ZODB.git@master [tox/tox_env/api.py:398]
ERROR: Could not detect requirement name for 'git+https://github.com/zopefoundation/ZODB.git@master', please specify one with #egg=your_package_name
py39: 2484 C exit 1 (1.31 seconds) /...examle> python -I -m pip install -e git+https://github.com/zopefoundation/ZODB.git@master pid=56779 [tox/execute/api.py:275]
  py39: FAIL code 1 (2.14 seconds)
  evaluation failed :( (2.26 seconds)

Minimal example

If possible, provide a minimal reproducer for the issue:

[tox]
env_list = py39
minversion = 4.0.2

[testenv]
deps = -e git+https://github.com/zopefoundation/ZODB.git@master#egg=ZODB

The URL is taken from a workin example using tox v3.
@jugmac00
Copy link
Member

jugmac00 commented Dec 8, 2022 •
edited
Loading

Hey @icemac

This looks like a duplicate of #2617

I started to create a PR to document the changed behavior yesterday in #2621 but I missed the scope. I'll update that PR once I get a chance.

You need to change your code like this:

[tox]
env_list = py39
minversion = 4.0.2

[testenv]
deps = -e git+https://github.com/zopefoundation/ZODB.git@master\#egg=ZODB

Thanks for you detailed report and sorry for the inconveniences!

@jugmac00 jugmac00 closed this as completed Dec 8, 2022
icemac pushed a commit to zopefoundation/ZEO that referenced this issue Dec 8, 2022
Use older tox version.
2c2a98c 
See tox-dev/tox#2638 for details.
icemac pushed a commit to zopefoundation/meta that referenced this issue Dec 8, 2022
Require an older tox version.
fb19c36 
See tox-dev/tox#2638 for details.
@icemac
Copy link
Author

icemac commented Dec 8, 2022

@jugmac00 I might be blind but I see no difference between my code and your suggestion.

@jugmac00
Copy link
Member

jugmac00 commented Dec 8, 2022

@icemac Sorry, I updated my code example. You need to escape the # sign.

@icemac icemac mentioned this issue Dec 9, 2022
Merged
@matthewhegarty
Copy link

Escaping the URL fixes the issue on Mac OS however it causes my Github action to break (tox version 4.4.3)

ERROR: could not install deps [-egit+[https://github.com/jazzband/tablib.git@master\#egg=tablib](https://github.com/jazzband/tablib.git@master/#egg=tablib), https://github.com/django/django/archive/main.tar.gz, -rrequirements/test.txt]; v = InvocationError("/home/runner/work/django-import-export/django-import-export/.tox/py311-djangomain-tablibdev/bin/python -m pip install '-egit+[https://github.com/jazzband/tablib.git@master\\#egg=tablib](https://github.com/jazzband/tablib.git@master//#egg=tablib)' https://github.com/django/django/archive/main.tar.gz -rrequirements/test.txt", 1)

deps:

deps =
    tablibdev: -egit+https://github.com/jazzband/tablib.git@master\#egg=tablib
    ...

https://github.com/django-import-export/django-import-export/actions/runs/4054995411/jobs/6977533371

@masenf
Copy link
Collaborator

masenf commented Jan 31, 2023

@matthewhegarty your CI runs are using tox 3, not tox 4.

In tox 3, the \# is interpreted literally, not as an escape.

It's unfortunate there is no syntax that can work for hash character in both tox 3 and tox 4.

Consider using the "minversion" conf key to set the minimum tox version to 4.0

@matthewhegarty
Copy link

Thanks for the response. I confirm that 'minversion' has solved the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@icemac @masenf @matthewhegarty @jugmac00