Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latest poetry-core with poetry 1.0 caused pip error during tox building #3001

Closed
2 of 3 tasks
fredrikaverpil opened this issue Sep 30, 2020 · 14 comments
Closed
2 of 3 tasks
Labels
kind/bug Something isn't working as expected status/needs-reproduction Issue needs a minimal reproduction to be confirmed

Comments

@fredrikaverpil
Copy link
Contributor

fredrikaverpil commented Sep 30, 2020

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: Linux
  • Poetry version: 1.0.10
  • Link of a Gist with the contents of your pyproject.toml file:

Issue

We use tox (with isolated_build=true) with poetry 1.0.10 and the build system in pyproject.toml looked like this:

[build-system]
requires = ["poetry-core>=1.0.0a9"]
build-backend = "poetry.core.masonry.api"

Jenkins log, from executing the tox testenv:

+ tox -e mytestenv
.package create: REDACTED/.tox/.package
.package installdeps: poetry-core>=1.0.0a9
mytestenv create: REDACTED/.tox/mytestenv
mytestenv inst: REDACTED/.tox/.tmp/package/1/pkg1-0.0.0.tar.gz
ERROR: invocation failed (exit code 2), logfile: REDACTED/.tox/mytestenv/log/mytestenv-1.log
================================== log start ===================================
Looking in indexes: https://REDACTED/api/pypi/pypi-remote/simple, https://REDACTED/api/pypi/custom/simple
Processing ./.tox/.tmp/package/1/pkg1-0.0.0.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run
    reqs, check_supported_wheels=not options.target_dir
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 340, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 483, in prepare_linked_requirement
    req, self.req_tracker, self.finder, self.build_isolation,
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 161, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 86, in _build_backend
    obj = import_module(mod_path)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/__init__.py", line 10, in <module>
    from .builder import Builder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builder.py", line 6, in <module>
    from .builders.sdist import SdistBuilder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builders/__init__.py", line 2, in <module>
    from .wheel import WheelBuilder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builders/wheel.py", line 18, in <module>
    from packaging.tags import sys_tags
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/_vendor/packaging/tags.py", line 7, in <module>
    import distutils.util
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/_distutils_hack/__init__.py", line 83, in create_module
    return importlib.import_module('setuptools._distutils')
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools'

Changing our build system to the following solved the problem:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"
@fredrikaverpil fredrikaverpil added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Sep 30, 2020
@abn
Copy link
Member

abn commented Sep 30, 2020

@fredrikaverpil I believe the issue is the system default pip version. There was a bug in pip (circa 19.1) that meant that the PEP 517 build environment was not correctly created with the right versions of setuptools etc. With a poetry managed virtual environment this should not be an issue anymore as we will enforce the version of pip used via virtualenv (#2666).

Maybe #2826 will also remove this issue for externally (tox) managed virtual environments as well. In the interim, I would suggest that you get tox to upgrade pip used int he environment. Note that by default (if not using virtualenv's seed wheels), the system Python version's default pip version will be used, not the version you have in your active environment (ie. if you did pip install --upgrade --user pip (user is redundant in new versions of pip). The system default version ususaly is older on

@abn
Copy link
Member

abn commented Sep 30, 2020

Also, this should not work unless your environment somehow already has poetry along with poetry-core

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.masonry.api"

@fredrikaverpil
Copy link
Contributor Author

Oops, I meant to write:

Changing our build system to the following solved the problem:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"

@fredrikaverpil
Copy link
Contributor Author

The funny thing is this happened over night without a change in tox.ini or pyproject.toml.

Ok, I will see which version of pip is in the system-Python.

@finswimmer
Copy link
Member

The funny thing is this happened over night without a change in tox.ini or pyproject.toml.

The funny thing about that, is that a colleague of mine showed me this today with exact the same words 😄

We simply update poetry to the freshly released 1.1 and the problem vanished.

@Shadowsong27
Copy link

I am seeing this as well (not in tox, but standard poetry install), less than 1 day since the last success built, strange thing is: we have always fixed the poetry version to 1.0.5, but once updated to the latest 1.1.0, it works fine.

@nbraem
Copy link

nbraem commented Oct 2, 2020

seeing this as well, workaround is to add setuptools to requires:

[build-system]
requires = ["poetry", "setuptools"]
build-backend = "poetry.masonry.api"

This way, you can still use poetry 1.1.0

@abn
Copy link
Member

abn commented Oct 3, 2020

@fredrikaverpil I am unable to reproduce this in a clean environment,

$ podman run --rm -i --entrypoint bash python:3.7 <<EOF
> pip install --quiet --upgrade pip
> pip install --quiet poetry tox
> pushd /opt
> poetry new foobar
> pushd foobar
> cat > tox.ini <<TOXEOF
> [tox]
> minversion = 3.3.0
> isolated_build = True
> envlist = py37
> 
> [testenv]
> commands =
>     pip list
> TOXEOF
> cat pyproject.toml
> tox -e py37
> EOF
/opt /
Created package foobar in foobar
/opt/foobar /opt /
[tool.poetry]
name = "foobar"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
.package create: /opt/foobar/.tox/.package
.package installdeps: poetry-core>=1.0.0
py37 create: /opt/foobar/.tox/py37
py37 inst: /opt/foobar/.tox/.tmp/package/1/foobar-0.1.0.tar.gz
py37 installed: foobar @ file:///opt/foobar/.tox/.tmp/package/1/foobar-0.1.0.tar.gz
py37 run-test-pre: PYTHONHASHSEED='2706476223'
py37 run-test: commands[0] | pip list
Package    Version
---------- -------
foobar     0.1.0
pip        20.2.3
setuptools 50.3.0
wheel      0.35.1
___________________________________ summary ____________________________________
  py37: commands succeeded
  congratulations :)

@abn
Copy link
Member

abn commented Oct 3, 2020

@fredrikaverpil these issues may be of relevance for you pypa/setuptools#2353 pypa/setuptools#2376 pypa/pip#6264.

@fredrikaverpil
Copy link
Contributor Author

Hm, a lot of people complaining about setuptools 50.0, but it all worked itself out fine on my end by using the following:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"

Does the above not make use of setuptools?

@mtkennerly
Copy link

@abn Those tickets are from a month ago, but I saw this issue start happening specifically within the last week in a GitHub Actions workflow, without any code changes in between.

It occurs only in certain combinations. Python (3.7 | 3.8) + Poetry (1.0.2 | 1.0.5) fail, whereas the same Poetry versions work with Python (3.5 | 3.6), and Poetry (1.0.10 | 1.1.0rc1) work with all four versions of Python tested. It doesn't matter whether Poetry is installed with pip or with get-poetry.py (called "script" in the workflow matrix).

@jhrmnn
Copy link
Contributor

jhrmnn commented Oct 9, 2020

I'm seeing the missing Setuptools error in environments created with Conda.

jhabriel added a commit to jhabriel/porepy that referenced this issue Nov 19, 2020
I'm not entirely sure what's causing the error. It seems to be related either to setuptools or to poetry, or maybe both. Thus, this is more or less a brute force fix. I've found the it here: python-poetry/poetry#3001
keileg pushed a commit to pmgbergen/porepy that referenced this issue Nov 19, 2020
I'm not entirely sure what's causing the error. It seems to be related either to setuptools or to poetry, or maybe both. Thus, this is more or less a brute force fix. I've found the it here: python-poetry/poetry#3001
kojiishi added a commit to kojiishi/east_asian_spacing that referenced this issue Dec 28, 2021
Installing the local directory from other projects using `pip` fails:
```sh
$ cd chws_tools
$ pip install ../east_asian_spacing
...
ModuleNotFoundError: No module named 'setuptools'
```

This change can work around the problem as described at:
  python-poetry/poetry#3001
@neersighted
Copy link
Member

Doesn't reproduce with any modern versions -- please open a new issue if you think you're hitting this.

@neersighted neersighted closed this as not planned Won't fix, can't repro, duplicate, stale Oct 5, 2022
@neersighted neersighted added status/needs-reproduction Issue needs a minimal reproduction to be confirmed and removed status/triage This issue needs to be triaged labels Oct 5, 2022
Copy link

github-actions bot commented Mar 1, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected status/needs-reproduction Issue needs a minimal reproduction to be confirmed
Projects
None yet
Development

No branches or pull requests

8 participants