Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extras names are incorrectly normalized in poetry-core >= 1.3.0 #6819

Closed
4 tasks done
ghost opened this issue Oct 16, 2022 · 8 comments
Closed
4 tasks done

Extras names are incorrectly normalized in poetry-core >= 1.3.0 #6819

ghost opened this issue Oct 16, 2022 · 8 comments
Labels
area/build-system Related to PEP 517 packaging (see poetry-core) status/external-issue Issue is caused by external project (platform, dep, etc)

Comments

@ghost
Copy link

ghost commented Oct 16, 2022

  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

This appears to be a regression introduced in poetry-core 1.3.0. I'm opening the issue here because issues are closed on the poetry-core repo.

When using poetry-core >= 1.3.0, names of extras are incorrectly normalized to change _ to -, causing their installation to fail:

❯ pip install .
Processing /home/nyuszika7h/test
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: requests[use-chardet-on-py3]<3.0.0,>=2.28.1 in ./.venv/lib/python3.10/site-packages (from test==0.1.0) (2.28.1)
WARNING: requests 2.28.1 does not provide the extra 'use-chardet-on-py3'
Requirement already satisfied: charset-normalizer<3,>=2 in ./.venv/lib/python3.10/site-packages (from requests[use-chardet-on-py3]<3.0.0,>=2.28.1->test==0.1.0) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.10/site-packages (from requests[use-chardet-on-py3]<3.0.0,>=2.28.1->test==0.1.0) (2022.9.24)
Requirement already satisfied: idna<4,>=2.5 in ./.venv/lib/python3.10/site-packages (from requests[use-chardet-on-py3]<3.0.0,>=2.28.1->test==0.1.0) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.venv/lib/python3.10/site-packages (from requests[use-chardet-on-py3]<3.0.0,>=2.28.1->test==0.1.0) (1.26.12)
Building wheels for collected packages: test
  Building wheel for test (pyproject.toml) ... done
  Created wheel for test: filename=test-0.1.0-py3-none-any.whl size=827 sha256=ad5626a8f9c6f9ceafa81586876887f6ffb6df743a34fb0ba50ecfd54371ad89
  Stored in directory: /tmp/pip-ephem-wheel-cache-53k7_wx9/wheels/ff/6c/d3/80cb9d9d69b88685ef924814b380c07445b80a9c85a2cb1b54
Successfully built test
Installing collected packages: test
Successfully installed test-0.1.0

With poetry-core 1.2.0:

❯ pip install .
Processing /home/nyuszika7h/test
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: requests[use_chardet_on_py3]<3.0.0,>=2.28.1 in /home/nyuszika7h/.local/share/pyenv/versions/3.10.7/lib/python3.10/site-packages (from test==0.1.0) (2.28.1)
Requirement already satisfied: certifi>=2017.4.17 in /home/nyuszika7h/.local/share/pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests[use_chardet_on_py3]<3.0.0,>=2.28.1->test==0.1.0) (2022.9.24)
Requirement already satisfied: idna<4,>=2.5 in /home/nyuszika7h/.local/share/pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests[use_chardet_on_py3]<3.0.0,>=2.28.1->test==0.1.0) (3.4)
Requirement already satisfied: charset-normalizer<3,>=2 in /home/nyuszika7h/.local/share/pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests[use_chardet_on_py3]<3.0.0,>=2.28.1->test==0.1.0) (2.1.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/nyuszika7h/.local/share/pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests[use_chardet_on_py3]<3.0.0,>=2.28.1->test==0.1.0) (1.26.12)
Collecting chardet<6,>=3.0.2
  Using cached chardet-5.0.0-py3-none-any.whl (193 kB)
Building wheels for collected packages: test
  Building wheel for test (pyproject.toml) ... done
  Created wheel for test: filename=test-0.1.0-py3-none-any.whl size=823 sha256=30ec1efd1fc1ca60f9ced0c8850a14acc5dfdc4c98dc2fa6840c4d6153dfe57f
  Stored in directory: /tmp/pip-ephem-wheel-cache-z_hwotnm/wheels/ff/6c/d3/80cb9d9d69b88685ef924814b380c07445b80a9c85a2cb1b54
Successfully built test
Installing collected packages: chardet, test
Successfully installed chardet-5.0.0 test-0.1.0
@ghost ghost added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Oct 16, 2022
@neersighted
Copy link
Member

The issue is not the normalization -- the issue is that the other side of the comparison is not normalized when looking for extras, as best I can tell from a quick skim. The normalization is specified in PEP 685.

PTAL @dimbleby

@neersighted
Copy link
Member

neersighted commented Oct 16, 2022

Ah, I just processed that this was pip install and not poetry install -- I think this might be a pip bug. They are required by spec to normalize extra names before making a comparison -- Poetry is writing the canonical name, which by spec should be considered equivalent.

@neersighted neersighted added area/build-system Related to PEP 517 packaging (see poetry-core) status/external-issue Issue is caused by external project (platform, dep, etc) and removed kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Oct 16, 2022
@dimbleby
Copy link
Contributor

yep, looks like basically a duplicate of python-poetry/poetry-plugin-export#150

here's a pip bug requesting pep 685 be implemented pypa/pip#11445

I've not looked at the pip code but I can't imagine that the part of pep685 which says "normalize extra names before comparison" can be very hard to do, perhaps someone who is affected by this would like to put together an MR over there

@dimbleby
Copy link
Contributor

dimbleby commented Oct 16, 2022

looks as though this was implemented in packaging some months ago - pypa/packaging#545 - but the subsequent release has gotten a bit stalled - pypa/packaging#569

However updates in the last couple of days suggest that it's coming imminently, and I'd guess that pip will pick that up in their next release.

So IMO poetry should hold firm here. Reverting the change, apart from being tedious, would reintroduce the bugs that it fixed (#6321) - ie it's not obviously making things better - and it looks reasonable to expect that the ecosystem is close to catching up with the specs.

@ofek
Copy link
Contributor

ofek commented Oct 17, 2022

@dimbleby
Copy link
Contributor

dimbleby commented Dec 8, 2022

It's taken somewhat longer than I'd guessed it would, but packaging 22.0 has now been released

So I guess that a pip release picking that up will happen in due course, and pip should soon agree with poetry - or rather with PEP685 - about how extras are normalized.

@dimbleby
Copy link
Contributor

dimbleby commented Mar 2, 2024

well it really is taking a while for pip to implement pep 685, pypa/pip#11445 seems to be harder than I'd have guessed.

nevertheless there is no poetry bug here and no intention of changing poetry behaviour, might as well close this one out I think

Copy link

github-actions bot commented Jul 7, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/build-system Related to PEP 517 packaging (see poetry-core) status/external-issue Issue is caused by external project (platform, dep, etc)
Projects
None yet
Development

No branches or pull requests

4 participants