[1.1.0b2] Bumping version leaves orphaned dist-info in site-packages #2918

MrGreenTea · 2020-09-15T14:52:01Z

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
OS version and name: elementary OS 5.1 Hera
Poetry version: 1.1.0b2
Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/MrGreenTea/992a1795471c00d727c02920d4288f16

Issue

When updating my packages version with poetry version old dist-info directories are left in site-packages.
This trips up pip, importlib and pkg_resources for example when trying to find the packages information.
To reproduce:
(run with poetry config virtualenvs.in-project true)

❯ poetry new example
❯ poetry install
❯ ls .venv/lib/python3.8/site-packages/example*.dist-info

.venv/lib/python3.8/site-packages/example-0.1.0.dist-info:
INSTALLER  METADATA  RECORD

❯ poetry version minor && poetry install
❯ ls .venv/lib/python3.8/site-packages/example*.dist-info

.venv/lib/python3.8/site-packages/example-0.1.0.dist-info:
INSTALLER  METADATA  RECORD

.venv/lib/python3.8/site-packages/example-0.2.0.dist-info:
INSTALLER  METADATA  RECORD

❯ poetry run pip show example

Name: example
Version: 0.2.0
Summary: 
Home-page: None
Author: Jonas Bulik
Author-email: jonas@bulik.dev
License: None
Location: /tmp/example/.venv/lib/python3.8/site-packages
Requires: 
Required-by:

❯ poetry version 0.1.0
❯ poetry run pip show example

Name: example
Version: 0.2.0
Summary: 
Home-page: None
Author: Jonas Bulik
Author-email: jonas@bulik.dev
License: None
Location: /tmp/example/.venv/lib/python3.8/site-packages
Requires: 
Required-by:

The text was updated successfully, but these errors were encountered:

finswimmer · 2020-09-15T18:02:05Z

Hello @MrGreenTea,

I guess #2855 is related, is it?

fin swimmer

MrGreenTea · 2020-09-15T18:39:09Z

Hello @MrGreenTea,

I guess #2855 is related, is it?

fin swimmer

@finswimmer Looks like the same cause, yeah.

jkbecker · 2021-02-02T22:52:03Z

I can reproduce that, and problematically, version bumping will

always report the newest version as being installed during poetry install
however, the actual code being run may or may not be the newest version

For example (excerpt from a discussion I had with sinoroc on the Discord trying to debug version issues of my project):

Is this currently being worked on somehow, or is there a clean workaround in the meantime?

sinoroc · 2021-02-03T07:35:00Z

Does running poetry run python -m pip uninstall YourProjectName before running poetry install again help? It might be necessary to run the pip uninstall step more than one time until all old dist-info directories are removed. Otherwise removing the old dist-info directories manually seems like a good workaround. Another thing would be to delete the virtual environment and recreate it from scratch.

ns-admetrics · 2021-03-01T16:46:31Z

Using pip uninstall has the issue that you don't know how often to run it (potentially there are multiple old versions), but pip uninstall does not return an error code if there was no such package.

fredrikaverpil · 2021-04-02T08:50:08Z

@ns-admetrics

Using pip uninstall has the issue that you don't know how often to run it (potentially there are multiple old versions), but pip uninstall does not return an error code if there was no such package.

It seems however, that you will get a certain error message whenever pip has removed all the different versions:

$ pip uninstall b
Found existing installation: b 0.0.5
Uninstalling b-0.0.5:
  Would remove:
    /home/fredrik/code/repos/a/.venv/bin/bclix
    /home/fredrik/code/repos/a/.venv/lib/python3.9/site-packages/b-0.0.5.dist-info/*
    /home/fredrik/code/repos/a/.venv/lib/python3.9/site-packages/b.pth
Proceed (y/n)? y
  Successfully uninstalled b-0.0.5

$ pip uninstall b
Found existing installation: b 0.0.4
Uninstalling b-0.0.4:
  Would remove:
    /home/fredrik/code/repos/a/.venv/lib/python3.9/site-packages/b-0.0.4.dist-info/*
Proceed (y/n)? y
  Successfully uninstalled b-0.0.4

$ pip uninstall b
WARNING: Skipping b as it is not installed.

ns-admetrics · 2021-04-07T16:16:57Z

It seems however, that you will get a certain error message whenever pip has removed all the different versions:

Yup but parsing messages is silly :) Re. pip (off-topic...), if the semantics of uninstall are declarative then it should uninstall all versions if multiple, if the semantics are imperative then it should fail in this WARNING case.

jkbecker · 2021-04-07T19:42:00Z

It's kind of annoying that pip doesn't exit with something non-zero when a package was not installed. I understand why it doesn't but if it did, one could just run uninstall in a while loop until the command fails to be sure it was completely cleaned up...

sinoroc · 2021-04-07T19:55:48Z

pip show ProjectName seems to give a usable return value.

So, it could be possible to run pip uninstall ProjectName until pip show ProjectName returns a non-zero value, if that helps...

jkbecker · 2021-04-07T20:34:33Z

Nice! Thank you, that is actually really helpful in my project as workaround for now.

abn · 2021-04-09T15:40:07Z

Can you please try the fix at #3900.

Using pipx

pipx install --suffix=@3900 'poetry @ git+https://github.com/python-poetry/poetry.git@refs/pull/3900/head'
alias poetry=poetry@3900
poetry config virtualenvs.in-project true
poetry new example
pushd example
poetry install
find .venv/ -type d -name 'example*.dist-info'
poetry version minor
poetry install
find .venv/ -type d -name 'example*.dist-info'
poetry run pip show example
poetry version 0.1.0
poetry install
poetry run pip show example

Using a container (podman | docker)

podman run --rm -i --entrypoint bash python:3.8 <<EOF
set -xe
python -m pip install -q git+https://github.com/python-poetry/poetry.git@refs/pull/3900/head
poetry config virtualenvs.in-project true
poetry new example
pushd example
poetry install
find .venv/ -type d -name 'example*.dist-info'
poetry version minor
poetry install
find .venv/ -type d -name 'example*.dist-info'
poetry run pip show example
poetry version 0.1.0
poetry install
poetry run pip show example
EOF

fredrikaverpil · 2021-04-09T18:41:43Z

@abn I just tried the #3900 fix (pipx approach) and I no longer see the lingering .dist-info folders accumulating. 👍
I tested this with path dependencies and git dependencies on Python 3.9.2 and 3.6.10.

However, I can see a couple of weird things. Maybe expected, but I figured I would mention them...

Output on Python 3.9.2 below. You can see how the "Installing..." never turns into the actual version on completed installation:

$ poetry@3900 install
Installing dependencies from lock file

Package operations: 6 installs, 0 updates, 0 removals

  • Installing certifi (2020.12.5): Installing...
  • Installing chardet (3.0.4): Installing...
  • Installing idna (2.8): Installing...
  • Installing urllib3 (1.22): Installing...
  • Installing requests (2.21.0): Installing...
  • Installing b (2.1.2 /home/fredrik/code/repos/b)

Installing the current project: a (1.0.0)

What was also a bit weird was that I somehow managed to break the output by first using a poetry 1.1.5:

$ poetry@work install
The virtual environment found in /home/fredrik/code/repos/a/.venv seems to be broken.
Recreating virtualenv a in /home/fredrik/code/repos/a/.venv
Installing dependencies from lock file

Package operations: 6 installs, 0 updates, 0 removals

  • Installing certifi (2020.12.5)
  • Installing chardet (3.0.4)
  • Installing idna (2.8)
  • Installing urllib3 (1.22)
  • Installing requests (2.21.0)
  • Installing b (2.1.2 /home/fredrik/code/repos/b)

Installing the current project: a (1.0.0)

and then I tried poetry@3900 again:

$ poetry@3900 install
Installing dependencies from lock file

Package operations: 0 installs, 5 updates, 0 removals

  • Updating certifi (2020.12.5 /home/fredrik/.cache/pyp
  • Updating certifi (2020.12.5 /home/fredrik/.cache/pypoetry/artifacts/d8/df/24/ed696681f34f8916b0aef99138db9a94e37d54684b9829af34a7fd4e39/certifi-2020.12.5-py2.py3-none-any.whl -> 2020.12.5): Installing...
  • Updating chardet (3.0.4 /home/fredrik/.cache/pypoetry/artifacts/8f/6f/1c/8085d730ad63c462222af30d0d01c4bd0caca5287e40b63c1fe8f529b7/chardet-3.0.4-py2.py3-none-any.whl -> 3.0.4): Installing...
  • Updating idna (2.8 /home/fredrik/.cache/pypoetry/artifacts/06/ea/a8/977db370296c645a9f7e1d772193cb28e1805639dace642b579a7d2cc0/idna-2.8-py2.py3-none-any.whl -> 2.8): Installing...
  • Updating urllib3 (1.22 /home/fredrik/.cache/pypoetry/artifacts/8a/d2/c3/8140b9cc76677757ce1397e48fec8a4c4c92e0ba44d0aa2aeb9bcf9b03/urllib3-1.22-py2.py3-none-any.whl -> 1.22): Installing...
  • Updating requests (2.21.0 /home/fredrik/.cache/pypoetry/artifacts/a2/2c/c6/5d7a41c3f8574633f2e590d06f41b47df7c5590c8ef73061d8941f8f45/requests-2.21.0-py2.py3-none-any.whl -> 2.21.0): Installing...

Installing the current project: a (1.0.0)

As you can see, that last time the output was very messy... I'm including a screenshot. Maybe this was because I was mixing poetry 1.1.5 and poetry@3900?

abn · 2021-04-09T19:00:11Z

That is expected. Because of #3876. There are followup fixes for that in #3891 and #3900.

abn · 2021-04-09T19:04:56Z

Oh, I misread the issue. Looks like there is a new issue in the lines not being updated during install. But this might not be related to the #3900 fix.

Also you will note that in 1.1.5 you will have seen;

The virtual environment found in /home/fredrik/code/repos/a/.venv seems to be broken.

This is because the new venv does not have pip in it (#2826).

dstnceswmer · 2024-02-27T20:10:34Z

Still seeing this behavior in v1.1.15

github-actions · 2024-03-29T00:22:06Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

MrGreenTea added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Sep 15, 2020

peldszus mentioned this issue Oct 31, 2020

Outdated metadata after version bump for local package #3289

Closed

3 tasks

nickray mentioned this issue Mar 1, 2021

Remove all globbed dist-infos in editable self-installs #3742

Closed

2 tasks

fredrikaverpil mentioned this issue Apr 2, 2021

Multiple installations of package when using poetry-dynamic-versioning with path install mtkennerly/poetry-dynamic-versioning#42

Closed

abn mentioned this issue Apr 9, 2021

site: use importlib instead of adhoc file searches #3900

Merged

kasteph closed this as completed in #3900 Apr 10, 2021

abn mentioned this issue Apr 13, 2021

install: ui does not update progress/completion #3934

Closed

abn removed the status/triage This issue needs to be triaged label Mar 3, 2022

glennmatthews mentioned this issue Jun 2, 2022

Add parent interface and bridge to interface serializer v1.4 nautobot/nautobot#1834

Merged

6 tasks

bryanculver mentioned this issue Jun 3, 2022

Implement Workaround for Poetry Cache CI Issues nautobot/nautobot#1848

Merged

2 tasks

hwmq mentioned this issue Oct 16, 2022

Poetry warning about duplicate *.dist-info directories which always contain one file named "REQUESTED" #6815

Closed

3 tasks

github-actions bot locked as resolved and limited conversation to collaborators Mar 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[1.1.0b2] Bumping version leaves orphaned dist-info in site-packages #2918

[1.1.0b2] Bumping version leaves orphaned dist-info in site-packages #2918

MrGreenTea commented Sep 15, 2020 •

edited

Loading

finswimmer commented Sep 15, 2020

MrGreenTea commented Sep 15, 2020

jkbecker commented Feb 2, 2021 •

edited

Loading

sinoroc commented Feb 3, 2021

ns-admetrics commented Mar 1, 2021

fredrikaverpil commented Apr 2, 2021

ns-admetrics commented Apr 7, 2021 •

edited

Loading

jkbecker commented Apr 7, 2021

sinoroc commented Apr 7, 2021 •

edited

Loading

jkbecker commented Apr 7, 2021

abn commented Apr 9, 2021

fredrikaverpil commented Apr 9, 2021 •

edited

Loading

abn commented Apr 9, 2021

abn commented Apr 9, 2021

dstnceswmer commented Feb 27, 2024

github-actions bot commented Mar 29, 2024

[1.1.0b2] Bumping version leaves orphaned dist-info in site-packages #2918

[1.1.0b2] Bumping version leaves orphaned dist-info in site-packages #2918

Comments

MrGreenTea commented Sep 15, 2020 • edited Loading

Issue

finswimmer commented Sep 15, 2020

MrGreenTea commented Sep 15, 2020

jkbecker commented Feb 2, 2021 • edited Loading

sinoroc commented Feb 3, 2021

ns-admetrics commented Mar 1, 2021

fredrikaverpil commented Apr 2, 2021

ns-admetrics commented Apr 7, 2021 • edited Loading

jkbecker commented Apr 7, 2021

sinoroc commented Apr 7, 2021 • edited Loading

jkbecker commented Apr 7, 2021

abn commented Apr 9, 2021

Using pipx

Using a container (podman | docker)

fredrikaverpil commented Apr 9, 2021 • edited Loading

abn commented Apr 9, 2021

abn commented Apr 9, 2021

dstnceswmer commented Feb 27, 2024

github-actions bot commented Mar 29, 2024

MrGreenTea commented Sep 15, 2020 •

edited

Loading

jkbecker commented Feb 2, 2021 •

edited

Loading

ns-admetrics commented Apr 7, 2021 •

edited

Loading

sinoroc commented Apr 7, 2021 •

edited

Loading

fredrikaverpil commented Apr 9, 2021 •

edited

Loading