pip uninstall not working with editable dependencies #5193
Comments
|
I can't reproduce this with |
|
It looks like this issue is python 2 specific. |
|
Still can't reproduce: |
|
Also, still with pyhon 2.7, when I pip install with $ pip install -e git+https://github.com/pallets/click.git#egg=click --src src
$ pip uninstall click
Not uninstalling click at /home/sbi-local/tmp/src/click, outside environment /home/sbi-local/tmp/venv2
Can't uninstall 'click'. No files were found to uninstall.No such issue with python 3. |
|
@di that is weird, I can reproduce consistently with python 2. I just reproduced my instructions above in a fresh ubuntu 16.04 lxc container: $ lxc launch ubuntu:16.04 ub
$ lxc exec ub bash
# apt update && apt install python virtualenv
# virtualenv venv
# source venv/bin/activate
# pip install -e git+https://github.com/pallets/click.git#egg=click
# pip uninstall click
Can't uninstall 'click'. No files were found to uninstall.
# virtualenv --version
15.0.1
# python --version
Python 2.7.12
# pip --version
pip 10.0.0b2 from /root/venv/local/lib/python2.7/site-packages/pip (python 2.7) |
|
I've also encountered the problem, however in my case the error message is slightly different: Steps to reproduce: If I downgrade pip to version 9.0.3, I can pip uninstall editable packages in the virtualenv (~/src/pytest-catchlog is a checkout of https://github.com/eisensheng/pytest-catchlog) |
|
@mgedmin What platform is this? What version of |
|
I'm on Ubuntu 17.10. I cannot say for sure which virtualenv version I used: ~/.venv/ was created a long time ago (Sep 10 2016, if I can trust the timestamp of the 'python2.7' symlink in ~/.venv/bin). |
|
I can reproduce using the latest virtualenv: |
|
I could reproduce this in a fresh clean Docker container: https://gist.github.com/mgedmin/0d5f8cadac30270a997c3a7b3bd69bf6 |
I too was unable to reproduce this using Python 3. |
|
I can reproduce this issue under Ubuntu 16.04 with Python 2.7.12 and virtualenv 15.2.0. After downgrading to pip 9.0.3 I can once again uninstall editable packages. |
|
I've created a PR for pip-tools to show this issue on Python 2.7 / Ubuntu 16. Run |
|
Well, consider myself nerdsniped 😆 I attempted to bisect this, hope this is helpful :) #!/usr/bin/env bash
set -euxo pipefail
export PIP_DISABLE_PIP_VERSION_CHECK=1 VIRTUALENV_NO_DOWNLOAD=1
finish() {
git checkout -- .
git clean -fxfd
}
trap finish ERR EXIT
# hotfix for `AttributeError: 'module' object has no attribute 'get_python_lib'
# cherry-picked from 310bcfc78fea8f1f2bc4f680b376e18bf5aeed07
if \
grep -q '^import sysconfig$' pip/locations.py && \
grep -q 'site_packages = sysconfig.get_python_lib()$' pip/locations.py; then
sed -i 's/sysconfig.get_python_lib()/sysconfig.get_path("purelib")/g' pip/locations.py
fi
mkdir tdir
(
cd tdir
echo 'from setuptools import setup; setup(name="pkg")' > setup.py
virtualenv venv -ppython2 > /dev/null
venv/bin/pip install ../ > /dev/null || exit 125
venv/bin/pip install -e . > /dev/null || exit 125
! (venv/bin/pip uninstall -y pkg | grep 'Not uninstalling pkg')
)I had to apply the patch listed in 310bcfc early so that the bisect was able to differentiate between a bunch of "broken" commits which ended with: git bisect start
git bisect good 9.0.3
git bisect bad HEAD
git bisect run ../bisect.shAfter the bisect run, it points at this: A further triage, the failure appears to come from a differing return value in pip/src/pip/_internal/req/req_uninstall.py Line 276 in d67d98d The difference appears to be the return values here: $ venv/bin/python -c 'import sysconfig; print(sysconfig.get_path("purelib"))'
/tmp/test/venv/local/lib/python2.7/dist-packages
$ venv3/bin/python -c 'import sysconfig; print(sysconfig.get_path("purelib"))'
/tmp/test/venv3/lib/python3.5/site-packages |
|
Is there any solution for this problem? |
|
@asottile So if |
|
@cjerdonek the key difference in the two outputs is in |
|
Okay, so in other words, the following line works / gives a different answer if you substitute the distutils version? |
|
Here's the eight values: FROM debian:stretch
RUN : \
&& apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
dumb-init \
python \
python3 \
virtualenv \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists
RUN : \
&& virtualenv /venv2 -ppython2.7 \
&& virtualenv /venv3 -ppython3
RUN /bin/echo -e '#!/usr/bin/env bash \n\
set -euxo pipefail \n\
/usr/bin/python -c "from distutils import sysconfig; print(sysconfig.get_python_lib())" \n\
/usr/bin/python -c "import sysconfig; print(sysconfig.get_path('"'"'purelib'"'"'))" \n\
/usr/bin/python3 -c "from distutils import sysconfig; print(sysconfig.get_python_lib())" \n\
/usr/bin/python3 -c "import sysconfig; print(sysconfig.get_path('"'"'purelib'"'"'))" \n\
/venv2/bin/python -c "from distutils import sysconfig; print(sysconfig.get_python_lib())" \n\
/venv2/bin/python -c "import sysconfig; print(sysconfig.get_path('"'"'purelib'"'"'))" \n\
/venv3/bin/python -c "from distutils import sysconfig; print(sysconfig.get_python_lib())" \n\
/venv3/bin/python -c "import sysconfig; print(sysconfig.get_path('"'"'purelib'"'"'))" \n\
' > t.sh
CMD ["dumb-init", "bash", "t.sh"]$ docker build -q -t test . && docker run --rm -ti test
sha256:b10eb0cb39a9a9e9c641a7496e1447047b850852f1573038d3fc43280a5961ef
+ /usr/bin/python -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/usr/lib/python2.7/dist-packages
+ /usr/bin/python -c 'import sysconfig; print(sysconfig.get_path('\''purelib'\''))'
/usr/local/lib/python2.7/dist-packages
+ /usr/bin/python3 -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/usr/lib/python3/dist-packages
+ /usr/bin/python3 -c 'import sysconfig; print(sysconfig.get_path('\''purelib'\''))'
/usr/lib/python3.5/site-packages
+ /venv2/bin/python -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/venv2/lib/python2.7/site-packages
+ /venv2/bin/python -c 'import sysconfig; print(sysconfig.get_path('\''purelib'\''))'
/venv2/local/lib/python2.7/dist-packages
+ /venv3/bin/python -c 'from distutils import sysconfig; print(sysconfig.get_python_lib())'
/venv3/lib/python3.5/site-packages
+ /venv3/bin/python -c 'import sysconfig; print(sysconfig.get_path('\''purelib'\''))'
/venv3/lib/python3.5/site-packages |
|
Thanks. It looks like not just one but three of the four pairs have different return values (and for different reasons), with only the last the same. So does this mean that one (or perhaps both) of the two functions |
|
Debian system python installs a symlink to make this work there
…On Fri, Aug 10, 2018, 3:56 AM Chris Jerdonek ***@***.***> wrote:
Thanks. It looks like not just one but three of the four pairs have
different return values (and for different reasons), with only the last the
same. So does this mean that one (or perhaps both) of the two functions
sysconfig.get_path() and distutils.sysconfig.get_python_lib() has a bug,
or are they not supposed to behave the same?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5193 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABugn7x9otsHDQrC3p8ThqiFW1wWqKKOks5uPWbHgaJpZM4TLm1R>
.
|
|
My question was whether the two functions are always supposed to have the same return value. |
|
I would imagine that they are intended to give the same result (I think sysconfig was added to the stdlib to pull certain generally-useful bits out of distutils into a more generic location). But Debian's messing round with system paths always ends up causing us problems. In this case, it may be that the amount of Debian specific special-casing already in virtualenv isn't sufficient to catch this case. |
|
I propose a workaround in #6918 (a similar workaround is already in place for pypy). |
sbidoul
changed the title
triage-new-issues
bot
removed
the
S: needs triage
pradyunsg
added
project: <downstream>
|
@kitterma This seems to be an issue introduced by Debian's patch for |
|
I took a look a this on Debian stable as well as unstable/testing. Click is now python3 only so the original problem with that repository can't occur. I tried the reproduction steps on python3 and it uninstalled fine. I don't know how much effort it's worth fixing python2 stuff now, but if someone will provide another example where the problem still occurs I will take a look at it. |
|
@pradyunsg this was caused by a change in pip fwiw: #5193 (comment) |
|
I think the main issue is why a platform incorrectly returns |
|
🤷 it used to work though is my point |
|
As I said earlier, if someone can give me a current example showing the problem, I'll look into it.
|
|
@kitterma this should reproduce on buster. $ mkdir editabletest
$ cd editabletest
$ echo "import setuptools; setuptools.setup(name='editabletest')" > setup.py
$ virtualenv -p python2 venv
$ venv/bin/pip install -e .
$ venv/bin/pip uninstall editabletest
...
Not uninstalling editabletest at /root/editabletest, outside environment /root/editabletest/venv
... |
|
I thought about this a bit more. Python has two ways to get information on where to install stuff: pip should be enforcing the modern scheme when it installs stuff, but can likely be more lenient when uninstalling. It could be possible for the uninstallation logic to try figure out which scheme a package is installed with, and use the right one automatically. This is worthwhilte to investigate IMO since it is still quite common for packages to be installed with the legacy scheme, either by an old pip (9.x is still in active use!) or an installation method that has not been modernised (e.g. editable). I’ll probably take a deeper look at the implementation on Sunday if nobody else figures this out before that. |
Thanks. Same thing happens on Unstable/Testing with python2.7. With python3.8 it works fine. |
|
Looking into this a bit more. On Debian Unstable, inside a python2 based virtualenv I see: In a python3 virtualenv it's different: So it's pretty clear why python3 works and python2 doesn't. Even though this may have be precipitated by a pip change, I think downstream is the correct resolution since it's connected to a Debian specific Python interpreter change (which isn't going to go away). |
|
Just to confirm, has anyone in this thread tried to file an issue to Debian? |
|
On Saturday, May 23, 2020 8:06:13 PM EDT Tzu-ping Chung wrote:
Just to confirm, has anyone in this thread tried to file an issue to Debian?
Since I'm the primary maintainer for virtualenv and pip in Debian right now,
it wouldn't make me any more aware of it than I am now.
Also, don't assume the previous discussion is right. I tried a patch very
similar to the linked pull request and it didn't solve the problem. That may
be on me for either not getting the change right or not testing correctly, but
I'm not 100% sure the previous discussion is on the right path.
|
|
I see, thanks @kitterma. |
|
@kitterma thanks for your interest in this issue. $ mkdir editabletest
$ cd editabletest
$ echo "import setuptools; setuptools.setup(name='editabletest')" > setup.py
$ virtualenv -p python2 venv
$ venv/bin/pip install https://github.com/sbidoul/pip/archive/editable-uninstall-2.7-sbi.zip
$ venv/bin/pip install -e .
$ venv/bin/pip uninstall editabletest
Found existing installation: editabletest 0.0.0
Uninstalling editabletest-0.0.0:
Would remove:
/.../editabletest/venv/lib/python2.7/site-packages/editabletest.egg-link
Proceed (y/n)? y
Successfully uninstalled editabletest-0.0.0 |
|
I can confirm it too now. I'm not sure what I was doing wrong before. Thanks. I will include this in the next pip upload for Debian Unstable, which means it will support Debian Bullseye. Since the Buster (Debian 10) version of virtualenv always uses the latest version of pip from pypi, I don't think there's any benefit to me backporting the patch (please let me know if that's wrong). |
|
I can confirm I have the bug on python2.7. What's the status of this ? @kitterma |
This issue is marked as "python 2 only". pip 21.0 dropped support for Python 2. Should this be closed? |
Description:
After installing an editable dependency from vcs, it cannot be uninstalled.
With pip 9 it did uninstall correctly, removing the .egg-link file.
What I've run:
I expected click to be uninstalled at this point.
If I downgrade to pip 9.0.3, it works:
The text was updated successfully, but these errors were encountered: