Improve detection of installed packages #1786
Conversation
| if site_packages: | ||
| self._site_packages = site_packages[0] | ||
| else: | ||
| self._site_packages = dist_packages[0] |
There was a problem hiding this comment.
I don't know where the site_packages property is used, so it isn't obvious to me, why only the first site_package is stored and why dist-packages are never stored when site_packages where found.
There was a problem hiding this comment.
The site_packages property is used in the InstalledRepository class (see https://github.com/python-poetry/poetry/blob/master/poetry/repositories/installed_repository.py#L32).
We only need the "site-packages" directory where pip will install the dependencies which is, as far as I know, the site-packages in virtual environments and most OS and dist-packages for Debian-based distributions.
There was a problem hiding this comment.
Ah, ok I got it. I wonder if something with sysconfig would be more reliable? But as far as I can see, we must be able to detect if we are inside a virtual environment and of we are on a posix or nt system, to get the correct path (purelib seems to be the one we are interested in), are we?
There was a problem hiding this comment.
Thanks for pointing me to sysconfig! Do you know if there are some systems where purelib and platlib are different? If there is none we could indeed use purelib and avoid the heuristics we are currently doing.
There was a problem hiding this comment.
I'm not sure. I've just discovered that module as well. There is something in PEP427 about the difference.
There was a problem hiding this comment.
In the end, it seems that all installed packages will reside in purelib and/or platlib, so to get populate the InstalledRepository we can just check both, I think. I'll test it out and see where it goes.
There was a problem hiding this comment.
After testing it out it seems that using sysconfig won't help us. While it works in some cases, for Debian-based systems, for instance, it won't return the directory we need. We could use site.getsitepackages() for non-virtual environments but we would need to find a way to replicate it for virtual environments which would lead to using the same heuristic this PR already uses.
I think we can leave the PR as-is for now since I tested it on Ubuntu, MacOS and Windows without any issue and it fixes the issues I mentioned above.
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Pull Request Check List
In version 1.0.0, we changed the way the installed packages are detected to improve the detection of the type of packages (directory vs git dependencies for instance). However, this introduced issues where Poetry always reinstalled the packages because it was detecting the wrong type of package or because it was not retrieving it from the correct directory. This PR should fix most of these issues.
Fixes #1711
Fixes #1612