Poetry includes a package in its own dependencies

[maksbotan]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS 11.6.2 Big Sur
• Poetry version: 1.1.13
• Link of a Gist with the contents of your pyproject.toml file: private, sorry. If it's relevant, I will think about a way to show it.

Issue

I've discovered a curious issue, when Poetry thinks that a package (one of root's dependencies) depends on itself. Extract from poetry lock --no-update:

PyPI: Getting info for pandorabox (0.4.21) from PyPI
   1: fact: pandorabox (0.4.21) depends on pandorabox (0.4.21)
   1: fact: pandorabox (0.4.21) depends on biopython (>=1.74,<2.0)
   1: fact: pandorabox (0.4.21) depends on dataclasses (0.6)
   1: fact: pandorabox (0.4.21) depends on importlib-metadata (>=1.0,<2.0)
   1: fact: pandorabox (0.4.21) depends on loguru (>=0.5,<0.6)
   1: fact: pandorabox (0.4.21) depends on oneq (>=1.10.2,<2.0.0)
   1: fact: pandorabox (0.4.21) depends on pydantic (>=1.6.1,<2.0.0)
   1: fact: pandorabox (0.4.21) depends on python_codon_tables (>=0.1.8,<0.2.0)
   1: selecting pandorabox[codon_optimization] (0.4.21)
   1: derived: python_codon_tables (>=0.1.8,<0.2.0)
   1: derived: oneq (>=1.10.2,<2.0.0)
   1: derived: importlib-metadata (>=1.0,<2.0)
   1: derived: dataclasses (==0.6)
   1: derived: pandorabox (==0.4.21)

Here you can see that poetry considers pandorabox (our private package) dependency, discovers its dependencies (from private PyPI) and selects it. However, for some reason poetry thinks that the package depends on itself.

Later on this package is considered and resolved again:

PyPI: Getting info for pandorabox (0.4.21) from PyPI
   1: fact: pandorabox (0.4.21) depends on dataclasses (0.6)
   1: fact: pandorabox (0.4.21) depends on importlib-metadata (>=1.0,<2.0)
   1: fact: pandorabox (0.4.21) depends on loguru (>=0.5,<0.6)
   1: fact: pandorabox (0.4.21) depends on oneq (>=1.10.2,<2.0.0)
   1: fact: pandorabox (0.4.21) depends on pydantic (>=1.6.1,<2.0.0)
   1: selecting pandorabox (0.4.21)

This time no self-dependency arises.

By adding some print() calls in poetry code I discovered that the first time pandorabox is considered, it carries [codon_optimization] extra, as specified in our pyproject.toml file. Second time it does not.

Exact dependency line from pyproject.toml is here:

pandorabox = {version = "^0.4.20", optional = true, extras = ["codon_optimization"]}

[dimbleby]

Sounds like it's just saying that package[extra] depends on package, which sounds entirely reasonable.

Is there an actual problem here?

[maksbotan]

Well, it does not sound reasonable to me. Can you provide a justification? 🤔

One possible problem is that between the two times the package is considered, it has different specification, missing source, resolved_ref and stuff in one of the cases. Not sure if I remember this correctly though. I can try to investigate.

[dimbleby]

One possible problem ...

I was asking whether you had seen an actual problem - sounds like not?

As far as I know the logs that you are reporting are completely normal when poetry has a package with extras. Of course if you can show that something is going wrong, that would be different...

[charbonnierg]

I don't know how poetry handles this, but it made me think of this example from resolvelib

Python package dependencies can include "extras", which are additional
dependencies that are installed "on demand". For instance, project X could
have an additional set of dependencies if PDF generation features are needed.
These can be defined for an extra "pdf" and requested on install as X[pdf].
The basic resolvelib algorithm cannot handle extras, as it builds a dependency
graph which needs to be static - the edges (dependencies) from a node
(candidate) must be fixed. Extras break this assumption.
To model projects with extras, we define a candidate as being a project with a
specific set of dependencies. This introduces a problem, as the resolver could
produce a solution that demands version 1.0 of X[foo] and version 2.0 of
X[bar]. This is impossible, as there is actually only one project X to be
installed. To address this, we inject an additional dependency for every
candidate with an extra - X[foo] version v depends on X version v. By doing
this, we constrain the solution to require a unique version of X.

[dimbleby]

#2887

The main trick here is that a package with extras will now have its non-extra version as a dependency to help the resolver figure out what needs to be done.

@maksbotan please close

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.