Poetry repeatedly re-queries PyPI for non-PyPI dependencies when resolving

[DFEvans]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• OS version and name: Windows 10 (in WSL2 Ubuntu)

• Poetry version: 1.1.13

Issue

When resolving dependencies (e.g. during a poetry add), Poetry will query PyPI for some package information, and cache the result, preventing future queries.

However, if a package is available via an alternative source defined in the pyproject.toml, Poetry will query PyPI for every iteration of the resolving algorithm, receiving a 404 each time. With enough non-PyPI dependencies, and enough iterations of the resolver, PyPI will eventually return an error stating that the caller needs to back off (having hit their endpoints with hundreds of failing queries).

At one level, this is because Poetry does not cache failed responses - if PyPiRepository._get_release_info fails, a PackageNotFound exception is raised which bubbles up beyond the caching layer.

It is still surprising that Poetry is querying PyPI at all, though, given that my dependencies have source = xxx defined. It looks like Poetry should be querying only the specified source repository for packages with this specified, but repository is coming through as None here, and it's falling back on enumerating all repositories in priority order in the else block. The trail gets quite confusing to follow from here on, but it appears that the various source_... attributes on the dependency object are not being propagated from the pyproject.toml / poetry.lock.

[lalmei]

Yeah saw similar behavior with default=true for a private repository.

[neersighted]

This is currently as designed -- see #6713 for a proposal to introduce the behavior you desire/expect.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.