poetry update <single-pkg> updates a different package with extras specification

[immerrr]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 16.04
• Poetry version: 1.1.2
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/immerrr/0d55d8958683d26a024153cade39b266

Issue

It looks like packages with extras circumvent the whitelisting restriction when updating a single package with

poetry update <single-pkg>

Here's a repro:

• in the pyproject.toml linked above, change watchdog version to be strictly 0.10.2, and do poetry update
• now, revert the watchdog version requirement to be ^0.10.2, but this time do poetry update decorator
• expected behaviour: poetry does not change anything, as there is no new version of decorator package
• actual behaviour:

watchdog is updated to `0.10.3`

$ poetry -V
Poetry version 1.1.2
$ poetry show --tree
decorator 4.4.2 Decorators for Humans
watchdog 0.10.2 Filesystem events monitoring
├── argh >=0.24.1
├── pathtools >=0.1.1
└── pyyaml >=3.10
$ poetry update decorator -vvv
Using virtualenv: /mnt/extraspace/virtualenvs/poetry-test-TzTtOsUh-py3.7
Updating dependencies
Resolving dependencies...
   1: fact: poetry-test is 0.1.0
   1: derived: poetry-test
   1: fact: poetry-test depends on watchdog[watchmedo] (^0.10.2)
   1: fact: poetry-test depends on decorator (^4.4.2)
   1: selecting poetry-test (0.1.0)
   1: derived: decorator (^4.4.2)
   1: derived: watchdog[watchmedo] (^0.10.2)
PyPI: No release information found for watchdog-0.3.4, skipping
PyPI: No release information found for watchdog-0.3.6, skipping
PyPI: No release information found for watchdog-0.6.1, skipping
PyPI: 2 packages found for watchdog >=0.10.2,<0.11.0
PyPI: No release information found for decorator-3.4.1, skipping
PyPI: No release information found for decorator-4.0.8, skipping
PyPI: 1 packages found for decorator >=4.4.2,<5.0.0
   1: selecting decorator (4.4.2)
   1: fact: watchdog[watchmedo] (0.10.3) depends on watchdog (0.10.3)
   1: fact: watchdog[watchmedo] (0.10.3) depends on pathtools (>=0.1.1)
   1: fact: watchdog[watchmedo] (0.10.3) depends on PyYAML (>=3.10)
   1: fact: watchdog[watchmedo] (0.10.3) depends on argh (>=0.24.1)
   1: selecting watchdog[watchmedo] (0.10.3)
   1: derived: argh (>=0.24.1)
   1: derived: PyYAML (>=3.10)
   1: derived: pathtools (>=0.1.1)
   1: derived: watchdog (0.10.3)
PyPI: No release information found for watchdog-0.3.4, skipping
PyPI: No release information found for watchdog-0.3.6, skipping
PyPI: No release information found for watchdog-0.6.1, skipping
PyPI: 1 packages found for watchdog 0.10.3
   1: selecting argh (0.26.2)
   1: selecting pyyaml (5.3.1)
   1: selecting pathtools (0.1.2)
   1: fact: watchdog (0.10.3) depends on pathtools (>=0.1.1)
   1: selecting watchdog (0.10.3)
   1: Version solving took 0.030 seconds.
   1: Tried 1 solutions.

Writing lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 1 update, 0 removals

  • Updating watchdog (0.10.2 -> 0.10.3): Pending...
  • Updating watchdog (0.10.2 -> 0.10.3): Installing...
  • Updating watchdog (0.10.2 -> 0.10.3)

What leads me to believe this is somehow related to the extras specification, is that ...

... if I remove the extras and repeat the repro with just `watchdog = {version = "^0.10.2"}`, watchdog is *not* updated to 0.10.3 which is the expected behaviour.

$ poetry -V
Poetry version 1.1.2
$ poetry show --tree
decorator 4.4.2 Decorators for Humans
watchdog 0.10.2 Filesystem events monitoring
└── pathtools >=0.1.1
$ poetry update decorator -vvv
Using virtualenv: /mnt/extraspace/virtualenvs/poetry-test-TzTtOsUh-py3.7
Updating dependencies
Resolving dependencies...
   1: fact: poetry-test is 0.1.0
   1: derived: poetry-test
   1: fact: poetry-test depends on watchdog (^0.10.2)
   1: fact: poetry-test depends on decorator (^4.4.2)
   1: selecting poetry-test (0.1.0)
   1: derived: decorator (^4.4.2)
   1: derived: watchdog (^0.10.2)
PyPI: No release information found for decorator-3.4.1, skipping
PyPI: No release information found for decorator-4.0.8, skipping
PyPI: 1 packages found for decorator >=4.4.2,<5.0.0
   1: selecting decorator (4.4.2)
   1: fact: watchdog (0.10.2) depends on pathtools (>=0.1.1)
   1: selecting watchdog (0.10.2)
   1: derived: pathtools (>=0.1.1)
   1: selecting pathtools (0.1.2)
   1: Version solving took 0.017 seconds.
   1: Tried 1 solutions.

Finding the necessary packages for the current system

No dependencies to install or update

[skylerwshaw]

@immerrr I found your issue after experiencing the same but was able to resolve it with the following:

• Run poetry lock --no-update, which was required in my case because my poetry.lock file was generated by a Poetry version earlier than 1.1. This regenerates the lock file with no version changes, a migration of lockfile formatting of sorts. Commit this update.
• Now run poetry update <single-pkg>. You'll likely find that only the one dependency updates, including its child dependencies.

Without digging, I think this happens because the poetry update command in 1.1+ can't read your old-format lockfile so regenerates the lockfile without awareness of the old lockfile's version locks.

cc @finswimmer

[immerrr]

@skylerwshaw that's good that at least in some real-life examples it is possible to work this around. However, please note that the repro included in the original message cannot have an old-format lockfile, because it only includes pyproject.toml, and the lockfile is then automatically generated from scratch by poetry v1.1.2.

[immerrr]

Btw, I have just retried the same repro with v1.1.4, because I had hope that the new fixes for extra dependencies would help, but looks like the behaviour is still the same, poetry updates watchdog even if I only request updates for decorator. And I can delete poetry.lock altogether to avoid any issues with the stale format:

$ rm poetry.lock
$ poetry update decorator -vvv
Using virtualenv: /mnt/extraspace/virtualenvs/poetry-test-TzTtOsUh-py3.7
Updating dependencies
Resolving dependencies...
   1: fact: poetry-test is 0.1.0
   1: derived: poetry-test
   1: fact: poetry-test depends on watchdog[watchmedo] (^0.10.2)
   1: fact: poetry-test depends on decorator (^4.4.2)
   1: selecting poetry-test (0.1.0)
   1: derived: decorator (^4.4.2)
   1: derived: watchdog[watchmedo] (^0.10.2)
PyPI: No release information found for watchdog-0.3.4, skipping
PyPI: No release information found for watchdog-0.3.6, skipping
PyPI: No release information found for watchdog-0.6.1, skipping
PyPI: 5 packages found for watchdog >=0.10.2,<0.11.0
PyPI: No release information found for decorator-3.4.1, skipping
PyPI: 1 packages found for decorator >=4.4.2,<5.0.0
   1: selecting decorator (4.4.2)
   1: fact: watchdog[watchmedo] (0.10.6) requires Python >=2.7, <3.6
   1: derived: not watchdog[watchmedo] (0.10.6)
   1: fact: watchdog[watchmedo] (0.10.5) requires Python >=2.7, <3.6
   1: derived: not watchdog[watchmedo] (0.10.5)
   1: fact: watchdog[watchmedo] (0.10.4) depends on watchdog (0.10.4)
   1: fact: watchdog[watchmedo] (0.10.4) depends on pathtools (>=0.1.1)
   1: fact: watchdog[watchmedo] (0.10.4) depends on PyYAML (>=3.10)
   1: fact: watchdog[watchmedo] (0.10.4) depends on argh (>=0.24.1)
   1: selecting watchdog[watchmedo] (0.10.4)
   1: derived: argh (>=0.24.1)
   1: derived: PyYAML (>=3.10)
   1: derived: pathtools (>=0.1.1)
   1: derived: watchdog (0.10.4)
PyPI: No release information found for argh-0.9.0, skipping
PyPI: 5 packages found for argh >=0.24.1
PyPI: No release information found for pyyaml-3.01, skipping
PyPI: No release information found for pyyaml-3.02, skipping
PyPI: No release information found for pyyaml-3.03, skipping
PyPI: No release information found for pyyaml-3.04, skipping
PyPI: No release information found for pyyaml-3.05, skipping
PyPI: No release information found for pyyaml-3.06, skipping
PyPI: No release information found for pyyaml-3.07, skipping
PyPI: No release information found for pyyaml-3.08, skipping
PyPI: No release information found for pyyaml-3.09, skipping
PyPI: 12 packages found for pyyaml >=3.10
PyPI: 2 packages found for pathtools >=0.1.1
PyPI: No release information found for watchdog-0.3.4, skipping
PyPI: No release information found for watchdog-0.3.6, skipping
PyPI: No release information found for watchdog-0.6.1, skipping
PyPI: 1 packages found for watchdog 0.10.4
   1: fact: watchdog (0.10.4) depends on pathtools (>=0.1.1)
   1: selecting watchdog (0.10.4)
   1: selecting pathtools (0.1.2)
   1: selecting argh (0.26.2)
   1: selecting pyyaml (5.4.1)
   1: Version solving took 0.060 seconds.
   1: Tried 1 solutions.

Writing lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 1 update, 0 removals

  • Updating watchdog (0.10.2 -> 0.10.4): Pending...
  • Updating watchdog (0.10.2 -> 0.10.4): Installing...
  • Updating watchdog (0.10.2 -> 0.10.4)

$ poetry --version
Poetry version 1.1.4

[Agalin]

Any update on this? On 1.1.6 not only does update <package> change version of a different package with extras but lock --no-update uptades it as well.

[stuaxo]

One thing I'd note, is that on systems that do this, it's possible to get into some pretty tricky situations where it's hard to work out what you can update, I've certainly experienced this with pip-tools/pip-compile and on ruby with rake.

[radoering]

Relates to #4612. The attached PR seems to resolve this issue, too.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.