pipenv install --lock creates broken/confusing state under some circumstances

[drakedevel]

I spent some time today trying to convert a project from requirements.txt to Pipfile with pipenv, but was confused to discover the wrong versions of some packages getting installed. Upon further investigation, I discovered that pipenv install --lock pkg1==version followed by pipenv install --lock pkg2==version, when pkg2 depends on pkg1 (with no version constraint) can result in a higher version of pkg1 being added to Pipfile.lock (but not actually installed), which then only takes effect on the next pipenv install.

Here is a terminal transcript showing the behavior with sqlalchemy==0.9.7 and graphene-sqlalchemy==1.1.1: https://gist.github.com/drakedevel/80457eca7bd2ff5fefbcb5192b67d88e

You can see that after the first install, sqlalchemy version 0.9.7 appears in Pipfile and Pipfile.lock as I'd expect. But, after graphene-sqlalchemy is installed, the Pipfile.lock contains sqlalchemy version 1.1.13, in direct conflict with the Pipfile's version constraint, and not reflecting the actual packages installed (per pipenv run pip freeze). You can see that the graphene-sqlalchemy install is satisfied with 0.9.7 and does not require an upgrade to proceed.

This seems pretty broken to me on a few levels -- what's the point of specifying version constraints in the Pipfile if they get silently overridden? And what's the point of Pipfile.lock if it doesn't reflect the actual packages pipenv installed?

[nateprewitt]

Hey @drakedevel, thanks for taking the time to put this issue together. This is a duplicate of #298 which has an open patch (#305) but we never made final decisions on what to do with conflicts and conflicting dependency specifications. We'll need to resolve that before we can move forward for a fix with this.

[nateprewitt]

I should also note this is a problem in pip itself too. Without a proper dependency resolver, we're going to have issues here. The best we can do in your situation is give a warning, but I think a lot of pinned versions aren't hard failures so we need to handle conflicts as gracefully as possible.

[kennethreitz]

Can you try this again with the latest version? I'm curious to know if our new dependency resolver will change this behavior @drakedevel :)

[drakedevel]

@nateprewitt ah yeah -- I got thrown off by the title of that one being about order. As with the initial report in #298, pip handles this (more or less accidentally) regardless of what order you have the requirements in. I do think there's a separate bug which is that the generated Pipfile.lock doesn't reflect the actually installed versions (although maybe moot now, see below).

@kennethreitz appears to be fixed! I just tried the exact same sequence of commands again, and now Pipfile, Pipfile.lock, and pip freeze all agree with the specified versions. I will give that project conversion another shot :)

[kennethreitz]

Excellent news! Bad news is that if you pass --hashes, we use the old behavior, so the bug still exists. But, we can work on that.