Dependency resolution and package instalation behaves wrongly when we have packages and dev-packages in Pipfile

[frenetic]

This bug is, somehow, related to https://github.com/kennethreitz/pipenv/issues/298

Take this simple and small Pipfile as example:

[packages]
Django = "==1.8.18"

[dev-packages]
model_mommy = "==1.2.6"

Running pipenv install --dev will install Django on its latest version.

Generated Pipfile.lock

{
    "_meta": {
        "hash": {
            "sha256": "76050c2c22d3ffbb4e6a7de1e38ebb380aa0d2e6b8fda5853b798061ed79ef63"
        },
        "host-environment-markers": {
            "implementation_name": "cpython",
            "implementation_version": "0",
            "os_name": "posix",
            "platform_machine": "x86_64",
            "platform_python_implementation": "CPython",
            "platform_release": "4.10.0-33-generic",
            "platform_system": "Linux",
            "platform_version": "#37~16.04.1-Ubuntu SMP Fri Aug 11 14:07:24 UTC 2017",
            "python_full_version": "2.7.12",
            "python_version": "2.7",
            "sys_platform": "linux2"
        },
        "pipfile-spec": 3,
        "requires": {},
        "sources": [
            {
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "django": {
            "hashes": [
                "sha256:d8e2fd119756ab10b43a31052c3c8efbc262064b81eecb7871372de4d37b1a94",
                "sha256:c7611cdd5e2539a443b7960c7cafd867d986c2720a1b44808deaa60ce3da50c7"
            ],
            "version": "==1.8.18"
        }
    },
    "develop": {
        "django": {
            "hashes": [
                "sha256:89162f70a74aac62a53f975128faba6099a7ef2c9d8140a41ae9d6210bda05cd",
                "sha256:1836878162dfdf865492bacfdff0321e4ee8f1e7d51d93192546000b54982b29"
            ],
            "version": "==1.11.5"
        },
        "model-mommy": {
            "hashes": [
                "sha256:84e994e91041b657b8d23c5fbbe897b914fac499507f6a21f1ae991c23883396"
            ],
            "version": "==1.2.6"
        },
        "pytz": {
            "hashes": [
                "sha256:c883c2d6670042c7bc1688645cac73dd2b03193d1f7a6847b6154e96890be06d",
                "sha256:03c9962afe00e503e2d96abab4e8998a0f84d4230fa57afe1e0528473698cdd9",
                "sha256:487e7d50710661116325747a9cd1744d3323f8e49748e287bc9e659060ec6bf9",
                "sha256:43f52d4c6a0be301d53ebd867de05e2926c35728b3260157d274635a0a947f1c",
                "sha256:d1d6729c85acea5423671382868627129432fba9a89ecbb248d8d1c7a9f01c67",
                "sha256:54a935085f7bf101f86b2aff75bd9672b435f51c3339db2ff616e66845f2b8f9",
                "sha256:39504670abb5dae77f56f8eb63823937ce727d7cdd0088e6909e6dcac0f89043",
                "sha256:ddc93b6d41cfb81266a27d23a79e13805d4a5521032b512643af8729041a81b4",
                "sha256:f5c056e8f62d45ba8215e5cb8f50dfccb198b4b9fbea8500674f3443e4689589"
            ],
            "version": "==2017.2"
        },
        "six": {
            "hashes": [
                "sha256:0ff78c403d9bccf5a425a6d31a12aa6b47f1c21ca4dc2573a7e2f32a97335eb1",
                "sha256:105f8d68616f8248e24bf0e9372ef04d3cc10104f1980f54d57b2ce73a5ad56a"
            ],
            "version": "==1.10.0"
        }
    }
}

Notice how, inside develop we have Django at version 1.11.5.
When we install with --dev it is overriding production requirements.

[frenetic]

How I think pipenv should behave:

1. Generate Pipfile.lock for packages first, instead of doing this for dev-packages first.
2. When generating the Pipfile.lock for dev-packages, check if it is trying to install a packages dependency. Check if the packages requirement meets the dev-packages requirement.

[kennethreitz]

This looks right to me, based on the current logic. They are treated as two independent lists.

[frenetic]

I was reading the code and this is how the code behaves.
From this point of view, it does look right and it is not code bug.

However, at the end user point of view, it does not make any sense.

Development dependencies should not override your production dependencies.
If this happens, developer machines/dockers/vagrants will be in a different consistency state from production, leading to bugs.

[kennethreitz]

I think we can resolve this by simply reading dev dependencies first, and overwriting them with default dependencies.

[kennethreitz]

fixed in master

[techalchemy]

BTW isn't this a duplicate of #298 and is that fixed as well now?

[kennethreitz]

looks like it

[techalchemy]

Nice, I had this flagged to fix whenever I had a chance

[frenetic]

Thank you all for your time and effort.

[kageurufu]

I believe this has regressed, I am able to reproduce this under pipenv 9.0.3

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
sqlalchemy = "==1.1.15"

[dev-packages]
flask-sqlalchemy = "*"

results in sqlalchemy==1.2.1 being installed and in develop section of Pipfile.lock