[10.0.0beta] Installation emits extra warnings for unrelated packages

[di]

• Pip version: 516adb4 (current release/10.0.0 HEAD)
• Python version: 3.6.4
• Operating system: Darwin 17.5.0 x86_64 i386

Description:

Running pip install <package> will emit warnings about missing dependencies for unrelated packages (essentially, the output of pip check). I would expect to see warnings only related to the package I'm attempting to install.

This seems similar to some fixed issues (#5134, #5160, #5195) but still occurs after the fixes for those issues.

What I've run:

The actual packages used don't matter: when a package A has unmet dependency B, installing C (with no shared dependencies with A) will warn that B is missing.

$ pip install freezegun
Collecting freezegun
  Using cached freezegun-0.3.10-py2.py3-none-any.whl
Collecting six (from freezegun)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting python-dateutil!=2.0,>=1.0 (from freezegun)
  Using cached python_dateutil-2.7.2-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, freezegun
Successfully installed freezegun-0.3.10 python-dateutil-2.7.2 six-1.11.0

$ pip uninstall six
Uninstalling six-1.11.0:
  Would remove:
    /private/tmp/env/lib/python3.6/site-packages/six-1.11.0.dist-info/*
    /private/tmp/env/lib/python3.6/site-packages/six.py
Proceed (y/n)? y
  Successfully uninstalled six-1.11.0

$ pip install black
Collecting black
  Using cached black-18.4a0-py36-none-any.whl
Collecting attrs>=17.4.0 (from black)
  Using cached attrs-17.4.0-py2.py3-none-any.whl
Collecting click (from black)
  Using cached click-6.7-py2.py3-none-any.whl
python-dateutil 2.7.2 requires six>=1.5, which is not installed. <============== I don't expect to see this line
freezegun 0.3.10 requires six, which is not installed.           <============== I don't expect to see this line
Installing collected packages: attrs, click, black
Successfully installed attrs-17.4.0 black-18.4a0 click-6.7

[cjerdonek]

Does this occur with 9.x, too?

[pradyunsg]

Hey @di!

This is intentional - to cover a user with an already broken environment, by telling them that they have a broken dependency if the operation they perform doesn't fix it.

I'm curious, is there some reason that you don't want to know that there's a broken dependency?

@cjerdonek This is new behaviour in pip 10. #5000

[di]

@pradyunsg I guess I was just surprised by it since it's new. This warning made me think that something went wrong with the command I had run, when in fact I had just botched up my environment days ago, and was just seeing the results much later.

Fine with me if this was planned all along and should be closed, I just think it's probably going to catch a lot of people off guard when they see it for the first time.

[cjerdonek]

I looked at the original PR, and doesn’t its news item suggest that only errors about what’s being installed will now be printed?

pip install now prints an error message when it installs an incompatible

version of a dependency. Otherwise, shouldn’t it say something like, “when it finds an error in the dependencies in your environment”?

…

On Tue, Apr 10, 2018 at 9:18 AM Dustin Ingram ***@***.***> wrote: @pradyunsg <https://github.com/pradyunsg> I guess I was just surprised by it since it's new. This warning made me think that something went wrong with the command I had run, when in fact I had just botched up my environment days ago, and was just seeing the results much later. Fine with me if this was planned all along and should be closed, I just think it's probably going to catch a lot of people off guard when they see it for the first time. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5196 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVt7r5ZN14seezKPwPoSiI6YBrVYkGkks5tnNs9gaJpZM4TNJZS> .

[pfmoore]

Until this issue came up, I never really considered the possibility of unrelated incompatibilities being flagged when something was installed. I doubt it's a particularly common situation - once your environment has no broken dependencies, anything new that gets flagged will be because of what you've just installed - but I agree the message is a little surprising if it's reporting an existing issue.

I don't think it's a major issue, personally - and in the long run we'll be getting a proper resolver which will avoid such broken dependencies in the first place - so I don't think it needs to be a release blocker. If someone wants to submit a PR fixing this for pip 10.1, I'd be fine with that.

[di]

I will agree that my various environments are probably uncommon -- I usually have a few unmet or broken dependencies. 🙂

Even with a proper resolver, though, won't this still be possible to reproduce? In the example above, what would change? Would pip uninstall refuse to uninstall the dependency without installing what depends on it? Or would it just uninstall everything that depends on it? Or would it work just like it does now?

[pfmoore]

I've added an issue against the resolver project to consider how to handle that gracefully. You're right that people can have unmet or broken dependencies. My assumption is that when they do, it's because they didn't realise it, and they would fix it once informed. And in the context of a full resolver, pip itself would in effect make sure that once an environment is "good", it will stay that way But there's certainly a usability question over how pip install foo with a full resolver should should handle such broken environments when first encountered - "helpfully" tidying up unrelated issues is a bit presumptuous, but conversely, not doing so means that we can't state unequivocally that pip won't leave you with a broken environment.

Going back to the current situation, I guess a key question is - when you saw those warnings (not in your test case, but in your actual environment), were you motivated to fix them, or did you want to leave things as they were? And if the latter, was that because you didn't believe the warnings mattered ("I'm sure it will work anyway") or something else (I find it hard to think of another reason you'd want to leave your environment broken, apart from "I don't want to deal with this now" - but that's fine, you're motivated to fix them, but we're not forcing you to do so immediately).

[di]

Well, I wasn't exactly in prod, so once I got past "Huh, why didn't that work" and "Wait, why does X even depend on Y?", I didn't care, but mostly because it was just a one-off environment that I was doing some testing in.

If it had been a more important environment I might have cared, but usually I'm either a) not pip installing willy nilly in said environments, or b) not actually looking at the output of pip install unless it has failed with an exit code. So I think this warning is either going to confuse me unnecessarily or just never get noticed, but again, I realize that I might not be the typical user.

[pfmoore]

Thanks for the feedback - it's definitely something we should monitor in real use, so it's good to get people's reactions. (And personally, I'm not that convinced you're particularly atypical - I suspect my reaction would be similar).

[cjerdonek]

My 2 cents: I don’t think “pip install foo” should report errors about dependencies not in foo’s dependency graph, unless perhaps it’s in a verbose mode and pip needs to compute that information anyways. For example, it could get annoying if you have a known issue with a dependency in your environment, and you’re individually installing a number of things. It could also serve to distract from any errors that _are_ a result of what you’re installing.

…

On Tue, Apr 10, 2018 at 12:46 PM Dustin Ingram ***@***.***> wrote: Well, I wasn't exactly in prod, so once I got past "Huh, why didn't that work" and "Wait, why does X even depend on Y?", I didn't care, but mostly because it was just a one-off environment that I was doing some testing in. If it had been a more important environment I might have cared, but usually I'm either a) not pip installing willy nilly in said environments, and b) not actually looking at the output of pip install unless it has failed with an exit code. So I think this warning is either going to confuse me unnecessarily or just never get noticed, but again, I realize that I might not be the typical user. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5196 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVt7ulc-sb-0khNOzL-LeHGuSegkTY8ks5tnQwLgaJpZM4TNJZS> .

[ben-albrecht]

This is intentional - to cover a user with an already broken environment, by telling them that they have a broken dependency if the operation they perform doesn't fix it.

I'm curious, is there some reason that you don't want to know that there's a broken dependency?

As another use-case, our project builds packages in a local virtualenv as part of the installation. During the build, users will get warnings about broken package dependencies in their pip environment, which is irrelevant to the local virtualenv build that is taking place and creates confusion as to whether the warnings are related to installing the software or not.

I agree that these warnings are generally helpful, but it would be nice to have a way to suppress them for specific use-cases (I could not find a flag in the help output, if it already exists).

[pradyunsg]

There's a --no-warn- ​<​ something ​>​ flag I'd added for this in the same PR. Would a separate option to specify packages to ignore (when they occurs as dependencies) would be useful?

…

On Wed, 18 Apr 2018, 22:14 Ben Albrecht, ***@***.***> wrote: This is intentional - to cover a user with an already broken environment, by telling them that they have a broken dependency if the operation they perform doesn't fix it. I'm curious, is there some reason that you don't want to know that there's a broken dependency? As another use-case, our project builds packages in a local virtualenv as part of the installation. During the build, users will get warnings about broken package dependencies in their pip environment, which is irrelevant to the local virtualenv build that is taking place and creates confusion as to whether the warnings are related to installing the software or not. I agree that these warnings are generally helpful, but it would be nice to have a way to suppress them for specific use-cases (I could not find an existing flag, if it already exists). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5196 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADH7ScsXYLc2kHxx9xtthMgY9vtx0wBbks5tp211gaJpZM4TNJZS> .

[di]

Would a separate option to specify packages to ignore (when they occurs as dependencies) would be useful?

Personally I don't think so. This issue is more about avoiding initial confusion than dealing w/ an annoying warning. Being able to explicitly ignore warnings for some set of packages doesn't really help with that first case.

[pradyunsg]

So... Any actionable ideas here?

[ben-albrecht]

Would a separate option to specify packages to ignore (when they occurs as
dependencies) would be useful?

For the use-case I described above, a flag for ignoring this warning altogether (rather than a package-specific warning) would suffice as a solution.

[pradyunsg]

For the use-case I described above, a flag for ignoring this warning altogether (rather than a package-specific warning) would suffice as a solution.

We have --no-warn-conflicts.

[di]

I think one actionable idea is what @cjerdonek mentions here:

My 2 cents: I don’t think “pip install foo” should report errors about dependencies not in foo’s dependency graph

I'm not familiar enough with pip to be able to say how actionable that is, but I think that this would resolve the original issue, at least for me.

[ben-albrecht]

We have --no-warn-conflicts

@pradyunsg - I was not aware of this. This is acceptable for my specific use-case. Thanks!

[pradyunsg]

My 2 cents: I don’t think “pip install foo” should report errors about dependencies not in foo’s dependency graph

I'm not familiar enough with pip to be able to say how actionable that is, but I think that this would resolve the original issue, at least for me.

Indeed. I'll explore this avenue.

[mitsuhiko]

It also brings up the warning for packages found by an egg info not yet installed. In particular it also warns about things required for setup (setup_required) and it warns for any system python 2.7 on mac because of global missing dependencies of matplotlib.

[pradyunsg]

Made a PR: #5457. If someone could take it for a spin and provide feedback, that'd be awesome. :)

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.