Add --ignore-packages to pip check

[q0w]

Closes #11157

[wolfv]

Looks great! Thank you for picking this up so quickly!

[q0w]

@uranusjr what to do?

1. Add a new arg to check_package_set to also ignore sub dependencies
2. Check ignore_packages outside of check_package_set, filtering conflicting, missing results

[dhirschfeld]

I think this will be very useful so am keen to get it in. Are there any blockers here... other than maintainers time to review?

[uranusjr]

Personally a separate option to control ignoring dependecies feels more correct; --ignore-packages feels like ignoring conflicts on those packages, not dependencies.

[q0w]

separate option to control

global command option?

[uranusjr]

That I’m not sure about, to be honest. A part of me wants to keep things as simple as possible (say just pip check --ignore=foo [--recursive-ignore]) but that has obvious downsides…

[uranusjr]

Logic looks fine, not sure about the option names and UI design in general so I’m going to leave this open for now.

[jaimergp]

Hi team, is there anything we can do to move this forward? It's been a year and this is a change some ecosystems would welcome a lot!

[uranusjr]

I added a minor tweak, shouldn’t affect functionality (if I got it right)

[uranusjr]

I pondered on the options names a bit; while I would most like pip check --ignore=..., this is probably the best compromise since PIP_IGNORE as an environment variable would be quite suboptimal. So unless someone raises any concerns I’ll merge this soon.

[jakirkham]

Thanks for all the hard work here! 🙏

Not to bikeshed, but if we want a shorter name, maybe we could use --skip-pkgs or similar?

That said, ok with the status quo. Having this implemented under any name would be welcome 😀

[sbidoul]

This one is in the 23.3 milestone but CI is red.

[q0w]

@sbidoul updated

[uranusjr]

Would be nice to have a docstring on this function explaining how the two arguments are different

[q0w]

like this?

[sbidoul]

I have had a quick look at this from a end-user perspective, and what strikes me is that the help texts for these two new options are not really helpful to me.

Is it correct that --ignore-packages causes pip check to not report errors about missing packages in the --ignore-packages list? Or does it rather not check direct dependencies for the packages listed? My intuition would say the former would be the best answer to the linked issue, but a very quick check seems to show it does the latter (I may be wrong about that).

Regarding --recursive-ignore the use case is not obvious me, nor what it does exactly. Indeed #11157 mentions ignoring third party packages. But those don't have dependencies? So how does a recursive ignore help with that original use case?

Finally, a little detail: since --ignore-packages does not accept a comma separated list, but can be repeated, it's probably best in the singular form.

[q0w]

Or does it rather not check direct dependencies for the packages listed?

Yes.

My intuition would say the former would be the best answer to the linked issue, but a very quick check seems to show it does the latter (I may be wrong about that).

For that you should use --recursive-ignore to specify that package could be someone's dependency. There is a test to demonstrate this behaviour.

In #11157 it should be use like pip check --ignore-packages ninja --recursive-ignore

[sbidoul]

Ok, then this goes against the intuition I get from reading the option name.
It seems @uranusjr had a similar reaction in #11159 (comment).

For that you should use --recursive-ignore to specify that package could be someone's dependency
In #11157 it should be use like pip check --ignore-packages ninja --recursive-ignore

But if a package is detected as missing then it is because it is a dependency of another package, otherwise how would pip check know it is missing? So why is --recursive-ignore needed?

I don't know maybe I'm having a bad day but this does not click. At least a better help text is needed, IMO.

I don't feel comfortable merging this myself before the release tomorrow but I'm happy to defer to other maintainers.

[pfmoore]

I don't feel comfortable merging this myself before the release tomorrow but I'm happy to defer to other maintainers.

I agree. There's no urgency here, let's give it time and see if we can come up with a more intuitive UI.

I think part of the problem here is that the examples givent in #11157 (ninja and cmake) have no dependencies, so there's no obvious answer to the question of "what if the package you're ignoring has dependencies?" Maybe we should look for a real-world use case where the ignored package does have dependencies, and base the behaviour on that?

Also, do we have a good intuition for what should happen if we have packages B and C installed, where B depends on A 1.0, and C depends on A 2.0, and we say pip check --ignore-package A¹? Should the check pass because we're ignoring A, or fail because no possible installation of A could help here anyway?

Footnotes

1. I agree with @sbidoul that the singular is better] ↩

[dhirschfeld]

Today, I could have really used this functionality.

Instead, I'll just have to remove pip check entirely from my recipe.

[pfmoore]

Today, I could have really used this functionality.

@dhirschfeld if you read my comment above and the discussion leading up to it, you'll see that there's some debate as to what "this functionality" even is. Could you clarify what precisely your situation is, and what particular variant of the proposed feature would have helped you?

[dhirschfeld]

Funnily enough it has to do with pypiwin32 which it appears you're a maintainer of @pfmoore! 😄

I think pypiwin32 is a repackage of pywin32 but it's PyPI page is effectively blank and I'm not really sure why it exists. Since the last published version was in early 2018 I don't think anybody should be depending on it (maybe that's wrong?) so when repackaging for conda-forge I instead specified pywin32 as the dependency.

Since pywin32 is a different package to pypiwin32, pip check complains. It would be good to be able to selectively ignore pypiwin32 so I could still use pip check to verify the other dependencies.

xref:

• conda-forge/staged-recipes@d5b0f23 (#25289)
• https://github.com/rob-blackbourn/httpx-negotiate-sspi/blob/master/pyproject.toml#L15

[pfmoore]

pypiwin32 was a stopgap from years ago when pywin32 didn't provide wheels, and yes, no-one should be using it now.

Why are you using pip check on a conda-managed environment in any case? But regardless I don't think it's a pip issue if you've made the conda metadata different from the standard metadata... I don't think it's really your call to modify the metadata of the httpx-negotiate-sspi` package (but maybe that's a normal thing for conda repackagers to do).

Regardless, none of this helps us to decide what the right behaviour for this feature should be, so we're still blocked on clarifying that.

[dhirschfeld]

but maybe that's a normal thing for conda repackagers to do

It's a normal thing to do to ensure packages work with conda-forge. The mis-specified dependency should be corrected upstream but as a community packaging effort, upstream packaging errors can be fixed by conda-forge maintainers - it's one of the great benefits/strengths of the community/ecosystem.

There are other cases where the pip metadata necessarily won't agree with conda pacakges and in those cases it would be good to be able to selectively ignore metadata which is incorrect or irrelevant for conda.

I think the main use-case of this feature is for conda and pip to work better together and IIUC that was the motivation for the original issue which this PR aims to resolve.

[dhirschfeld]

Regardless, none of this helps us to decide what the right behaviour for this feature should be, so we're still blocked on clarifying that.

Sure, there are still some open questions. I'll make some time to put down my thoughts so we can hopefully agree on the right API...

[henryiii]

FYI, this would have been useful to ignore uv for hatch in conda-forge.