Deprecate --no-binary disabling the wheel cache

[sbidoul]

Currently, --no-binary disables reading the cache of locally built wheels.

The wheel cache is however populated in some cases (such as when using pip wheel --no-binary or pip install --no-binary for projects with a pyproject.toml.

We want --no-binary to mean only one thing, namely "don't download wheels".

So we deprecate this behaviour and add a no-binary-enable-wheel-cache feature flag that will become the default in the future.

The recommended alternative is to use --no-cache or separate cache directories using --cache-dir.

[cuhe-la]

hi @sbidoul ,

DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache.

It sounds like we deprecate --no-binary completely. Can we add a sentence to deprecation message about this part:

We want --no-binary to mean only one thing, namely "don't download wheels".

[mid-kid]

Piping in to say thanks for this change. I frequently work in environments where prebuilt wheels don't satisfy me, and the recent PEP5something changes had made the installation of various pip packages take ages between various reinstalls of the same set of wheels. This helps a lot.

[bmerry]

After this change, is there a way to prevent a specific package from being installed from a cached wheel? Currently when a package does not ship a wheel but pip builds one locally and caches it, that wheel may be linked against specific versions/sonames of a library (this particularly happens with pycuda). If the underlying library is upgraded (with a new soname), you'll need to reinstall the package from source. There are some workarounds, but they're not ideal:

1. You can delete the cached wheel, but you have to know where to find it, which is an implementation detail that I'd rather not bake into scripts.
2. You can disable the cache completely, but then any other packages you install at the same time won't benefit from the cache either.

[sbidoul]

@bmerry there is currently no mechanism to selectively disable the cache per project.

The current alternatives are

• --no-cache to disable the cache completely, as you noted
• --cache-dir to use different caches depending on the context (build settings, etc)
• pip cache remove to remove a project from the cache by name

[bmerry]

Thanks. I wasn't aware of pip cache remove and I think that'll address my use case well enough. I suspect it'll also remove the cache of the source package, which is not ideal but it's not the end of the world to download it again.

[sbidoul]

I suspect it'll also remove the cache of the source package

No, because the pip cache command only addresses the cache of locally built wheels, and leaves the http cache alone...
Which is not obvious, admittedly, but that is another story ;)

[bmerry]

I suspect it'll also remove the cache of the source package

No, because the pip cache command only addresses the cache of locally built wheels, and leaves the http cache alone... Which is not obvious, admittedly, but that is another story ;)

Not obvious indeed, but in this case exactly the behaviour I wanted :-)

[cclauss]

What is the new equivalent for PIP_NO_BINARY ?

PIP_NO_BINARY=psycopg2 pipenv install now raises DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453

[sbidoul]

@cclauss I'm not sure to understand your question. Can you elaborate?

[cclauss]

Our project uses the syntax PIP_NO_BINARY=psycopg2 pipenv install but that now raises the above deprecation warning. Is that a different environment variable that can be used which does not raise the deprecation warning?

Is PIP_NO_BINARY documented somewhere?

[sbidoul]

PIP_NO_BINARY is the equivalent of --no-binary which is documented at https://pip.pypa.io/en/stable/cli/pip_install/.
This option is not deprecated. What is going to change is the fact that this option also disables the cache of locally built wheel.

[alanawss]

Now

Defaulting to user installation because normal site-packages is not writeable
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at #11453

[lrq3000]

For those wondering how to force pip install to use the sdist instead of wheels, simply combine --no-binary with --no-cache, eg.:

pip install --upgrade mypkg --no-binary "mypkg" --no-cache

This is especially useful in combination with --config-setting="--build-option=--yourflag" to pass custom build flags to your setup.py, such as for explicit optional cythonization (see reedsolo for a full practical example).

Note: using --no-binary ":all:" can also be an alternative in some situations, but your dependencies will also be using sdists, and they may fail, especially cython which is hard to compile.