Support pip wheel subcommand

[kvelicka]

I've been trying uv on work projects to assess its coverage and suitability of being a full pip/pip-tools replacement for us. One of the features of pip that we use is pip wheel, which seems to not be supported by uv currently and I couln't find an open ticket discussing it, so I'm making one here.

[zanieb]

We have all the plumbing to do this, I'm not sure where it fits on our roadmap though.

I'm curious how this would relate to

• Expose uv build build frontend (pypa/build replacement) #1510

[henryiii]

Pip would prefer to remove the wheel subcommand AFIAK, and has refused adding a matching sdist command. I think uv build --wheel (along with uv build --sdist and uv build, which builds both) would be better than adding uv pip wheel.

pip wheel does one thing differently, by the way - it includes any wheels it also builds (but not just downloads), while build --wheel only gives you the wheel you asked for. But most of the time that's actually a bug (trying to upload wheels you don't own), and pip download is there if you want to actually get existing files from PyPI (not in uv yet though).

[zanieb]

Thanks for the additional context. I wonder what we can provide for people who want to generate wheels for all their dependencies though. It makes sense for pre-building and sharing when they aren't distributed by the original package.

[sbidoul]

I wonder what we can provide for people who want to generate wheels for all their dependencies though. It makes sense for pre-building and sharing when they aren't distributed by the original package.

I echo this. pip wheel is useful to download and build all parts of an app for deployment. For instance in a multi-stage Dockerfile it is common to have a fat build stage with all build tools to create the wheels, then a slim stage which only installs the wheels built in the previous stage.

[henryiii]

But it doesn’t actually put wheels in the wheelhouse if there’s already a built wheel, IIRC? Pip download will get everything you need then you can loop over any SDists and build them if you need them?

[sbidoul]

But it doesn’t actually put wheels in the wheelhouse if there’s already a built wheel, IIRC?

@henryiii I'm not entirely sure what you mean with that?

Pip download will get everything you need then you can loop over any SDists and build them if you need them?

Sure. pip wheel is convenient because it does exactly that for you.

Actually pip wheel is quite easy to implement and maintain because, it basically does everything pip install does except it stores the wheels (whether they are downloaded or built locally) into a directory instead of unpacking them in site-packages. I'd argue it is simpler than pip download, from my experience with that part of the pip code base.

[sbidoul]

That said, build, wheel and download are all useful, they are for different use cases.

[henryiii]

because it does exactly that for you ... whether they are downloaded or built locally

That's the problem, it does not store wheels that are downloaded. So if you depend on numpy, you will not get a numpy wheel in your wheelhouse unless there is no numpy wheel for that platform. This causes twine wheelhouse/* to work until you build on a platform without numpy wheels, then it crashes because you can't push numpy to PyPI. We have to work around that in cibuildwheel, as well as all the other builders that have an option to use pip wheel.

[sbidoul]

That's the problem, it does not store wheels that are downloaded. So if you depend on numpy, you will not get a numpy wheel in your wheelhouse unless there is no numpy wheel for that platform.

That's strange. It is not my understanding of how pip wheel works. I use it daily and it always stores a wheel for each top level requirement and all their dependencies, whether dependent wheels exist in the index (in which case it will download them to the wheelhouse) or not (in which case it will download the sdists, build them and store the resulting wheels in the wheelhouse).

Now if the purpose is uploading wheels to an index, I'd use build -w, or pip wheel --no-deps, which are more or less equivalent?

[henryiii]

I’ll investigate, maybe it changed or there’s an another reason it was behaving like that.

[akx]

I'd like an uv pip wheel sort of command; it's very useful for multi-stage Docker image builds where you might need e.g. the full Python dev kit to build wheels, and then want to just use those cached wheels in a subsequent stage, á la

FROM python:3.12 AS requirements
COPY requirements.txt /wheels/requirements.txt
RUN cd /wheels && pip wheel --no-cache-dir -r requirements.txt

FROM python:3.12-slim AS runtime
RUN --mount=type=cache,ro,from=requirements,source=/wheels,target=/wheels pip install --no-cache-dir --no-index --find-links /wheels /wheels/*.whl

[potiuk]

I'd like an uv pip wheel sort of command; it's very useful for multi-stage Docker image builds where you might need e.g. the full Python dev kit to build wheels, and then want to just use those cached wheels in a subsequent stage, á la

Small comment on that one (I do not contest the need for pip wheel of course just explaining how we are doing it in Airflow). I found that you can do quite a bit better than that by just copying the whole venv installed in the "build stage". If you keep it in the same location, this will work out of the box as well.

Smthing like:

FROM python:3.12 AS requirements
COPY requirements.txt /requirements.txt
RUN python -m venv /home/user/.venv && /home/user/.ven/bin/python -m pip install -r /requirements.txt

FROM python:3.12-slim AS runtime
COPY --from=requirements ~/home/user/.venv /home/user/.venv

That saves the hassle of running pip/uv install twice.

[mmerickel]

Just to second this issue, my pipeline is dependent on the recursive nature of pip wheel to download all dependencies into a wheelhouse which we can then install offline. Using pip download is possible but would require another loop to build wheels for everything into a proper wheelhouse and I'd really be looking for the uv equivalent to support the full cycle directly from requirements file -> wheelhouse.

[pradyunsg]

I tend to think of pip wheel as being useful for multiple roles:

• "give me a bunch of wheels" by default.
• "give me a wheel for this package" via an opt-in (--no-deps).

The default is not the most common operation during development, and the functional behaviour provided by pypa/build is better suited to the right behaviours during package development workflows. Ideally, these two should be under a logically different commands in uv.

None the less, my two cents would be that this shim should not be provided and uv should instead focus on its own dedicated CLI outside of the pip namespace; keeping these two "consumer" and "publisher" pieces separated.

[vigneshmanick]

To second this issue, In our usecase we build the wheel once (pip wheel . -w dist)and store it as an artifact which is then used by subsequent pipelines. This helps us to ensure that the production and ci environments are consistent and also helps to manually check the wheel contents without having to resort to additional commands.

[mmerickel]

FWIW I added a concrete example of how I use pip wheel to #7148 if it's helpful in motivating this feature.

[notatallshaw]

The command uv build now exists, does this not cover this requirement?

[mmerickel]

The issue is that the new command doesn’t build the dependencies. Neither the local dependencies in the workspace nor the remote dependencies from the package index.

[Ralith]

This would be useful for packaging Blender extensions, which need their dependencies bundled as wheels in a subdirectory: https://docs.blender.org/manual/en/dev/advanced/extensions/python_wheels.html

[sliva0]

This issue is essential to adopt uv in a project I'm working on to make reproducible offline installations possible, and it doesn't seem to move forward. Is there a way I can contribute to working on this as a rust novice?

[abhiaagarwal]

This issue has become a blocker for hardened python images, as we use want to use uv in a build step to and then copy over the dependencies into a venv we can't overwrite (Iron Bank images as a concrete example https://repo1.dso.mil/dsop/opensource/python/python313)

Similar to other people, I'd like to build reproducible wheels in an environment with dev dependencies (cmake), as the hardened images don't have anything to work with. My workaround has been using uv export to generate a requirements.txt, then pull an ubi9 image to build the wheels, and then load them into the required venv, but this, as you can imagine, is not fun.

[mmerickel]

Building on my earlier comments, I'll expand slightly with a concrete workaround.

I'm hacking around the lack of this feature right now by using a combination of uv export and pip. A reminder that the workspace itself is a bunch of independent packages with their own build backend etc that can be built using other build tools like pip wheel, python -m build, or uv build so you can pick things apart and build them as needed without uv commands. For example:

uv export --no-hashes --locked --format requirements-txt > requirements.txt
grep -v '^-e ' requirements.txt > requirements.remote.txt

# build all remote wheels
pip wheel -w wheels --find-links wheels -r requirements.remote.txt

# build all local packages to wheels
uv build --all-packages --wheel -o wheels

You can expand this further, for example I'm doing similar things for other dependency groups independently, or build requirements first,, and I'm not actually using uv build directly because I'm caching the local wheels based on git commits since uv build does not cache things well to avoid rebuilding 20 packages if only 1 changed.

The goal of something like this ticket is that we could just directly use uv build like this uv build --all-packages --wheel -o wheels and tack on another flag like --recursively-build-all-deps.

[notatallshaw]

pip wheel -w wheels --find-links wheels -r requirements.remote.txt

You can also add --no-deps here to save pip from reading and resolving dependencies, since you already have fully resolved the dependencies with uv.

[abhiaagarwal]

I've started working on this, please let me know if anyone else is. Most of the plumbing is already there.

[mmerickel]

I should note it’d be nice if there was an option to build only the dependencies and not the local packages - and specifically only the dependencies that aren’t in the target destination already similar to how --find-links works. :-)

[abhiaagarwal]

I should note it’d be nice if there was an option to build only the dependencies and not the local packages - and specifically only the dependencies that aren’t in the target destination already similar to how --find-links works. :-)

I'll need to think about the right way to do this, but my general design idea is:

uv build: build wheels (already exists)
uv build --all-packages: build wheels for all workspace packages (already exists)
uv build --with-deps: build wheels + dependencies
uv build --all-packages --with-deps: build wheels + dependencies for all workspace packages
uv pip wheel: same as uv build --with-deps, provided as shim

I'm not sure what's the best way to support installing only dependencies.

[Tremeschin]

@abhiaagarwal These new proposed options looks great! I may have a novel use case for them btw 😄

Basically, since it would download and build a fully reproducible offline installation talked previously, we can side-load all wheels on a pyapp binary (I have a fork for it) for a simpler solution on entirely offline executables :)

My two cents; It would also be nice if we could specify --platform {windows,linux,macos} --arch {x86_64,arm64} etc. for getting cross platform files, and also support for uv pip download (as uv download?) for individual/outlier packages like pytorch and its many flavors, but this might be out of this issue's scope or complex to implement haha

Looking forward for progress on this! 👍🏻

[T-256]

I've started working on this, please let me know if anyone else is. Most of the plumbing is already there.

Hey, it's great to hear that, there are some of my thoughts about combined version of pip wheel and pip download:
#3163 (comment)

I'm not sure what's the best way to support installing only dependencies.

As mentioned in examples in there, I think it could be solved by index filtering (May need to have extra index configuration in pyproject.toml to explicitly split them.)

[abhiaagarwal]

@Tremeschin yeah, that's the thing, pip install / wheel / download are all expressions of the same base concept, except doing slightly different things (ie. wheel is just download, except it also builds if it can't pull a wheel directly from the index).

[abhiaagarwal]

@T-256 you read my mind, I was also thinking of the exact nomenclature of uv collect, because uv pip download/wheel can be implemented using the exact same logic in uv build except for the actual building part. Since uv uses its cache first, the act of "downloading a wheel" is already abstracted away (we just need to query the cache for the wheel, it'll download it if it doesn't exist, and then actually package it as a .whl).

It also ties in to a higher-level abstraction of the cache itself, ie uv cache populate or something of that sort

[mmerickel]

I'm not sure what's the best way to support installing only dependencies.

I'd request a --only-deps option on both uv export and on uv build. It would trigger the flow to exclude anything that is local and only build things that it'd normally have to get from an index. With uv export the rendered lock files wouldn't contain the local packages. With uv build it would not build the local packages, just the external things they depend on.

[henryiii]

You have to build local packages to get the dependency list, though? You can shortcut it in the case of a PEP 621 package without dependencies in the dynamic array, and some backends let you shortcut the build with the get metadata hook, but sometimes you have to build the package to get the deps.

[mmerickel]

That dependency information is already in the lock file but otherwise yes you would need to build/discard it to get the metadata if you didn't have a lock file.

[edwardpeek-crown-public]

FWIW our use case for pip wheel and dependencies we already know what the dependencies we want are, so just run pip wheel --no-deps $DEPENDENCY for each of them.

You can get a such a list already using [uv] pip compile -e $PROJECT_DIR so full --only-deps functionality might not be needed for an MVP implementation.

[Spindel]

I've got multiple uses of "pip wheel", one of them is as many others, artifacts for a build or multi-stage pipeline, where we also separate dev dependencies (linters, etc) from real deps, so we can always use the same for child pipelines. Sometimes these end up in a kind of "dev container" that's used as a base, in order to prevent updated packages in repositories from causing new fun issues.

The other case is to make sure our internal repo has cached binary builds of certain projects for non x86_64 platforms, fex. lxml, cryptography, etc.

So, for that reason, being able to download or build both --dev and normal deps is needed.

[kimvais]

pip wheel does one thing differently, by the way - it includes any wheels it also builds (but not just downloads), while build --wheel only gives you the wheel you asked for. But most of the time that's actually a bug (trying to upload wheels you don't own), and pip download is there if you want to actually get existing files from PyPI (not in uv yet though).

Actually, including wheels you don't "own" is desirable for several use cases, namely for private PyPI-like repositories (e.g. Google Artifact registry, AWS CodeArtifact et al.) at least:

1. Security. If you can publish all the dependencies in your private repo, there is not even a theoretical code injection via PyPI vulnerability as you can use --index-url instead of --extra-index-url. Consider package "companyname-foobar-security- if you use--extra-index-url` and someone guesses the package name and pushes the package with that name and most satisfactory version number -> RCE.
2. Faster build times when you had to version lock a dependency, or the binary wheel isn't available for your runtime and arch combination for some other reason and each uv pip install needs to rebuild the package from sdist.