Build backend: Add fast path #9556

konstin · 2024-12-01T20:39:09Z

Going through PEP 517 to build a package is slow, so when building a package with the uv build backend, we can call into the uv build backend directly. This is the basis for the uv build --list.

This does not enable the fast path for general source dependencies.

There is a possible difference in execution if the latest uv version is newer than the one currently running: The PEP 517 path would use the latest version, while the fast path uses the current version.

Please review commit-by-commit

Benchmark

built_with_uv, using the fast path:

$ hyperfine "~/projects/uv/target/profiling/uv build"
Time (mean ± σ):       9.2 ms ±   1.1 ms    [User: 4.6 ms, System: 4.6 ms]
Range (min … max):     6.4 ms …  12.7 ms    290 runs

hatcling_editable, with hatchling being optimized for fast startup times:

$ hyperfine "~/projects/uv/target/profiling/uv build"
Time (mean ± σ):     270.5 ms ±  18.4 ms    [User: 230.8 ms, System: 44.5 ms]
Range (min … max):   250.7 ms … 298.4 ms    10 runs

charliermarsh · 2024-12-01T21:09:10Z

This is sick!

charliermarsh · 2024-12-01T22:24:18Z

crates/uv-cli/src/lib.rs

+    /// backend, but use a fast path that calls into the build backend directly. This option forces
+    /// always using PEP 517.
+    #[arg(long)]
+    pub no_fast_path: bool,


I think we should give this a more descriptive name, like --use-external-uv or something. Since we might add other "fast paths" in the future. What do you think?

Good idea. I thought about something like --pep517, maybe --force-pep517?

--force-pep517 seems reasonable...

charliermarsh · 2024-12-01T22:26:21Z

crates/uv/src/commands/build_frontend.rs

+    version_id: Option<&str>,
+    build_output: BuildOutput,
+) -> Result<String> {
+    let sdist = if fast_path {


I might structure these as just two entirely separate methods? That's always my instinct when I have a method that takes a bool and entirely forks behavior on that bool. Sort of a matter of preference, though.

I like those, in this case i already know that I'll add more forks (listing vs. building) here later :)

The callsite will move later

It's not exactly prettier, but it avoids copying the logic

Going through PEP 517 to build a package is slow, so when building a package with the uv build backend, we can call into the uv build backend directly. This is the basis for the `uv build --list`. This does not enable the fast path for general source dependencies. There is a possible difference in execution if the latest uv version is newer than the one currently running: The PEP 517 path would use the latest version, while the fast path uses the current version. ### Benchmark `built_with_uv`, using the fast path: ``` $ hyperfine "~/projects/uv/target/profiling/uv build" Time (mean ± σ): 9.2 ms ± 1.1 ms [User: 4.6 ms, System: 4.6 ms] Range (min … max): 6.4 ms … 12.7 ms 290 runs ``` `hatcling_editable`, with hatchling being optimized for fast startup times: ``` $ hyperfine "~/projects/uv/target/profiling/uv build" Time (mean ± σ): 270.5 ms ± 18.4 ms [User: 230.8 ms, System: 44.5 ms] Range (min … max): 250.7 ms … 298.4 ms 10 runs ```

No tests since we can't use the uv build backend through PEP 517 as we don't have a uv wheel at test time.

See #9556 (comment)

BurntSushi

Nice! The benchmark shows a rather large perf difference. Do you have a sense of where this is coming from?

And that also makes me curious about the perf difference between the fast path and --force-pep517. That seems like a more apples-to-apples comparison? (Unless I'm misunderstanding the benchmark in the PR description.)

BurntSushi · 2024-12-03T14:24:02Z

crates/uv-cli/src/lib.rs

+    /// backend, but use a fast path that calls into the build backend directly. This option forces
+    /// always using PEP 517.
+    #[arg(long)]
+    pub no_fast_path: bool,


konstin · 2024-12-03T16:25:48Z

For all the python build backends, we pay a cost for each import (see also https://peps.python.org/pep-0690/). A build backend is using a lot of features, so it gets slowed down by a lot of imports, even something manually optimized to have lazy imports such as hatchling.

For uv-against-uv, we paying the cost for setting up the build env, syncing it, spawning python, the spawning uv from python. We do this for each hook we call (should be build_sdist and build_wheel here).

Here's a uv-against-uv benchmark:

UV_PREVIEW=1 hyperfine "../../../target/profiling/uv build --find-links ../../../target/wheels/ --preview" "UV_PREVIEW=1 ../../../target/profiling/uv build --find-links ../../../target/wheels/ --preview --force-pep517"
Benchmark 1: ../../../target/profiling/uv build --find-links ../../../target/wheels/ --preview
  Time (mean ± σ):       9.7 ms ±   1.7 ms    [User: 4.7 ms, System: 4.9 ms]
  Range (min … max):     6.5 ms …  21.7 ms    265 runs
 
Benchmark 2: UV_PREVIEW=1 ../../../target/profiling/uv build --find-links ../../../target/wheels/ --preview --force-pep517
  Time (mean ± σ):      87.7 ms ±   4.2 ms    [User: 61.0 ms, System: 28.4 ms]
  Range (min … max):    79.6 ms …  98.5 ms    33 runs
 
Summary
  ../../../target/profiling/uv build --find-links ../../../target/wheels/ --preview ran
    9.08 ± 1.61 times faster than UV_PREVIEW=1 ../../../target/profiling/uv build --find-links ../../../target/wheels/ --preview --force-pep517

To illustrate the overhead from calling python, this is the cost for some std modules you'll want for building distributions:

$ hyperfine "python -c ''"
Benchmark 1: python -c ''
  Time (mean ± σ):       7.9 ms ±   1.2 ms    [User: 5.9 ms, System: 2.0 ms]
  Range (min … max):     6.2 ms …  10.4 ms    383 runs
$ hyperfine "python -c 'import sys'"
Benchmark 1: python -c 'import sys'
  Time (mean ± σ):       8.1 ms ±   1.2 ms    [User: 5.9 ms, System: 2.1 ms]
  Range (min … max):     6.3 ms …  10.7 ms    342 runs
$ hyperfine "python -c 'import sys, os, subprocess, pathlib, zipfile'"
Benchmark 1: python -c 'import sys, os, subprocess, pathlib, zipfile'
  Time (mean ± σ):      17.5 ms ±   2.1 ms    [User: 14.3 ms, System: 3.1 ms]
  Range (min … max):    15.1 ms …  24.6 ms    170 runs

The uv build backend imports sys and subprocess lazily, and has to spawn uv, so we can look at their isolated overhead:

$ UV_PREVIEW=1 hyperfine "python -c 'import sys, subprocess; from uv import build_wheel'"
Benchmark 1: python -c 'import sys, subprocess; from uv import build_wheel'
  Time (mean ± σ):      14.7 ms ±   1.8 ms    [User: 11.8 ms, System: 2.9 ms]
  Range (min … max):    12.4 ms …  20.2 ms    219 runs
$ hyperfine -N "uv build-backend --version"
Benchmark 1: uv build-backend --version
  Time (mean ± σ):       2.4 ms ±   0.3 ms    [User: 0.9 ms, System: 1.5 ms]
  Range (min … max):     1.6 ms …   3.1 ms    968 runs

BurntSushi · 2024-12-03T16:50:30Z

Wow. So cool.

This is #9556, but at the level of all other builds, including the resolver and installer. Going through PEP 517 to build a package is slow, so when building a package with the uv build backend, we can call into the uv build backend directly. This fast path is gated through preview. Since the uv wheel is not available at test time, I've manually confirmed the feature by comparing `uv venv && cargo run pip install . -v --preview --reinstall .` and `uv venv && cargo run pip install . -v --reinstall .`. Do we need a global option to disable the fast path? There is one for `uv build` because `--force-pep517` moves `uv build` much closer to a `pip install` from source that a user of a library would experience.

This is like #9556, but at the level of all other builds, including the resolver and installer. Going through PEP 517 to build a package is slow, so when building a package with the uv build backend, we can call into the uv build backend directly instead: No temporary virtual env, no temp venv sync, no python subprocess calls, no uv subprocess calls. This fast path is gated through preview. Since the uv wheel is not available at test time, I've manually confirmed the feature by comparing `uv venv && cargo run pip install . -v --preview --reinstall .` and `uv venv && cargo run pip install . -v --reinstall .`. Do we need a global option to disable the fast path? There is one for `uv build` because `--force-pep517` moves `uv build` much closer to a `pip install` from source that a user of a library would experience.See discussion at #9610 (comment)

This is like #9556, but at the level of all other builds, including the resolver and installer. Going through PEP 517 to build a package is slow, so when building a package with the uv build backend, we can call into the uv build backend directly instead: No temporary virtual env, no temp venv sync, no python subprocess calls, no uv subprocess calls. This fast path is gated through preview. Since the uv wheel is not available at test time, I've manually confirmed the feature by comparing `uv venv && cargo run pip install . -v --preview --reinstall .` and `uv venv && cargo run pip install . -v --reinstall .`. When hacking the preview so that the python uv build backend works without the setting the direct build also (wheel built with `maturin build --profile profiling`), we can see the perfomance difference: ``` $ hyperfine --prepare "uv venv" --warmup 3 \ "UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview" \ "target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/" Benchmark 1: UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview Time (mean ± σ): 33.1 ms ± 2.5 ms [User: 25.7 ms, System: 13.0 ms] Range (min … max): 29.8 ms … 47.3 ms 73 runs Benchmark 2: target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/ Time (mean ± σ): 115.1 ms ± 4.3 ms [User: 54.0 ms, System: 27.0 ms] Range (min … max): 109.2 ms … 123.8 ms 25 runs Summary UV_PREVIEW=1 target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --preview ran 3.48 ± 0.29 times faster than target/profiling/uv pip install --no-deps --reinstall scripts/packages/built-by-uv --find-links target/wheels/ ``` Do we need a global option to disable the fast path? There is one for `uv build` because `--force-pep517` moves `uv build` much closer to a `pip install` from source that a user of a library would experience (See discussion at #9610), but uv overall doesn't really make guarantees around the build env of dependencies, so I consider the direct build a valid option. Best reviewed commit-by-commit, only the last commit is the actual implementation, while the preview mode introduction is just a refactoring touching too many files.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.5.5` -> `0.5.6` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.5.6`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#056) [Compare Source](astral-sh/uv@0.5.5...0.5.6) ##### Enhancements - Add `--dry-run` to `uv pip uninstall` ([#9557](astral-sh/uv#9557)) - Allow `--constraints` and `--overrides` in `uv tool install` ([#9547](astral-sh/uv#9547)) - Display removed Python executables on uninstall ([#9459](astral-sh/uv#9459)) - Warn when keyring has no password for `uv publish` ([#8827](astral-sh/uv#8827)) - Add suggested action when `.python-version` pin is incompatible with the project ([#9590](astral-sh/uv#9590)) - Improve error messages for mismatches in `tool.uv.sources` ([#9482](astral-sh/uv#9482)) - Use constraints in trace rather than irrelevant `requires-python` ([#9529](astral-sh/uv#9529)) ##### Preview features - Add `uv python install --default` ([#8650](astral-sh/uv#8650)) - Fix Python executable installation when multiple patch versions are requested ([#9607](astral-sh/uv#9607)) - Build backend: Revamp `include` / `exclude` ([#9525](astral-sh/uv#9525)) - Build backend: Add fast path ([#9556](astral-sh/uv#9556)) - Build backend: Add functions to collect file list ([#9602](astral-sh/uv#9602)) - Build backend: Default excludes ([#9552](astral-sh/uv#9552)) - Build backend: Refactoring before list ([#9558](astral-sh/uv#9558)) - Build backend: Warn when visiting over 10k files ([#9523](astral-sh/uv#9523)) ##### Configuration - Make `check-url` available in configuration files ([#9032](astral-sh/uv#9032)) ##### Performance - Avoid adding non-extra package with extra dependencies ([#9540](astral-sh/uv#9540)) - Avoid cloning `String` in marker evaluation ([#9598](astral-sh/uv#9598)) ##### Rust API - `uv-pep508`: Add more methods for simplifying `extra`-related expressions ([#9469](astral-sh/uv#9469)) ##### Bug fixes - Allow `file:` URLs to include package names ([#9493](astral-sh/uv#9493)) - Avoid using IDs across PubGrub states ([#9538](astral-sh/uv#9538)) - Consistently enforce requested-vs.-built metadata when retrieving wheels ([#9484](astral-sh/uv#9484)) - Do not show empty version specifier in `uv tool list` ([#9605](astral-sh/uv#9605)) - Include Git member information when getting metadata from cache ([#9388](astral-sh/uv#9388)) - Include base installation directory in uv run PATH ([#9585](astral-sh/uv#9585)) - Insert backslash when appending to system drive ([#9488](astral-sh/uv#9488)) - Normalize paths when lowering Git dependencies ([#9595](astral-sh/uv#9595)) - Omit origin when comparing requirements ([#9570](astral-sh/uv#9570)) - Override `manylinux_compatible` with `--python-platform` ([#9526](astral-sh/uv#9526)) - Pass extra when evaluating lockfile markers ([#9539](astral-sh/uv#9539)) - Propagate markers for recursive extras in resolver ([#9509](astral-sh/uv#9509)) - Respect path dependencies within Git dependencies ([#9594](astral-sh/uv#9594)) - Support recursive extras with marker in `pip compile -r pyproject.toml` ([#9535](astral-sh/uv#9535)) - Don't emit unpinned warning for proxy packages ([#9497](astral-sh/uv#9497)) - Fix `--refresh-package` flag mentioned as `--refresh-dependency` ([#9486](astral-sh/uv#9486)) - Handle Windows AV/EDR file locks during script installations ([#9543](astral-sh/uv#9543)) - Re-enable conflicting extra/group tests and fix regression from [#9540](astral-sh/uv#9540) ([#9582](astral-sh/uv#9582)) ##### Documentation - Add missing word to docs for `run.md` ([#9527](astral-sh/uv#9527)) - Add policies reference section and license document ([#9367](astral-sh/uv#9367)) - Fix typo in entry point docs ([#9491](astral-sh/uv#9491)) - Fix up version in prior uninstall instructions ([#9485](astral-sh/uv#9485)) - Mention `uv pip` behavior in build system note ([#9586](astral-sh/uv#9586)) - Update build failures document ([#9584](astral-sh/uv#9584)) - Correct wording for multiple sources section ([#9504](astral-sh/uv#9504)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

konstin added the preview Experimental behavior label Dec 1, 2024

konstin requested review from BurntSushi and charliermarsh December 1, 2024 20:39

Base automatically changed from konsti/shared-state to main December 1, 2024 22:20

charliermarsh reviewed Dec 1, 2024

View reviewed changes

charliermarsh approved these changes Dec 1, 2024

View reviewed changes

konstin added 6 commits December 2, 2024 16:21

Extract: prepare_output_directory

d94fefe

The callsite will move later

Extract build plan determination

5b3af1b

Make PyProjectToml::check_build_system reusable

70e677e

It's not exactly prettier, but it avoids copying the logic

Refactor message annotations

87b3514

Add uv build --no-fast-path option

c3add8a

No tests since we can't use the uv build backend through PEP 517 as we don't have a uv wheel at test time.

konstin force-pushed the konsti/build-backend-fast-path branch from 02d3be2 to c3add8a Compare December 2, 2024 15:26

konstin enabled auto-merge (squash) December 2, 2024 15:26

konstin merged commit 5b27dec into main Dec 2, 2024
64 checks passed

konstin deleted the konsti/build-backend-fast-path branch December 2, 2024 15:37

konstin added a commit that referenced this pull request Dec 3, 2024

Build backend: Rename --no-fast-path to --force-pep517

d6e3406

See #9556 (comment)

konstin mentioned this pull request Dec 3, 2024

Build backend: Rename --no-fast-path to --force-pep517 #9600

Merged

konstin added a commit that referenced this pull request Dec 3, 2024

Build backend: Rename --no-fast-path to --force-pep517

65793c0

See #9556 (comment)

konstin added a commit that referenced this pull request Dec 3, 2024

Build backend: Rename --no-fast-path to --force-pep517 (#9600)

cadba18

See #9556 (comment)

BurntSushi reviewed Dec 3, 2024

View reviewed changes

This was referenced Dec 3, 2024

Build backend: Add direct builds to the resolver and installer #9621

Merged

Build backend tracking issue #8779

Open

BrewTestBot mentioned this pull request Dec 3, 2024

uv 0.5.6 Homebrew/homebrew-core#199944

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build backend: Add fast path #9556

Build backend: Add fast path #9556

konstin commented Dec 1, 2024

charliermarsh commented Dec 1, 2024

charliermarsh Dec 1, 2024

konstin Dec 1, 2024

charliermarsh Dec 1, 2024

BurntSushi Dec 3, 2024

charliermarsh Dec 1, 2024

konstin Dec 1, 2024

BurntSushi left a comment

BurntSushi Dec 3, 2024

konstin commented Dec 3, 2024 •

edited

Loading

BurntSushi commented Dec 3, 2024

Build backend: Add fast path #9556

Build backend: Add fast path #9556

Conversation

konstin commented Dec 1, 2024

Benchmark

charliermarsh commented Dec 1, 2024

charliermarsh Dec 1, 2024

Choose a reason for hiding this comment

konstin Dec 1, 2024

Choose a reason for hiding this comment

charliermarsh Dec 1, 2024

Choose a reason for hiding this comment

BurntSushi Dec 3, 2024

Choose a reason for hiding this comment

charliermarsh Dec 1, 2024

Choose a reason for hiding this comment

konstin Dec 1, 2024

Choose a reason for hiding this comment

BurntSushi left a comment

Choose a reason for hiding this comment

BurntSushi Dec 3, 2024

Choose a reason for hiding this comment

konstin commented Dec 3, 2024 • edited Loading

BurntSushi commented Dec 3, 2024

konstin commented Dec 3, 2024 •

edited

Loading