Build backend: Revamp include/exclude #9525
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
konstin
force-pushed
the
konsti/build-backend-warn-extra
branch
from
e7da29a to
c777459
Compare
konstin
force-pushed
the
konsti/build-backend-warn-extra
branch
from
56d3ea5 to
e6479cb
Compare
When building the source distribution, we always need to include `pyproject.toml` and the module, when building the wheel, we always include the module but nothing else at top level. Since we only allow a single module per wheel, that means that there are no specific wheel includes. This means we have source includes, source excludes, wheel excludes, but no wheel includes: This is defined by the module root, plus the metadata files and data directories separately. Extra source dist includes are currently unused (they can't end up in the wheel currently), but it makes sense to model them here, they will be needed for any sort of procedural build step. This results in the following fields being relevant for inclusions and exclusion: * project.readme: PEP 621 * project.license-files: PEP 639 * module_root: `Path` * source_include: `Vec<Glob>` * source_exclude: `Vec<Glob>` * wheel_exclude: `Vec<Glob>` * data: `Map<KnownDataName, PathBuf>` An opinionated choice is that that wheel excludes always contain the source excludes: Otherwise you could have a path A in the source tree that gets included when building the wheel directly from the source tree, but not when going through the source dist as intermediary, because A is in source excludes, but not in the wheel excludes. This has been a source of errors previously. In the process, i fixed a bug where we would skip directories and only include the files and were missing license due to absolute globs.
konstin
force-pushed
the
konsti/build-backend-warn-extra
branch
from
e6479cb to
f30bd1e
Compare
konstin
force-pushed
the
konsti/build-backend-warn-extra
branch
from
f30bd1e to
00d4956
Compare
BurntSushi
reviewed
Dec 3, 2024
| /// | ||
| /// There are no specific wheel includes. There must only be one top level module, and all data | ||
| /// files must either be under the module root or in a data directory. Most packages store small | ||
| /// data in the module root alongside the source code. |
There was a problem hiding this comment.
This was a helpful comment, thank you!
tmeijn
pushed a commit
to tmeijn/dotfiles
that referenced
this pull request
Dec 5, 2024
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.5.5` -> `0.5.6` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.5.6`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#056) [Compare Source](astral-sh/uv@0.5.5...0.5.6) ##### Enhancements - Add `--dry-run` to `uv pip uninstall` ([#​9557](astral-sh/uv#9557)) - Allow `--constraints` and `--overrides` in `uv tool install` ([#​9547](astral-sh/uv#9547)) - Display removed Python executables on uninstall ([#​9459](astral-sh/uv#9459)) - Warn when keyring has no password for `uv publish` ([#​8827](astral-sh/uv#8827)) - Add suggested action when `.python-version` pin is incompatible with the project ([#​9590](astral-sh/uv#9590)) - Improve error messages for mismatches in `tool.uv.sources` ([#​9482](astral-sh/uv#9482)) - Use constraints in trace rather than irrelevant `requires-python` ([#​9529](astral-sh/uv#9529)) ##### Preview features - Add `uv python install --default` ([#​8650](astral-sh/uv#8650)) - Fix Python executable installation when multiple patch versions are requested ([#​9607](astral-sh/uv#9607)) - Build backend: Revamp `include` / `exclude` ([#​9525](astral-sh/uv#9525)) - Build backend: Add fast path ([#​9556](astral-sh/uv#9556)) - Build backend: Add functions to collect file list ([#​9602](astral-sh/uv#9602)) - Build backend: Default excludes ([#​9552](astral-sh/uv#9552)) - Build backend: Refactoring before list ([#​9558](astral-sh/uv#9558)) - Build backend: Warn when visiting over 10k files ([#​9523](astral-sh/uv#9523)) ##### Configuration - Make `check-url` available in configuration files ([#​9032](astral-sh/uv#9032)) ##### Performance - Avoid adding non-extra package with extra dependencies ([#​9540](astral-sh/uv#9540)) - Avoid cloning `String` in marker evaluation ([#​9598](astral-sh/uv#9598)) ##### Rust API - `uv-pep508`: Add more methods for simplifying `extra`-related expressions ([#​9469](astral-sh/uv#9469)) ##### Bug fixes - Allow `file:` URLs to include package names ([#​9493](astral-sh/uv#9493)) - Avoid using IDs across PubGrub states ([#​9538](astral-sh/uv#9538)) - Consistently enforce requested-vs.-built metadata when retrieving wheels ([#​9484](astral-sh/uv#9484)) - Do not show empty version specifier in `uv tool list` ([#​9605](astral-sh/uv#9605)) - Include Git member information when getting metadata from cache ([#​9388](astral-sh/uv#9388)) - Include base installation directory in uv run PATH ([#​9585](astral-sh/uv#9585)) - Insert backslash when appending to system drive ([#​9488](astral-sh/uv#9488)) - Normalize paths when lowering Git dependencies ([#​9595](astral-sh/uv#9595)) - Omit origin when comparing requirements ([#​9570](astral-sh/uv#9570)) - Override `manylinux_compatible` with `--python-platform` ([#​9526](astral-sh/uv#9526)) - Pass extra when evaluating lockfile markers ([#​9539](astral-sh/uv#9539)) - Propagate markers for recursive extras in resolver ([#​9509](astral-sh/uv#9509)) - Respect path dependencies within Git dependencies ([#​9594](astral-sh/uv#9594)) - Support recursive extras with marker in `pip compile -r pyproject.toml` ([#​9535](astral-sh/uv#9535)) - Don't emit unpinned warning for proxy packages ([#​9497](astral-sh/uv#9497)) - Fix `--refresh-package` flag mentioned as `--refresh-dependency` ([#​9486](astral-sh/uv#9486)) - Handle Windows AV/EDR file locks during script installations ([#​9543](astral-sh/uv#9543)) - Re-enable conflicting extra/group tests and fix regression from [#​9540](astral-sh/uv#9540) ([#​9582](astral-sh/uv#9582)) ##### Documentation - Add missing word to docs for `run.md` ([#​9527](astral-sh/uv#9527)) - Add policies reference section and license document ([#​9367](astral-sh/uv#9367)) - Fix typo in entry point docs ([#​9491](astral-sh/uv#9491)) - Fix up version in prior uninstall instructions ([#​9485](astral-sh/uv#9485)) - Mention `uv pip` behavior in build system note ([#​9586](astral-sh/uv#9586)) - Update build failures document ([#​9584](astral-sh/uv#9584)) - Correct wording for multiple sources section ([#​9504](astral-sh/uv#9504)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When building the source distribution, we always need to include
pyproject.tomland the module, when building the wheel, we always include the module but nothing else at top level. Since we only allow a single module per wheel, that means that there are no specific wheel includes. This means we have source includes, source excludes, wheel excludes, but no wheel includes: This is defined by the module root, plus the metadata files and data directories separately.Extra source dist includes are currently unused (they can't end up in the wheel currently), but it makes sense to model them here, they will be needed for any sort of procedural build step.
This results in the following fields being relevant for inclusions and exclusion:
pyproject.toml(always included in the source dist)PathVec<Glob>Vec<Glob>Vec<Glob>Map<KnownDataName, Path>An opinionated choice is that that wheel excludes always contain the source excludes: Otherwise you could have a path A in the source tree that gets included when building the wheel directly from the source tree, but not when going through the source dist as intermediary, because A is in source excludes, but not in the wheel excludes. This has been a source of errors previously.
In the process, I fixed a bug where we would skip directories and only include the files and were missing license due to absolute globs.