Support globs as cache keys in `tool.uv.cache-keys` #7268

charliermarsh · 2024-09-10T18:37:56Z

Summary

This has been asked for a few times. There are risks that these checks could be slow, but they're buyer-beware.

Closes #7246.

konstin · 2024-09-10T18:39:57Z

crates/uv-cache-info/src/cache_info.rs

+                        };
+                        let paths = match glob::glob_with(
+                            pattern,
+                            MatchOptions {


What's the motivation for these?

I think require_literal_separator is the only one that doesn't match the default (I was wrong, I need to change case_sensitive to true). Without require_literal_separator, *.py matches subdirectories, which I think is wrong?

@BurntSushi would know though. Without require_literal_separator, I think ** and * are identical.

I think that since ** is supported, making * not match through a / makes sense. Because users can use **/*.py to get the behavior of "match any .py file in any sub-directory."

charliermarsh · 2024-09-10T18:54:16Z

Honestly, we might want to have some set of directories that we manually exclude from search, like Ruff's exclude feature... .git, etc.

BurntSushi · 2024-09-11T18:40:21Z

crates/uv-cache-info/src/cache_info.rs

+                        };
+                        let paths = match glob::glob_with(
+                            pattern,
+                            MatchOptions {


I think that since ** is supported, making * not match through a / makes sense. Because users can use **/*.py to get the behavior of "match any .py file in any sub-directory."

BurntSushi · 2024-09-11T18:41:01Z

crates/uv-cache-info/src/cache_info.rs

+                            warn!("Failed to convert pattern to string: {}", path.display());
+                            continue;
+                        };
+                        let paths = match glob::glob_with(


How come you ended up going this route instead of globwalk?

It seemed a bit simpler given that we tend to have one glob if any. What do you think?

Aside from this needing to do a directory traversal for each glob (which would be mitigated I suppose if only one glob is used, but I'm not sure if we know that's the common case), my main gripe would be that glob doesn't support curly brackets. And switching to another glob implementation could have subtle breaking changes in the future.

These are probably super rare cases, but e.g., foo{a,b}bar is interpreted literally by glob (I believe), where as globset will treat it as fooabar OR foobbar`.

Ok I'm gonna switch it up. But probably in a separate PR.

BurntSushi · 2024-09-11T18:41:32Z

crates/uv-settings/src/settings.rs

+    /// crate. For example, to invalidate the cache whenever a `.toml` file in the project directory
+    /// or any of its subdirectories is modified, you can specify `cache-keys = [{ file = "**/*.toml" }]`.
+    /// Note that the use of globs can be expensive, as uv may need to walk the filesystem to
+    /// determine whether any files have changed.


This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.4.9` -> `0.4.13` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.4.13`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0413) [Compare Source](astral-sh/uv@0.4.12...0.4.13) ##### Enhancements - Add `socks` support ([#7503](astral-sh/uv#7503)) - Avoid warning about bad Python interpreter links for empty project environment directories ([#7527](astral-sh/uv#7527)) - Improve invalid environment warning messages ([#7544](astral-sh/uv#7544)) - Use more verbose spelling of "virtualenv" during creation ([#7523](astral-sh/uv#7523)) - Do not use a user-facing warning for "Waiting to acquire lock..." message ([#7502](astral-sh/uv#7502)) ##### Performance - Use a single buffer for hints on resolver errors ([#7497](astral-sh/uv#7497)) ##### Bug fixes - Allow Python pre-releases to be used if they are first on the `PATH` ([#7470](astral-sh/uv#7470)) - Avoid deleting the project environment directory if it is not a virtual environment ([#7522](astral-sh/uv#7522)) - Do not error if the `CACHEDIR.TAG` file exists but cannot be written to ([#7550](astral-sh/uv#7550)) - Treat invalid platform as more compatible than invalid Python ([#7556](astral-sh/uv#7556)) - Use portable paths when serializing sources ([#7504](astral-sh/uv#7504)) - Compute resolver hints using the final reduced derivation tree ([#7546](astral-sh/uv#7546)) - Bump the wheel and sdist cache versions ([#7560](astral-sh/uv#7560)) - Heal cache entries with missing source distributions ([#7559](astral-sh/uv#7559)) ##### Rust libraries - Bump minimum supported Rust version from 1.80 -> 1.81 ##### Documentation - Add `UV_LINK_MODE` to Docker caching example ([#7510](astral-sh/uv#7510)) - Clarify behavior of of overrides in CLI reference ([#7537](astral-sh/uv#7537)) ### [`v0.4.12`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0412) [Compare Source](astral-sh/uv@0.4.11...0.4.12) ##### Enhancements - Allow users to provide pre-defined metadata for resolution ([#7442](astral-sh/uv#7442)) - Invalidate existing tool environments on Python interpreter mismatch ([#7451](astral-sh/uv#7451)) ##### Bug fixes - Avoid fatal error when searching for egg-info with missing directory ([#7498](astral-sh/uv#7498)) ##### Documentation - Add note on cache growth for self-hosted GitHub runners ([#5757](astral-sh/uv#5757)) ### [`v0.4.11`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0411) [Compare Source](astral-sh/uv@0.4.10...0.4.11) ##### Enhancements - Add `--no-editable` support to `uv sync` and `uv export` ([#7371](astral-sh/uv#7371)) - Add support for `--only-dev` to `uv sync` and `uv export` ([#7367](astral-sh/uv#7367)) - Add support for remaining pip-supported file extensions ([#7387](astral-sh/uv#7387)) - Generate shell completion for `uvx` ([#7388](astral-sh/uv#7388)) - Include `uv export` command in `requirements.txt` output ([#7374](astral-sh/uv#7374)) - Prune unzipped source distributions in `uv cache prune --ci` ([#7446](astral-sh/uv#7446)) - Warn when trying to `uv sync` a package without build configuration ([#7420](astral-sh/uv#7420)) - Support requests for pre-releases in the `--python` option ([#7335](astral-sh/uv#7335)) ##### Bug fixes - Avoid erroneous version warning for `.dist-info` directories ([#7444](astral-sh/uv#7444)) - Avoid removing seed packages for `uv venv --seed` environments ([#7410](astral-sh/uv#7410)) - Avoid unnecessary progress bar initializations ([#7412](astral-sh/uv#7412)) - Error when `tool.uv.sources` contains duplicate package names ([#7383](astral-sh/uv#7383)) - Include `--branch` et al when resolving unnamed URLs in `uv add` ([#7447](astral-sh/uv#7447)) - Include `dev-dependencies` in `--no-sources` invocations ([#7408](astral-sh/uv#7408)) - Include the parent interpreter in Python discovery when `--system` is used ([#7440](astral-sh/uv#7440)) - Respect `--no-sources` in PEP 723 scripts ([#7409](astral-sh/uv#7409)) - Respect `pyproject.toml` credentials from user-provided requirements ([#7474](astral-sh/uv#7474)) - Use consistent PyPI cache bucket ([#7443](astral-sh/uv#7443)) - Use unambiguous relative paths in `uv export` ([#7378](astral-sh/uv#7378)) ##### Documentation - Add documentation on platform-specific dependencies ([#7411](astral-sh/uv#7411)) - Add documentation for passing installer options on Linux ([#6839](astral-sh/uv#6839)) - Separate project data from configuration settings ([#7053](astral-sh/uv#7053)) ##### Error messages - Hint at missing `project.name` ([#6803](astral-sh/uv#6803)) - Surface dedicated `project.name` error for workspaces ([#7399](astral-sh/uv#7399)) - Remove duplicate warning for settings discovery errors ([#7384](astral-sh/uv#7384)) ### [`v0.4.10`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0410) [Compare Source](astral-sh/uv@0.4.9...0.4.10) ##### Enhancements - Allow `uv tool upgrade --all` to continue on individual upgrade failure ([#7333](astral-sh/uv#7333)) - Support globs as cache keys in `tool.uv.cache-keys` ([#7268](astral-sh/uv#7268)) - Add Python package (`__main__.py`) support to `uv run` ([#7281](astral-sh/uv#7281)) - Add zip application support to `uv run` ([#7289](astral-sh/uv#7289)) - Add `--token` option to `self update` command ([#7279](astral-sh/uv#7279)) ##### Performance - Use `globwalk` for `cache-keys` matching ([#7337](astral-sh/uv#7337)) ##### Bug fixes - Always treat archive-like requirements as local files ([#7364](astral-sh/uv#7364)) - Apply `--no-install` options when constructing resolution ([#7277](astral-sh/uv#7277)) - Avoid clobbering existing `py.typed` files contents in `uv init` ([#7338](astral-sh/uv#7338)) - Avoid enforcing platform compatibility when validating lockfile ([#7305](astral-sh/uv#7305)) - Avoid installing transitive dev dependencies ([#7318](astral-sh/uv#7318)) - Avoid selecting prerelease Python installations without opt-in ([#7300](astral-sh/uv#7300)) - Fix PPC64 page size in binary builds. ([#7298](astral-sh/uv#7298)) - Include pre-release Python versions in `uv python list` ([#7290](astral-sh/uv#7290)) - Make version ID optional for source builds ([#7362](astral-sh/uv#7362)) - Support relative paths in `uv add --script` ([#7301](astral-sh/uv#7301)) ##### Documentation - Fix documentation typos for `uv build --build-constraint` flag ([#7330](astral-sh/uv#7330)) - Fix grammatical error in CLI docs ([#7353](astral-sh/uv#7353)) ##### Error messages - Add dedicated lock errors for wheel-only distributions ([#7307](astral-sh/uv#7307)) - Avoid treating `.whl` sources as source distributions ([#7303](astral-sh/uv#7303)) - Clarify Python requirement source for script incompatibilities ([#7339](astral-sh/uv#7339)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

charliermarsh requested review from BurntSushi and konstin September 10, 2024 18:37

charliermarsh force-pushed the charlie/glob branch from 9abc707 to 7b17f5d Compare September 10, 2024 18:39

charliermarsh marked this pull request as ready for review September 10, 2024 18:39

charliermarsh added enhancement New feature or request configuration Settings and such labels Sep 10, 2024

konstin approved these changes Sep 10, 2024

View reviewed changes

Support globs as cache keys

81c4602

charliermarsh force-pushed the charlie/glob branch from 7b17f5d to 81c4602 Compare September 10, 2024 18:50

sbidoul mentioned this pull request Sep 11, 2024

Make metadata caching of local projects opt-in when it is dynamic? #7282

Open

BurntSushi approved these changes Sep 11, 2024

View reviewed changes

charliermarsh merged commit 58a157a into main Sep 11, 2024
58 checks passed

charliermarsh deleted the charlie/glob branch September 11, 2024 19:31

BrewTestBot mentioned this pull request Sep 13, 2024

uv 0.4.10 Homebrew/homebrew-core#189400

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support globs as cache keys in `tool.uv.cache-keys` #7268

Support globs as cache keys in `tool.uv.cache-keys` #7268

charliermarsh commented Sep 10, 2024 •

edited

Loading

konstin Sep 10, 2024

charliermarsh Sep 10, 2024

charliermarsh Sep 10, 2024

BurntSushi Sep 11, 2024

charliermarsh commented Sep 10, 2024

BurntSushi Sep 11, 2024

BurntSushi Sep 11, 2024

charliermarsh Sep 11, 2024

BurntSushi Sep 11, 2024

BurntSushi Sep 11, 2024

charliermarsh Sep 11, 2024

BurntSushi Sep 11, 2024

Support globs as cache keys in tool.uv.cache-keys #7268

Support globs as cache keys in tool.uv.cache-keys #7268

Conversation

charliermarsh commented Sep 10, 2024 • edited Loading

Summary

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charliermarsh commented Sep 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Support globs as cache keys in `tool.uv.cache-keys` #7268

Support globs as cache keys in `tool.uv.cache-keys` #7268

charliermarsh commented Sep 10, 2024 •

edited

Loading