Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

uv 0.4.12 failing to read distribution cache in github actions #7543

Closed
nijel opened this issue Sep 19, 2024 · 8 comments · Fixed by #7559
Closed

uv 0.4.12 failing to read distribution cache in github actions #7543

nijel opened this issue Sep 19, 2024 · 8 comments · Fixed by #7559
Assignees
@charliermarsh
Labels
bug Something isn't working cache Caching of packages and metadata

Comments

@nijel
Copy link
Contributor

nijel commented Sep 19, 2024 •
edited
Loading

This seems almost like #7485, but I still see it happen with 0.4.12:

 error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: mercurial==6.8.1
  Caused by: /home/runner/work/_temp/setup-uv-cache/built-wheels-v3/pypi/mercurial/6.8.1/kYDLbz0owpJdwyMLRFXvG/mercurial-6.8.1.tar.gz does not appear to be a Python project, as neither `pyproject.toml` nor `setup.py` are present in the directory

The cache was created by 0.4.12 as uv version is in GitHub cache key:

 uv cache restored from GitHub Actions cache with key: setup-uv-1-x86_64-unknown-linux-gnu-0.4.12
-dc3406fd913fe4622566cb4edc384b2e3b933f9767e05f304e1f3ec73a14f457

When executed without a cache, it runs just fine.

PS: Action log can be seen here: https://github.com/WeblateOrg/weblate/actions/runs/10940191066/job/30372045488
@zanieb zanieb added bug Something isn't working cache Caching of packages and metadata labels Sep 19, 2024
@charliermarsh
Copy link
Member

At least on a first pass I can't reproduce that locally, but I'll keep trying.

@charliermarsh
Copy link
Member

Actually, I think I have an idea as to how this could happen if you're sharing that cache between multiple Python versions. Is that possible? I reproduced with some manual cache editing:

error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: mercurial==6.8.1
  Caused by: /Users/crmarsh/.cache/uv/built-wheels-v3/pypi/mercurial/6.8.1/bZrFqUG39NkQoZTpIu0WD/mercurial-6.8.1.tar.gz does not appear to be a Python project, as neither `pyproject.toml` nor `setup.py` are present in the directory

@charliermarsh charliermarsh self-assigned this Sep 19, 2024
@nijel
Copy link
Contributor Author

nijel commented Sep 19, 2024

Ah, that is most likely it. We use multiple Python versions, and I didn't realize that the cache key doesn't include it.

@zanieb
Copy link
Member

zanieb commented Sep 19, 2024

I feel like the cache should be robust to multiple Python versions

@charliermarsh
Copy link
Member

Yeah it should.

@charliermarsh
Copy link
Member

In general it is robust to multiple Python versions, platforms, etc. But there's a bug here whereby if you run uv cache prune --ci (which runs via the GitHub Action), we perform some modifications that break assumptions.

@charliermarsh
Copy link
Member

I'll fix this today.

nijel added a commit to WeblateOrg/weblate that referenced this issue Sep 19, 2024
@nijel
chore(ci): separate uv cache per Python version
a4f461a 
It doesn't make sense share the cache of complied wheels becuase these
are version specific.

See also astral-sh/uv#7543
@nijel
Copy link
Contributor Author

nijel commented Sep 19, 2024

In our case, the GitHub cache really should be per Python version, so I've just changed that. Without that, it is not really useful as compiled wheels are not compatible across Python versions.

@charliermarsh charliermarsh mentioned this issue Sep 19, 2024
Merged
charliermarsh added a commit that referenced this issue Sep 19, 2024
@charliermarsh
Heal cache entries with missing source distributions (#7559)
f3463b3 
## Summary

`uv cache prune --ci` will remove the source distribution directory. If
we then need to build a _different_ wheel (e.g., you're building a
package that has Python minor version-specific wheels), we fail, because
we expect the source to be there.

Now, if the source is missing, we re-download it. It would be slightly
easier to just _ignore_ that revision, but that would mean we'd also
lose the already-built wheels -- so if you ran against many Python
versions, we'd continuously lose the cached data.

Closes #7543.

## Test Plan

We can add tests, but they _need_ to build non-pure Python wheels, which
tends to be expensive...

For reference:

```console
$ cargo run venv --python 3.12
$ cargo run pip install mercurial==6.8.1 --verbose
$ cargo run cache prune --ci
$ cargo run venv --python 3.11
$ cargo run pip install mercurial==6.8.1 --verbose
```

I also did this with a local `.tar.gz` that I downloaded from PyPI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@charliermarsh charliermarsh

Labels
bug Something isn't working cache Caching of packages and metadata
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Heal cache entries with missing source distributions astral-sh/uv
3 participants
@nijel @charliermarsh @zanieb