Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gracefully handle missing files/directories in cache storage #6147

Closed
paveldikov opened this issue Aug 16, 2024 · 2 comments · Fixed by #6284
Closed

gracefully handle missing files/directories in cache storage #6147

paveldikov opened this issue Aug 16, 2024 · 2 comments · Fixed by #6284
Assignees
Labels
bug Something isn't working

Comments

@paveldikov
Copy link
Contributor

Not sure if this is a common pattern, but we like to store the cache directory on /var/tmp as the usual home directory may be on a network share (and extremely slow I/O as a result).

This works great for us, but it also happens that /var/tmp is affected by a periodic cleanup process that deletes old files/directories. Once this happens, we get a bunch of error messages along the lines of:

  Caused by: Failed to install: packaging-24.1-py3-none-any.http.whl (packaging==24.1)
  Caused by: failed to read directory `/var/tmp/pavel/.cache/uv/archive-v0/REDACTED`
  Caused by: No such file or directory (os error 2)

rm -rf'ing the cache dir (or uv cache clean -- but not uv cache prune) seems to work around this problem fine, but it's quite a severe thing to do.

@charliermarsh
Copy link
Member

Agree, I think we should be robust to this. I don't know if we can be robust to /var/tmp being cleaned up while uv is running though.

@charliermarsh charliermarsh added the bug Something isn't working label Aug 16, 2024
@paveldikov
Copy link
Contributor Author

I think catching ENOENT and friends (at arbitrary depth?) and re-caching the distribution from scratch should be robust enough? I am trying to think of the various types of race conditions in there, but assuming that the hardlinking/copying into venv comes right at the end, this should be reasonably complete.

(symlink mode will not be helped by this, but that is probably fine?)

@charliermarsh charliermarsh self-assigned this Aug 20, 2024
charliermarsh added a commit that referenced this issue Aug 20, 2024
## Summary

Closes #6147.

## Test Plan

- `cargo run pip install flask --no-binary flask --cache-dir foo
--reinstall`
- `rm -rf foo/archive-v0`
- `cargo run pip install flask --no-binary flask --cache-dir foo
--reinstall`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants