Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

universal-lock: if we write file paths to a lock file, we need to make them portable #4307

Closed
Tracked by #3347
BurntSushi opened this issue Jun 13, 2024 · 3 comments
Closed
Tracked by #3347
Labels
preview Experimental behavior

Comments

@BurntSushi
Copy link
Member

While working on #4266, I realized that the file paths we write to a lock file are platform specific. Specifically, on Windows, we use \ as a path separator, while on Unix, we use / as a path separator. I believe this implies that if developers on Windows and Unix work on the same project, they will generate different lock files. This is, I believe, untenable.

I think we either need to find a way to not write file paths to a lock file at all (this is what Cargo does), or we need to ensure we normalize all file paths into a portable representation.

It's not clear to me that the latter is strictly possible in all cases, but I think we are always writing Unix paths to requirements.txt by either converting \ to /, or using file URLs.

@zanieb
Copy link
Member

zanieb commented Jun 13, 2024

See also #3804

@charliermarsh
Copy link
Member

In the interim, should we just call to_slash on the paths? That library exists for this purpose IIRC.

@charliermarsh charliermarsh added the preview Experimental behavior label Jun 14, 2024
BurntSushi pushed a commit that referenced this issue Jun 14, 2024
## Summary

This would be a lightweight solution to
#4307 that doesn't fully engage
with all the possibilities in the design space (but would unblock
cross-platform for now).
@BurntSushi
Copy link
Member Author

Closed by #4324.

We should perhaps still explore the design space of "don't write paths to a lock file," but Charlie makes the point that this would require reading the pyproject.toml in order to understand the lock file. I believe, at present, the lock file is "hermetic," which could be conceivably useful (and it is at least conceptually simpler). My very vague thinking on the matter at present is the following:

  1. I wouldn't be surprised if we wind up needing to read pyproject.toml and give up the hermetic lock file approach for some other reason. But I don't know what that reason is yet.
  2. I have a nagging suspicion that even by converting paths to Unix paths, there are still some cases that we'll get wrong in the sense that it isn't portable. But I'm finding it difficult to articulate those cases since, even if the paths aren't in the lock file, they have to be somewhere (like pyproject.toml, which needs to be valid UTF-8 and thus inherently already prevents some kinds of file paths from being expressible as-is).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Experimental behavior
Projects
None yet
Development

No branches or pull requests

3 participants