Reduce temporary path length of extensions extracted from wheels #224
Comments
stewartmiles
added a commit
to stewartmiles/distlib
that referenced
this issue
Sep 20, 2024
Wheel names can be long and paths to mounted wheels can be long. Previously `util.path_to_cache_dir()` was unconditionally building cache paths using the absolute path of the cache directory joined with the absolute path to a wheel being mounted. This makes it very easy to generate in huge paths that exceed the maximum path length (MAX_PATH) on Windows machines without the [registry change](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry) to support long paths. A user-wide shared location for cached data is potentially easy enough to poison with incorrect extensions, so using the absolute path of a wheel being mounted to cache extensions doesn't seem to add any more protection and results in very long hard to read paths that can exceed Windows path limits in out of the box installations. Fixes pypa#224
stewartmiles
added a commit
to stewartmiles/distlib
that referenced
this issue
Sep 20, 2024
Wheel names can be long and paths to mounted wheels can be long. Previously `util.path_to_cache_dir()` was unconditionally building cache paths using the absolute path of the cache directory joined with the absolute path to a wheel being mounted. This makes it very easy to generate in huge paths that exceed the maximum path length (MAX_PATH) on Windows machines without the [registry change](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry) to support long paths. A user-wide shared location for cached data is potentially easy enough to poison with incorrect extensions, so using the absolute path of a wheel being mounted to cache extensions doesn't seem to add any more protection and results in very long hard to read paths that can exceed Windows path limits in out of the box installations. Fixes pypa#224
stewartmiles
added a commit
to stewartmiles/distlib
that referenced
this issue
Sep 20, 2024
Wheel names can be long and paths to mounted wheels can be long. Previously `util.path_to_cache_dir()` was unconditionally building cache paths using the absolute path of the cache directory joined with the absolute path to a wheel being mounted. This makes it very easy to generate in huge paths that exceed the maximum path length (MAX_PATH) on Windows machines without the [registry change](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry) to support long paths. A user-wide shared location for cached data is potentially easy enough to poison with incorrect extensions, so using the absolute path of a wheel being mounted to cache extensions doesn't seem to add any more protection and results in very long hard to read paths that can exceed Windows path limits in out of the box installations. Fixes pypa#224
stewartmiles
added a commit
to stewartmiles/distlib
that referenced
this issue
Sep 23, 2024
Wheel names can be long and paths to mounted wheels can be long. Previously `util.path_to_cache_dir()` was unconditionally building cache paths using the absolute path of the cache directory joined with the absolute path to a wheel being mounted. This makes it very easy to generate in huge paths that exceed the maximum path length (MAX_PATH) on Windows machines without the [registry change](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry) to support long paths. A user-wide shared location for cached data is potentially easy enough to poison with incorrect extensions, so using the absolute path of a wheel being mounted to cache extensions doesn't seem to add any more protection and results in very long hard to read paths that can exceed Windows path limits in out of the box installations. Fixes pypa#224
vsajip
pushed a commit
that referenced
this issue
Oct 7, 2024
Wheel names can be long and paths to mounted wheels can be long. Previously `util.path_to_cache_dir()` was unconditionally building cache paths using the absolute path of the cache directory joined with the absolute path to a wheel being mounted. This makes it very easy to generate in huge paths that exceed the maximum path length (MAX_PATH) on Windows machines without the [registry change](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry) to support long paths. A user-wide shared location for cached data is potentially easy enough to poison with incorrect extensions, so using the absolute path of a wheel being mounted to cache extensions doesn't seem to add any more protection and results in very long hard to read paths that can exceed Windows path limits in out of the box installations. Fixes #224
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
While looking into #222 I noticed that wheels are extracted into a subdirectory of the platform cache directory (e.g
~/.distlib). For example, when mounting a wheel with extensions from/home/user/path/to/my/wheel.whlthe extension in the wheel (e.gfoo.so) will be unpacked to~/.distlib/dylib-cache/PYTHON_VERSION/--home--user--path--to--my--wheel.whl.cache/foo.so. While this is just a bit ugly on Windows systems without long filenames enabled this can easily result in paths that exceed the maximum path length.Describe the solution you'd like
It would be great to reduce the path length by unpacking to a directory relative to just the wheel filename rather than the entire path to the wheel on the filesystem.
Describe alternatives you've considered
N/A
Additional context
N/A
The text was updated successfully, but these errors were encountered: