PurePosixPath no longer correctly parses PureWindowsPath

[domdfcoding]

Bug report

In pathlib prior to Python 3.12, passing a PureWindowsPath to a PurePosixPath resulted in the Windows separator (\) being converted to the POSIX separator (/). However, in the current main branch the backslashes are preserved in the PurePosixPath object.

Here is an example which illustrates this:

Python 3.12.0a7 (tags/v3.12.0a7:b861ba4, Apr  6 2023, 16:09:18) [Clang 10.0.0 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> print(pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")))
a\b\c
>>> print(pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")).as_posix())
a\b\c 

Python 3.11.2 (tags/v3.11.2:878ead1, Mar  9 2023, 16:26:59) [Clang 10.0.0 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> print(pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")))
a/b/c
>>> print(pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")).as_posix())
a/b/c 

(The behaviour is the same if using PosixPath or WindowsPath on the relevant platform; it's not specific to the "pure" variants.)

---

Before the recent refactoring of the module, passing one Path or PurePath object to another resulted in the _parts attribute being inspected. This was a list of the individual path elements (e.g. ['a', 'b', 'c'] for the path a\b\c). The _parts attribute was removed in GH-102476 and replaced with _tail, but with slightly different semantics.

The current code replaces any os.altsep in the path with os.sep, which for WindowsPath replaces / with \ but for PosixPath does nothing as there is no alternative separator. However, the following will produce a correct result:

Python 3.12.0a7+ (heads/main:bd2ed06, Apr 19 2023, 15:27:47) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> pathlib.PurePosixPath._flavour.altsep = "\\"
>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")).as_posix()
'a/b/c'

Thus I think the problem can be isolated to these lines here:

cpython/Lib/pathlib.py

Lines 316 to 324 in da2273f

	@classmethod
	def _parse_path(cls, path):
	if not path:
	return '', '', []
	sep = cls._flavour.sep
	altsep = cls._flavour.altsep
	if altsep:
	path = path.replace(altsep, sep)
	drv, root, rel = cls._flavour.splitroot(path)

Your environment

• CPython versions tested on: 3.8, 3.11 and 3.12 (alpha 7 and HEAD)
• Operating system and architecture: Ubuntu 20.04 and Windows 10

Linked PRs

• GH-103631: Fix PurePosixPath(PureWindowsPath(...)) separator handling #104949
• [3.12] GH-103631: Fix PurePosixPath(PureWindowsPath(...)) separator handling (GH-104949) #104991

[eryksun]

If constructing a POSIX path directly from a Windows path is supported, it won't be implemented by adding backslash as an alternate path separator for POSIX paths. Backslash is a normal name character in POSIX, not a reserved path separator.

As a workaround, you can get a POSIX path before passing the path to the PurePosixPath constructor.

>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c").as_posix())
PurePosixPath('a/b/c')

[barneygale]

In pathlib prior to Python 3.12, passing a PureWindowsPath to a PurePosixPath resulted in the Windows separator (\) being converted to the POSIX separator (/)

Only for relative paths. If you try to pass an absolute path, things quickly go south:

barney@acorn ~ $ python3.8
Python 3.8.10 (default, Mar 13 2023, 10:26:41) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"c:\a\b\c"))
PurePosixPath('c:\\/a/b/c')
>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"\\server\share\a\b\c"))
PurePosixPath('\\\\server\\share\\/a/b/c')

[barneygale]

FYI, the behaviour was changed in #101667 and #102454 in order to fix this bug:

• Inconsistent behavior of pathlib.WindowsPath with drive paths #80486

As a result, these all do the same thing:

>>> from pathlib import PurePosixPath, PureWindowsPath
>>> from os import fspath
>>> PurePosixPath(r"c:\a\b\c")
PurePosixPath('c:\\a\\b\\c')
>>> PurePosixPath(PureWindowsPath(r"c:\a\b\c"))
PurePosixPath('c:\\a\\b\\c')
>>> PurePosixPath(fspath(PureWindowsPath(r"c:\a\b\c")))
PurePosixPath('c:\\a\\b\\c')

We tweaked the docs slightly to make clear that all os.PathLike objects are treated similarly:

diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst
index c8a734ecad8e..8e91936680fa 100644
--- a/Doc/library/pathlib.rst
+++ b/Doc/library/pathlib.rst
@@ -105,8 +105,9 @@ we also call *flavours*:
       PurePosixPath('setup.py')
 
    Each element of *pathsegments* can be either a string representing a
-   path segment, an object implementing the :class:`os.PathLike` interface
-   which returns a string, or another path object::
+   path segment, or an object implementing the :class:`os.PathLike` interface
+   where the :meth:`~os.PathLike.__fspath__` method returns a string,
+   such as another path object::
 
       >>> PurePath('foo', 'some/path', 'bar')
       PurePosixPath('foo/some/path/bar')

Eryk's suggestion looks ideal to me, and clearly signals intentions.

[mwichmann]

As a workaround, you can get a POSIX path before passing the path to the PurePosixPath constructor.

>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c").as_posix())
PurePosixPath('a/b/c')

The problem with this is it means your code has to be system-aware. The case I've run across this, just figuring out it was broken now, uses a PureWindowsPath when processing a file of pathnames, so it can be read in an agnostic way (that is, either forward or backward slashes in the file work no matter what the platform), then converts it to concrete paths by calling Path() on the result, which used to work to produce suitable "native" paths. Now it's going to have to LBYL to figure out whether to as_posix or not.

[barneygale]

Hm! Backslashes can appear in Posix filenames. How would I specify such a path in your file of pathnames? Or is that not a problem because you create all these paths from within your program?

[barneygale]

For your use case, would this work?

pathlib.Path(pathlib.PureWindowsPath(yourpath).as_posix())

as_posix() returns a path with forward slashes as separators, which is valid for both WindowsPath and PosixPath.

Alternatively:

pathlib.Path(yourpath.replace('\\', '/'))

[mwichmann]

For your use case, would this work?

pathlib.Path(pathlib.PureWindowsPath(r"a\b\c").as_posix())

as_posix() returns a path with forward slashes as separators, which is valid for both WindowsPath and PosixPath.

The usecase is more specifically a list of tests to run (cmdline or from a file), and we know we're not creating tests with particularly "odd" names, so it's a (somewhat) controlled namespace. We've wanted to display the names the way they were entered if they were on Windows, so if someone actually typed foo\bar it should look like foo\bar was being run, but that's just a nicety; normalizing all paths to "POSIX style" isn't unreasonable - somebody already tried to push that idea.

[barneygale]

Gotcha.

Path() will switch the forward slashes back to backslashes on Windows, which sounds like what you want?

[barneygale]

Upon further consideration I don't think this is worth the backwards-compatibility break. PR up: #104949

[barneygale]

Thanks for reporting + discussing, all. This is fixed in:

• 328422c / GH-103631: Fix PurePosixPath(PureWindowsPath(...)) separator handling #104949
• 305d78b / [3.12] GH-103631: Fix PurePosixPath(PureWindowsPath(...)) separator handling (GH-104949) #104991

3.12 beta 2 should contain the fix.