Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-88569: add ntpath.isreserved() #95486

Merged
merged 38 commits into from
Jan 26, 2024
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
6b89ecc
gh-88569: add `os.path.isreserved()`
barneygale Jul 31, 2022
3e08e4a
Fix tests
barneygale Jul 31, 2022
49ba439
Remove implementation in `genericpath`
barneygale Jul 31, 2022
238b3e3
Apply suggestions from code review
barneygale Aug 9, 2022
f647502
Apply suggestions from code review
barneygale Aug 10, 2022
f07c7ad
Remove tests for pathlib.PurePath.is_reserved().
barneygale Aug 10, 2022
dc857c9
Speed up isreserved('.') and '..'
barneygale Aug 12, 2022
f0fd2c8
Note change to algorithm in pathlib docs.
barneygale Aug 12, 2022
3b51db8
Update Lib/ntpath.py
barneygale Aug 12, 2022
06cb428
Update Doc/library/os.path.rst
barneygale Aug 13, 2022
79c0be4
Update Lib/posixpath.py
barneygale Aug 13, 2022
0a0db6a
Restore `os.fspath()` call in `posixpath.isreserved()`
barneygale Aug 16, 2022
7145b86
Apply suggestions from code review
barneygale Aug 23, 2022
9f74b64
posixpath.isreserved(): return True for paths with NUL characters
barneygale Aug 23, 2022
91b2bb3
ntpath.isreserved(): minor tweaks
barneygale Aug 23, 2022
e6aff58
ntpath.isreserved(): restore initial splitdrive() call
barneygale Aug 23, 2022
e6a2c0b
ntpath.isreserved(): avoid calling `splitdrive()` repeatedly.
barneygale Aug 23, 2022
14dde15
Update Lib/ntpath.py
barneygale Aug 23, 2022
fab274a
Add `isreservedname()` for discussion.
barneygale Aug 23, 2022
936dcc8
Apply suggestions from code review
barneygale Aug 23, 2022
3fb127f
Undo posixpath changes
barneygale Jan 8, 2024
002d951
Merge branch 'main' into os-path-isreserved
barneygale Jan 8, 2024
c8ed711
Update version numbers.
barneygale Jan 8, 2024
c772b25
Fix syntax, whitespace.
barneygale Jan 8, 2024
3fbef57
Make `isreservedname()` private
barneygale Jan 8, 2024
4b34274
Update Lib/ntpath.py
barneygale Jan 8, 2024
a877677
Tighten up `PurePath.is_reserved()` exception handling.
barneygale Jan 8, 2024
3721c8c
Use `str(self)` to support non-os.PathLike implementations.
barneygale Jan 8, 2024
b905d2f
Deprecate `pathlib.PurePath.is_reserved()`
barneygale Jan 8, 2024
2756ffb
Add note about approximate and changing Windows rules; remove doctest.
barneygale Jan 8, 2024
c03c672
Update Doc/library/os.path.rst
barneygale Jan 8, 2024
4085ff5
Update what's new.
barneygale Jan 8, 2024
b4b3d0b
Mention deprecation in NEWS
barneygale Jan 8, 2024
9e2d21f
Address review feedback
barneygale Jan 8, 2024
44c37cb
Point to `os.path.isreserved()` in whatsnew deprecation notices.
barneygale Jan 8, 2024
f9033b3
Merge branch 'main' into os-path-isreserved
barneygale Jan 8, 2024
efb7681
Merge branch 'main' into os-path-isreserved
barneygale Jan 14, 2024
e398c3f
Address review feedback
barneygale Jan 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions Doc/library/os.path.rst
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,28 @@ the :mod:`glob` module.)
.. versionadded:: 3.12


.. function:: isreserved(path)

Return ``True`` if *path* is a reserved pathname on the current system.

On Windows, reserved filenames include those that end with a space or dot;
those that contain colons (i.e. file streams such as "name:stream"),
wildcard characters (i.e. ``'*?"<>'``), pipe, or ASCII control characters;
as well as DOS device names such as "NUL", "CON", "CONIN$", "CONOUT$",
"AUX", "PRN", "COM1", and "LPT1".

.. note::

This function approximates rules for reserved paths on most Windows
systems. These rules change over time in various Windows releases.
This function may be updated in future Python releases as changes to
the rules become broadly available.

.. availability:: Windows.

.. versionadded:: 3.13


.. function:: join(path, *paths)

Join one or more path segments intelligently. The return value is the
Expand Down
13 changes: 6 additions & 7 deletions Doc/library/pathlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -541,14 +541,13 @@ Pure paths provide the following methods and properties:
reserved under Windows, ``False`` otherwise. With :class:`PurePosixPath`,
``False`` is always returned.

>>> PureWindowsPath('nul').is_reserved()
True
>>> PurePosixPath('nul').is_reserved()
False

File system calls on reserved paths can fail mysteriously or have
unintended effects.
.. versionchanged:: 3.13
Windows path names that contain a colon, or end with a dot or a space,
are considered reserved. UNC paths may be reserved.

.. deprecated-removed:: 3.13 3.15
barneygale marked this conversation as resolved.
Show resolved Hide resolved
This method is deprecated; use :func:`os.path.isreserved` to detect
reserved paths on Windows.

.. method:: PurePath.joinpath(*pathsegments)

Expand Down
15 changes: 15 additions & 0 deletions Doc/whatsnew/3.13.rst
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,9 @@ os
os.path
-------

* Add :func:`os.path.isreserved` to check if a path is reserved on the current
system. This function is only available on Windows.
(Contributed by Barney Gale in :gh:`88569`.)
* On Windows, :func:`os.path.isabs` no longer considers paths starting with
exactly one (back)slash to be absolute.
(Contributed by Barney Gale and Jon Foster in :gh:`44626`.)
Expand Down Expand Up @@ -486,6 +489,12 @@ Deprecated
security and functionality bugs. This includes removal of the ``--cgi``
flag to the ``python -m http.server`` command line in 3.15.

* :mod:`pathlib`:

* :meth:`pathlib.PurePath.is_reserved` is deprecated and scheduled for
removal in Python 3.15. Use :func:`os.path.isreserved` to detect reserved
paths on Windows.

* :mod:`sys`: :func:`sys._enablelegacywindowsfsencoding` function.
Replace it with :envvar:`PYTHONLEGACYWINDOWSFSENCODING` environment variable.
(Contributed by Inada Naoki in :gh:`73427`.)
Expand Down Expand Up @@ -697,6 +706,12 @@ Pending Removal in Python 3.15
:func:`locale.getlocale()` instead.
(Contributed by Hugo van Kemenade in :gh:`111187`.)

* :mod:`pathlib`:

* :meth:`pathlib.PurePath.is_reserved` is deprecated and scheduled for
removal in Python 3.15. Use :func:`os.path.isreserved` to detect reserved
paths on Windows.

* :class:`typing.NamedTuple`:

* The undocumented keyword argument syntax for creating NamedTuple classes
Expand Down
41 changes: 39 additions & 2 deletions Lib/ntpath.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
__all__ = ["normcase","isabs","join","splitdrive","splitroot","split","splitext",
"basename","dirname","commonprefix","getsize","getmtime",
"getatime","getctime", "islink","exists","lexists","isdir","isfile",
"ismount", "expanduser","expandvars","normpath","abspath",
"curdir","pardir","sep","pathsep","defpath","altsep",
"ismount","isreserved","expanduser","expandvars","normpath",
"abspath","curdir","pardir","sep","pathsep","defpath","altsep",
"extsep","devnull","realpath","supports_unicode_filenames","relpath",
"samefile", "sameopenfile", "samestat", "commonpath", "isjunction"]

Expand Down Expand Up @@ -330,6 +330,43 @@ def ismount(path):
return False


_reserved_chars = frozenset(
{chr(i) for i in range(32)} |
{'"', '*', ':', '<', '>', '?', '|', '/', '\\'}
)

_reserved_names = frozenset(
{'CON', 'PRN', 'AUX', 'NUL', 'CONIN$', 'CONOUT$'} |
{f'COM{c}' for c in '123456789\xb9\xb2\xb3'} |
{f'LPT{c}' for c in '123456789\xb9\xb2\xb3'}
)

def isreserved(path):
"""Return true if the pathname is reserved by the system."""
# Refer to "Naming Files, Paths, and Namespaces":
# https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file
path = os.fsdecode(splitroot(path)[2]).replace(altsep, sep)
return any(_isreservedname(name) for name in reversed(path.split(sep)))

def _isreservedname(name):
"""Return true if the filename is reserved by the system."""
name = os.fsdecode(name)
barneygale marked this conversation as resolved.
Show resolved Hide resolved
# Trailing dots and spaces are reserved.
if name.endswith(('.', ' ')) and name not in ('.', '..'):
return True
# Wildcards, separators, colon, and pipe (*?"<>/\:|) are reserved.
# ASCII control characters (0-31) are reserved.
# Colon is reserved for file streams (e.g. "name:stream[:type]").
if _reserved_chars.intersection(name):
return True
# DOS device names are reserved (e.g. "nul" or "nul .txt"). The rules
# are complex and vary across Windows versions. On the side of
# caution, return True for names that may not be reserved.
if name.partition('.')[0].rstrip(' ').upper() in _reserved_names:
return True
return False


# Expand paths beginning with '~' or '~user'.
# '~' means $HOME; '~user' means that user's home directory.
# If the path doesn't begin with '~', or if the user or $HOME is unknown,
Expand Down
28 changes: 7 additions & 21 deletions Lib/pathlib/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,15 +33,6 @@
]


# Reference for Windows paths can be found at
# https://learn.microsoft.com/en-gb/windows/win32/fileio/naming-a-file .
_WIN_RESERVED_NAMES = frozenset(
{'CON', 'PRN', 'AUX', 'NUL', 'CONIN$', 'CONOUT$'} |
{f'COM{c}' for c in '123456789\xb9\xb2\xb3'} |
{f'LPT{c}' for c in '123456789\xb9\xb2\xb3'}
)


class _PathParents(Sequence):
"""This object provides sequence-like access to the logical ancestors
of a path. Don't try to construct it yourself."""
Expand Down Expand Up @@ -433,18 +424,13 @@ def is_absolute(self):
def is_reserved(self):
"""Return True if the path contains one of the special names reserved
by the system, if any."""
if self.pathmod is not ntpath or not self.name:
return False

# NOTE: the rules for reserved names seem somewhat complicated
# (e.g. r"..\NUL" is reserved but not r"foo\NUL" if "foo" does not
# exist). We err on the side of caution and return True for paths
# which are not considered reserved by Windows.
if self.drive.startswith('\\\\'):
# UNC paths are never reserved.
return False
name = self.name.partition('.')[0].partition(':')[0].rstrip(' ')
return name.upper() in _WIN_RESERVED_NAMES
msg = ("pathlib.PurePath.is_reserved() is deprecated and scheduled "
"for removal in Python 3.15. Use os.path.isreserved() to "
"detect reserved paths on Windows.")
warnings.warn(msg, DeprecationWarning, stacklevel=2)
if self.pathmod is ntpath:
return self.pathmod.isreserved(self)
return False

def as_uri(self):
"""Return the path as a URI."""
Expand Down
56 changes: 56 additions & 0 deletions Lib/test/test_ntpath.py
Original file line number Diff line number Diff line change
Expand Up @@ -981,6 +981,62 @@ def test_ismount(self):
self.assertTrue(ntpath.ismount(b"\\\\localhost\\c$"))
self.assertTrue(ntpath.ismount(b"\\\\localhost\\c$\\"))

def test_isreserved(self):
self.assertFalse(ntpath.isreserved(''))
barneygale marked this conversation as resolved.
Show resolved Hide resolved
self.assertFalse(ntpath.isreserved('.'))
self.assertFalse(ntpath.isreserved('..'))
self.assertFalse(ntpath.isreserved('/'))
self.assertFalse(ntpath.isreserved('/foo/bar'))
# A name that ends with a space or dot is reserved.
self.assertTrue(ntpath.isreserved('foo.'))
self.assertTrue(ntpath.isreserved('foo '))
# ASCII control characters are reserved.
self.assertTrue(ntpath.isreserved('\foo'))
# Wildcard characters, colon, and pipe are reserved.
self.assertTrue(ntpath.isreserved('foo*bar'))
self.assertTrue(ntpath.isreserved('foo?bar'))
self.assertTrue(ntpath.isreserved('foo"bar'))
self.assertTrue(ntpath.isreserved('foo<bar'))
self.assertTrue(ntpath.isreserved('foo>bar'))
self.assertTrue(ntpath.isreserved('foo:bar'))
self.assertTrue(ntpath.isreserved('foo|bar'))
# Case-insensitive DOS-device names are reserved.
self.assertTrue(ntpath.isreserved('nul'))
self.assertTrue(ntpath.isreserved('aux'))
self.assertTrue(ntpath.isreserved('prn'))
self.assertTrue(ntpath.isreserved('con'))
self.assertTrue(ntpath.isreserved('conin$'))
self.assertTrue(ntpath.isreserved('conout$'))
# COM/LPT + 1-9 or + superscript 1-3 are reserved.
self.assertTrue(ntpath.isreserved('COM1'))
self.assertTrue(ntpath.isreserved('LPT9'))
self.assertTrue(ntpath.isreserved('com\xb9'))
self.assertTrue(ntpath.isreserved('com\xb2'))
self.assertTrue(ntpath.isreserved('lpt\xb3'))
# DOS-device name matching ignores characters after a dot or
# a colon and also ignores trailing spaces.
self.assertTrue(ntpath.isreserved('NUL.txt'))
self.assertTrue(ntpath.isreserved('PRN '))
self.assertTrue(ntpath.isreserved('AUX .txt'))
self.assertTrue(ntpath.isreserved('COM1:bar'))
self.assertTrue(ntpath.isreserved('LPT9 :bar'))
# DOS-device names are only matched at the beginning
# of a path component.
self.assertFalse(ntpath.isreserved('bar.com9'))
self.assertFalse(ntpath.isreserved('bar.lpt9'))
# The entire path is checked, except for the drive.
self.assertTrue(ntpath.isreserved('c:/bar/baz/NUL'))
self.assertTrue(ntpath.isreserved('c:/NUL/bar/baz'))
self.assertFalse(ntpath.isreserved('//./NUL'))
# Bytes are supported.
self.assertFalse(ntpath.isreserved(b''))
barneygale marked this conversation as resolved.
Show resolved Hide resolved
self.assertFalse(ntpath.isreserved(b'.'))
self.assertFalse(ntpath.isreserved(b'..'))
self.assertFalse(ntpath.isreserved(b'/'))
self.assertFalse(ntpath.isreserved(b'/foo/bar'))
self.assertTrue(ntpath.isreserved(b'foo.'))
self.assertTrue(ntpath.isreserved(b'nul'))

def assertEqualCI(self, s1, s2):
"""Assert that two strings are equal ignoring case differences."""
self.assertEqual(s1.lower(), s2.lower())
Expand Down
48 changes: 6 additions & 42 deletions Lib/test/test_pathlib/test_pathlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,12 @@ def test_is_relative_to_several_args(self):
with self.assertWarns(DeprecationWarning):
p.is_relative_to('a', 'b')

def test_is_reserved_deprecated(self):
P = self.cls
p = P('a/b')
with self.assertWarns(DeprecationWarning):
p.is_reserved()

def test_match_empty(self):
P = self.cls
self.assertRaises(ValueError, P('a').match, '')
Expand Down Expand Up @@ -414,13 +420,6 @@ def test_is_absolute(self):
self.assertTrue(P('//a').is_absolute())
self.assertTrue(P('//a/b').is_absolute())

def test_is_reserved(self):
P = self.cls
self.assertIs(False, P('').is_reserved())
self.assertIs(False, P('/').is_reserved())
self.assertIs(False, P('/foo/bar').is_reserved())
self.assertIs(False, P('/dev/con/PRN/NUL').is_reserved())

def test_join(self):
P = self.cls
p = P('//a')
Expand Down Expand Up @@ -1082,41 +1081,6 @@ def test_div(self):
self.assertEqual(p / P('./dd:s'), P('C:/a/b/dd:s'))
self.assertEqual(p / P('E:d:s'), P('E:d:s'))

def test_is_reserved(self):
P = self.cls
self.assertIs(False, P('').is_reserved())
self.assertIs(False, P('/').is_reserved())
self.assertIs(False, P('/foo/bar').is_reserved())
# UNC paths are never reserved.
self.assertIs(False, P('//my/share/nul/con/aux').is_reserved())
# Case-insensitive DOS-device names are reserved.
self.assertIs(True, P('nul').is_reserved())
self.assertIs(True, P('aux').is_reserved())
self.assertIs(True, P('prn').is_reserved())
self.assertIs(True, P('con').is_reserved())
self.assertIs(True, P('conin$').is_reserved())
self.assertIs(True, P('conout$').is_reserved())
# COM/LPT + 1-9 or + superscript 1-3 are reserved.
self.assertIs(True, P('COM1').is_reserved())
self.assertIs(True, P('LPT9').is_reserved())
self.assertIs(True, P('com\xb9').is_reserved())
self.assertIs(True, P('com\xb2').is_reserved())
self.assertIs(True, P('lpt\xb3').is_reserved())
# DOS-device name mataching ignores characters after a dot or
# a colon and also ignores trailing spaces.
self.assertIs(True, P('NUL.txt').is_reserved())
self.assertIs(True, P('PRN ').is_reserved())
self.assertIs(True, P('AUX .txt').is_reserved())
self.assertIs(True, P('COM1:bar').is_reserved())
self.assertIs(True, P('LPT9 :bar').is_reserved())
# DOS-device names are only matched at the beginning
# of a path component.
self.assertIs(False, P('bar.com9').is_reserved())
self.assertIs(False, P('bar.lpt9').is_reserved())
# Only the last path component matters.
self.assertIs(True, P('c:/baz/con/NUL').is_reserved())
self.assertIs(False, P('c:/NUL/con/baz').is_reserved())


class PurePathSubclassTest(PurePathTest):
class cls(pathlib.PurePath):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Add :func:`ntpath.isreserved`, which identifies reserved pathnames on
Windows; reserved names include "NUL", "AUX" and "CON".

Deprecate :meth:`pathlib.PurePath.is_reserved`.
Loading