bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win)

Doc/library/shutil.rst

@@ -51,7 +51,9 @@ Directory and files operations
 .. function:: copyfile(src, dst, *, follow_symlinks=True)
 
    Copy the contents (no metadata) of the file named *src* to a file named
-   *dst* and return *dst*.  *src* and *dst* are path names given as strings.
+   *dst* and return *dst* in the most efficient way possible.
+   *src* and *dst* are path names given as strings.
+
    *dst* must be the complete target file name; look at :func:`shutil.copy`
    for a copy that accepts a target directory path.  If *src* and *dst*
    specify the same file, :exc:`SameFileError` is raised.
@@ -74,6 +76,10 @@ Directory and files operations
       Raise :exc:`SameFileError` instead of :exc:`Error`.  Since the former is
       a subclass of the latter, this change is backward compatible.
 
+   .. versionchanged:: 3.8
+      Platform-specific fast-copy syscalls are used internally in order to copy
+      the file more efficiently. See
+      :ref:`shutil-platform-dependent-efficient-copy-operations` section.
 
 .. exception:: SameFileError
 
@@ -163,6 +169,11 @@ Directory and files operations
       Added *follow_symlinks* argument.
       Now returns path to the newly created file.
 
+   .. versionchanged:: 3.8
+      Platform-specific fast-copy syscalls are used internally in order to copy
+      the file more efficiently. See
+      :ref:`shutil-platform-dependent-efficient-copy-operations` section.
+
 .. function:: copy2(src, dst, *, follow_symlinks=True)
 
    Identical to :func:`~shutil.copy` except that :func:`copy2`
@@ -185,6 +196,11 @@ Directory and files operations
       file system attributes too (currently Linux only).
       Now returns path to the newly created file.
 
+   .. versionchanged:: 3.8
+      Platform-specific fast-copy syscalls are used internally in order to copy
+      the file more efficiently. See
+      :ref:`shutil-platform-dependent-efficient-copy-operations` section.
+
 .. function:: ignore_patterns(\*patterns)
 
    This factory function creates a function that can be used as a callable for
@@ -241,6 +257,10 @@ Directory and files operations
       Added the *ignore_dangling_symlinks* argument to silent dangling symlinks
       errors when *symlinks* is false.
 
+   .. versionchanged:: 3.8
+      Platform-specific fast-copy syscalls are used internally in order to copy
+      the file more efficiently. See
+      :ref:`shutil-platform-dependent-efficient-copy-operations` section.
 
 .. function:: rmtree(path, ignore_errors=False, onerror=None)
 
@@ -314,6 +334,11 @@ Directory and files operations
    .. versionchanged:: 3.5
       Added the *copy_function* keyword argument.
 
+   .. versionchanged:: 3.8
+      Platform-specific fast-copy syscalls are used internally in order to copy
+      the file more efficiently. See
+      :ref:`shutil-platform-dependent-efficient-copy-operations` section.
+
 .. function:: disk_usage(path)
 
    Return disk usage statistics about the given path as a :term:`named tuple`
@@ -370,6 +395,29 @@ Directory and files operations
    operation. For :func:`copytree`, the exception argument is a list of 3-tuples
    (*srcname*, *dstname*, *exception*).
 
+.. _shutil-platform-dependent-efficient-copy-operations:
+
+Platform-dependent efficient copy operations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Starting from Python 3.8 all functions involving a file copy (:func:`copyfile`,
+:func:`copy`, :func:`copy2`, :func:`copytree`, and :func:`move`) use
+platform-specific "fast-copy" syscalls in order to copy the file more
+efficiently (see :issue:`33671`).
+"fast-copy" means that the copying operation occurs within the kernel, avoiding
+the use of userspace buffers in Python as in "``outfd.write(infd.read())``".
+
+On OSX `fcopyfile`_ is used to copy the file content (not metadata).
+On Linux, Solaris and other POSIX platforms
+where :func:`os.sendfile` supports copies between 2 regular file descriptors
+:func:`os.sendfile` is used.
+On Windows `CopyFile`_ is used by all copy functions except :func:`copyfile`.
+
+If the fast-copy operation fails and no data was written in the destination
+file then shutil will silently fallback on using less efficient
+:func:`copyfileobj` function internally.
+
+.. versionadded:: 3.8
 
 .. _shutil-copytree-example:
 
@@ -654,6 +702,11 @@ Querying the size of the output terminal
 
    .. versionadded:: 3.3
 
+.. _`CopyFile`:
+   https://msdn.microsoft.com/en-us/library/windows/desktop/aa363851(v=vs.85).aspx
+
+.. _`fcopyfile`:
+   http://www.manpagez.com/man/3/fcopyfile/
+
 .. _`Other Environment Variables`:
    http://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html#tag_002_003
-

Doc/whatsnew/3.8.rst

@@ -90,10 +90,20 @@ New Modules
 Improved Modules
 ================
 
-
 Optimizations
 =============
 
+* :func:`shutil.copyfile`, :func:`shutil.copy`, :func:`shutil.copy2`,
+  :func:`shutil.copytree` and :func:`shutil.move` use platform specific
+  "fast-copy" syscalls in order to copy the file more efficiently.
+  "fast-copy" means that the copying operation occurs within the kernel,
+  avoiding the use of userspace buffers in Python as in
+  "``outfd.write(infd.read())``".
+  The speedup for copying a 512MB file within the same partition is about +26%
+  on Linux, +50% on OSX and +48% on Windows. Also, much less CPU cycles are
+  consumed.
+  (Contributed by Giampaolo Rodola' in :issue:`25427`.)
+
 * The default protocol in the :mod:`pickle` module is now Protocol 4,
   first introduced in Python 3.4.  It offers better performance and smaller
   size compared to Protocol 3 available since Python 3.0.

Lib/shutil.py

@@ -42,6 +42,16 @@
 except ImportError:
     getgrnam = None
 
+posix = nt = None
+if os.name == 'posix':
+    import posix
+elif os.name == 'nt':
+    import nt
+    import _winapi
+
+_HAS_SENDFILE = posix and hasattr(os, "sendfile")
+_HAS_FCOPYFILE = posix and hasattr(posix, "_fcopyfile")
+
 __all__ = ["copyfileobj", "copyfile", "copymode", "copystat", "copy", "copy2",
            "copytree", "move", "rmtree", "Error", "SpecialFileError",
            "ExecError", "make_archive", "get_archive_formats",
@@ -72,6 +82,10 @@ class RegistryError(Exception):
     """Raised when a registry operation with the archiving
     and unpacking registries fails"""
 
+class _GiveupOnFastCopy(Exception):
+    """Raised as a signal to fallback on using raw read()/write()
+    file copy when fast-copy functions fail to do so.
+    """
 
 def copyfileobj(fsrc, fdst, length=16*1024):
     """copy data from file-like object fsrc to file-like object fdst"""
@@ -81,6 +95,104 @@ def copyfileobj(fsrc, fdst, length=16*1024):
             break
         fdst.write(buf)
 
+def _fastcopy_osx(fsrc, fdst):
+    """Copy 2 regular mmap-like files by using high-performance
+    fcopyfile() syscall (OSX only).
+    """
+    try:
+        infd = fsrc.fileno()
+        outfd = fdst.fileno()
+    except Exception as err:
+        raise _GiveupOnFastCopy(err)  # not a regular file
+
+    try:
+        posix._fcopyfile(infd, outfd)
+    except OSError as err:
+        if err.errno in {errno.EINVAL, errno.ENOTSUP}:
+            raise _GiveupOnFastCopy(err)
+        else:
+            raise err from None
+
+def _fastcopy_win(fsrc, fdst):
+    """Copy 2 files by using high-performance CopyFileW (Windows only)."""
+    _winapi.CopyFileExW(fsrc, fdst, 0)
+
+def _fastcopy_sendfile(fsrc, fdst):
+    """Copy data from one regular mmap-like fd to another by using
+    high-performance sendfile() method.
+    This should work on Linux >= 2.6.33 and Solaris only.
+    """
+    global _HAS_SENDFILE
+    try:
+        infd = fsrc.fileno()
+        outfd = fdst.fileno()
+    except Exception as err:
+        raise _GiveupOnFastCopy(err)  # not a regular file
+
+    # Hopefully the whole file will be copied in a single call.
+    # sendfile() is called in a loop 'till EOF is reached (0 return)
+    # so a bufsize smaller or bigger than the actual file size
+    # should not make any difference, also in case the file content
+    # changes while being copied.
+    try:
+        blocksize = max(os.fstat(infd).st_size, 2 ** 23)  # min 8MB
+    except Exception:
+        blocksize = 2 ** 27  # 128MB
+
+    offset = 0
+    while True:
+        try:
+            sent = os.sendfile(outfd, infd, offset, blocksize)
+        except OSError as err:
+            if err.errno == errno.ENOTSOCK:
+                # sendfile() on this platform (probably Linux < 2.6.33)
+                # does not support copies between regular files (only
+                # sockets).
+                _HAS_SENDFILE = False
+                raise _GiveupOnFastCopy(err)
+
+            if err.errno == errno.ENOSPC:  # filesystem is full
+                raise err from None
+
+            # Give up on first call and if no data was copied.
+            if offset == 0 and os.lseek(outfd, 0, os.SEEK_CUR) == 0:
+                raise _GiveupOnFastCopy(err)
+
+            raise err from None
+        else:
+            if sent == 0:
+                break  # EOF
+            offset += sent
+
+def _fastcopy_fileobj(fsrc, fdst):
+    """Copy 2 regular mmap-like fds by using zero-copy sendfile(2)
+    (Linux) and fcopyfile(2) (OSX) syscalls.
+    In case of error fallback on using plain read()/write() if no
+    data was copied.
+    """
+    # Note: copyfileobj() is left alone in order to not introduce any
+    # unexpected breakage. Possible risks by using zero-copy calls
+    # in copyfileobj() are:
+    # - fdst cannot be open in "a"(ppend) mode
+    # - fsrc and fdst may be open in "t"(ext) mode
+    # - fsrc may be a BufferedReader (which hides unread data in a buffer),
+    #   GzipFile (which decompresses data), HTTPResponse (which decodes
+    #   chunks).
+    # - possibly others
+    if _HAS_SENDFILE:
+        try:
+            return _fastcopy_sendfile(fsrc, fdst)
+        except _GiveupOnFastCopy:
+            pass
+
+    if _HAS_FCOPYFILE:
+        try:
+            return _fastcopy_osx(fsrc, fdst)
+        except _GiveupOnFastCopy:
+            pass
+
+    return copyfileobj(fsrc, fdst)
+
 def _samefile(src, dst):
     # Macintosh, Unix.
     if hasattr(os.path, 'samefile'):
@@ -117,9 +229,13 @@ def copyfile(src, dst, *, follow_symlinks=True):
     if not follow_symlinks and os.path.islink(src):
         os.symlink(os.readlink(src), dst)
     else:
+        if os.name == 'nt':
+            _fastcopy_win(src, dst)
+            return dst
+
         with open(src, 'rb') as fsrc:
             with open(dst, 'wb') as fdst:
-                copyfileobj(fsrc, fdst)
+                _fastcopy_fileobj(fsrc, fdst)
     return dst
 
 def copymode(src, dst, *, follow_symlinks=True):
@@ -1015,7 +1131,6 @@ def disk_usage(path):
 
 elif os.name == 'nt':
 
-    import nt
     __all__.append('disk_usage')
     _ntuple_diskusage = collections.namedtuple('usage', 'total used free')