Skip to content

Commit

Permalink
Backport patch to fix hash-based pyc's with imp module
Browse files Browse the repository at this point in the history
For background, see
python/cpython#8130

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
  • Loading branch information
phmccarty authored and djklimes committed Jul 9, 2018
1 parent 8ac43a8 commit ea026c5
Show file tree
Hide file tree
Showing 3 changed files with 55 additions and 2 deletions.
51 changes: 51 additions & 0 deletions fix-pyc-imp.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
From c16fead8678e080c076ccd57fcb7437698a6636f Mon Sep 17 00:00:00 2001
From: Benjamin Peterson <benjamin@python.org>
Date: Thu, 5 Jul 2018 22:27:48 -0700
Subject: [PATCH] Always return bytes from _HackedGetData.get_data().

Ensure the imp.load_source shim always returns bytes by reopening the file in
binary mode if needed. Hash-based pycs have to receive the source code in bytes.

It's tempting to change imp.get_suffixes() to always return 'rb' as a mode, but
that breaks some stdlib tests and likely 3rdparty code, too.

Closes bpo-34056.
---
Lib/imp.py | 11 ++++-------
.../next/Library/2018-07-05-22-45-46.bpo-34056.86isrU.rst | 3 +++
2 files changed, 7 insertions(+), 7 deletions(-)
create mode 100644 Misc/NEWS.d/next/Library/2018-07-05-22-45-46.bpo-34056.86isrU.rst

diff --git a/Lib/imp.py b/Lib/imp.py
index 866464b245b2..fb83e91810f7 100644
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -144,15 +144,12 @@ def get_data(self, path):
if self.file and path == self.path:
if not self.file.closed:
file = self.file
- else:
- self.file = file = open(self.path, 'r')
+ if 'b' not in file.mode:
+ file.close()
+ if self.file.closed:
+ self.file = file = open(self.path, 'rb')

with file:
- # Technically should be returning bytes, but
- # SourceLoader.get_code() just passed what is returned to
- # compile() which can handle str. And converting to bytes would
- # require figuring out the encoding to decode to and
- # tokenize.detect_encoding() only accepts bytes.
return file.read()
else:
return super().get_data(path)
diff --git a/Misc/NEWS.d/next/Library/2018-07-05-22-45-46.bpo-34056.86isrU.rst b/Misc/NEWS.d/next/Library/2018-07-05-22-45-46.bpo-34056.86isrU.rst
new file mode 100644
index 000000000000..edc0135efc60
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2018-07-05-22-45-46.bpo-34056.86isrU.rst
@@ -0,0 +1,3 @@
+Ensure the loader shim created by ``imp.load_module`` always returns bytes
+from its ``get_data()`` function. This fixes using ``imp.load_module`` with
+:pep:`552` hash-based pycs.
4 changes: 3 additions & 1 deletion python3.spec
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Name: python3
Version: 3.7.0
Release: 155
Release: 156
License: Python-2.0
Summary: The Python Programming Language
Url: http://www.python.org
Expand All @@ -14,6 +14,7 @@ Patch4: 0004-Add-avx2-and-avx512-support.patch
Patch5: 0005-Build-avx2-and-avx512-versions-of-the-math-library.patch
Patch6: 0001-Add-pybench-for-pgo-optimization.patch
Patch7: hashcompile.patch
Patch8: fix-pyc-imp.patch


BuildRequires: bzip2
Expand Down Expand Up @@ -101,6 +102,7 @@ The Python Programming Language.
%patch5 -p1
%patch6 -p1
# patch7 -p1
%patch8 -p1

pushd ..
cp -a Python-%{version} Python-avx2
Expand Down
2 changes: 1 addition & 1 deletion release
Original file line number Diff line number Diff line change
@@ -1 +1 @@
155
156

0 comments on commit ea026c5

Please sign in to comment.