gh-121103: Put free-threaded libraries in `lib/python3.14t` #121293

colesbury · 2024-07-02T20:22:16Z

On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation.

Issue: Free-threaded libraries should be in lib/python3.14t (configure) #121103

On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation.

colesbury · 2024-07-02T20:43:48Z

pip, uv, venv, setuptools, seem to work, but I don't entirely know what I'm doing, so I'd appreciate a thorough review.

This also fixes the test_sysconfig failure on Windows.

hroncok · 2024-07-03T09:39:43Z

Does this mean that all of the pure Python standard library files would need to be installed in lib/python3.14t
as well? I.e. when I have both the "normal" and the freethreaded Python installed, do I need all the standard library modules twice?

encukou · 2024-07-03T12:32:12Z

I don't have very deep knowledge in this area, so I can't easily do a thorough review :(
I'd trust @FFY00 and @hroncok's reviews if they can contribute their time for it.

erlend-aasland · 2024-07-03T12:41:05Z

Does this mean that all of the pure Python standard library files would need to be installed in lib/python3.14t as well? I.e. when I have both the "normal" and the freethreaded Python installed, do I need all the standard library modules twice?

As far as I can tell, from a quick look at this PR: yes.

hroncok · 2024-07-03T13:01:19Z

As far as I can tell, from a quick look at this PR: yes.

I thought so.

The real question is: Is this the outcome we anticipated?

If not, I suggest keeping the stdlib and platstdlib values intact.

colesbury · 2024-07-03T14:41:32Z

Yes, that was the intention. The goal is to separate the installation. We run into a lot of potential problems if we have partially overlapping stdlib files.

erlend-aasland · 2024-07-03T16:26:16Z

Yes, that was the intention. The goal is to separate the installation. We run into a lot of potential problems if we have partially overlapping stdlib files.

FTR, I totally agree with this.

colesbury · 2024-07-03T16:31:23Z

To expand on my previous comment: if we have kept a common stdlib and platstdlib, the Python files would be shared, but the .so files would not. So I'd be concerned that if someone installed, e.g. 3.13.0 default and 3.13.8 free-threaded to the same prefix, then the .py files and native .so libraries might not match. Keeping things separate in lib/python3.13t vs. lib/python3.13 avoids this problem.

colesbury · 2024-07-10T15:05:42Z

I plan to merge this tomorrow if there are no objections

ned-deily

Other than the nit comment, +1. This looks like it does what it promises. We still have a bit more to do before a free-threaded build can be installed to the same prefix as a non-free-threaded build without conflicts, primarily due to duplicate symlinked names in the bin and lib/pkgconfig directories. But this isn't really a new problem as there are similar conflicts when sharing with a debug build prior to this PR. I'd say let's merge this PR now and we can follow up in another PR for which I want to do some more investigation.

Makefile.pre.in

miss-islington-app · 2024-07-11T20:21:40Z

Thanks @colesbury for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

…thonGH-121293) On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation. (cherry picked from commit e8c91d9) Co-authored-by: Sam Gross <colesbury@gmail.com>

bedevere-app · 2024-07-11T20:21:52Z

GH-121631 is a backport of this pull request to the 3.13 branch.

…H-121293) (#121631) On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation. (cherry picked from commit e8c91d9) Co-authored-by: Sam Gross <colesbury@gmail.com>

…thon#121293) On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation.

AlexWaygood · 2024-08-06T11:16:03Z

Lib/site.py

+        if hasattr(sys, 'abiflags') and 't' in sys.abiflags:
+            abi_thread = 't'
+        else:
+            abi_thread = ''
        if os.sep == '/':
            libdirs = [sys.platlibdir]
            if sys.platlibdir != "lib":
                libdirs.append("lib")

            for libdir in libdirs:
                path = os.path.join(prefix, libdir,
-                                    f"{implementation}{ver[0]}.{ver[1]}",
+                                    f"{implementation}{ver[0]}.{ver[1]}{abi_thread}",


@colesbury -- would you be able to update the docs for the site module to reflect these changes to how Python finds the site-packages directory?

(The context is I'm currently working on reimplementing Python's logic here for Astral's work-in-progress type checker. I was reading through the site.py source code, and it surprised me to find logic here that wasn't documented!)

Thanks for the pointer. I put up a PR to update the docs: #122737

pythongh-121103: Put free-threaded libraries in lib/python3.14t

03772aa

On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation.

colesbury added topic-free-threading needs backport to 3.13 bugs and security fixes labels Jul 2, 2024

bedevere-app bot mentioned this pull request Jul 2, 2024

Free-threaded libraries should be in lib/python3.14t (configure) #121103

Closed

Fix blurb

ed2dde7

colesbury marked this pull request as ready for review July 2, 2024 20:42

colesbury requested review from FFY00, vsajip, erlend-aasland and corona10 as code owners July 2, 2024 20:42

bedevere-app bot added the awaiting core review label Jul 2, 2024

colesbury requested review from encukou, Yhg1s and ned-deily July 2, 2024 20:42

Make ABI logic in sysconfig a bit more consistent with everything else.

6b46646

This also fixes the test_sysconfig failure on Windows.

corona10 approved these changes Jul 3, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Jul 3, 2024

ned-deily approved these changes Jul 11, 2024

View reviewed changes

Makefile.pre.in Outdated Show resolved Hide resolved

colesbury added 2 commits July 11, 2024 14:22

Use tab in Makefile.pre.in

d72fb70

Merge branch 'main' into pythongh-121103-libdir

432701c

erlend-aasland approved these changes Jul 11, 2024

View reviewed changes

colesbury merged commit e8c91d9 into python:main Jul 11, 2024
36 checks passed

bedevere-app bot removed the awaiting merge label Jul 11, 2024

colesbury deleted the gh-121103-libdir branch July 11, 2024 20:21

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Jul 11, 2024

colesbury mentioned this pull request Jul 13, 2024

Having two identical executables for free-threaded version is misleading pyenv/pyenv#3015

Open

10 tasks

AlexWaygood reviewed Aug 6, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-121103: Put free-threaded libraries in `lib/python3.14t` #121293

gh-121103: Put free-threaded libraries in `lib/python3.14t` #121293

colesbury commented Jul 2, 2024 •

edited by bedevere-app bot

Loading

colesbury commented Jul 2, 2024

hroncok commented Jul 3, 2024 •

edited

Loading

encukou commented Jul 3, 2024

erlend-aasland commented Jul 3, 2024

hroncok commented Jul 3, 2024 •

edited

Loading

colesbury commented Jul 3, 2024

erlend-aasland commented Jul 3, 2024

colesbury commented Jul 3, 2024

colesbury commented Jul 10, 2024

ned-deily left a comment

miss-islington-app bot commented Jul 11, 2024

bedevere-app bot commented Jul 11, 2024

AlexWaygood Aug 6, 2024

colesbury Aug 6, 2024

gh-121103: Put free-threaded libraries in lib/python3.14t #121293

gh-121103: Put free-threaded libraries in lib/python3.14t #121293

Conversation

colesbury commented Jul 2, 2024 • edited by bedevere-app bot Loading

colesbury commented Jul 2, 2024

hroncok commented Jul 3, 2024 • edited Loading

encukou commented Jul 3, 2024

erlend-aasland commented Jul 3, 2024

hroncok commented Jul 3, 2024 • edited Loading

colesbury commented Jul 3, 2024

erlend-aasland commented Jul 3, 2024

colesbury commented Jul 3, 2024

colesbury commented Jul 10, 2024

ned-deily left a comment

Choose a reason for hiding this comment

miss-islington-app bot commented Jul 11, 2024

bedevere-app bot commented Jul 11, 2024

AlexWaygood Aug 6, 2024

Choose a reason for hiding this comment

colesbury Aug 6, 2024

Choose a reason for hiding this comment

gh-121103: Put free-threaded libraries in `lib/python3.14t` #121293

gh-121103: Put free-threaded libraries in `lib/python3.14t` #121293

colesbury commented Jul 2, 2024 •

edited by bedevere-app bot

Loading

hroncok commented Jul 3, 2024 •

edited

Loading

hroncok commented Jul 3, 2024 •

edited

Loading