Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-120974: Make asyncio swap_current_task safe in free-threaded build #122317

Merged

Conversation

colesbury
Copy link
Contributor

@colesbury colesbury commented Jul 26, 2024

Use a critical section around the modifications to current_tasks so that the swapping is atomic.

…ed build

Use a critical section around the modifications to `current_tasks`.
@colesbury
Copy link
Contributor Author

!buildbot nogil refleaks

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 846f6f8 🤖

The command will test the builders whose names match following regular expression: nogil refleaks

The builders matched are:

  • AMD64 Fedora Rawhide NoGIL refleaks PR
  • PPC64LE Fedora Rawhide NoGIL refleaks PR
  • aarch64 Fedora Rawhide NoGIL refleaks PR
  • AMD64 Ubuntu NoGIL Refleaks PR

Py_hash_t hash, PyObject *task)
{
PyObject *prev_task;
if (_PyDict_GetItemRef_KnownHash(current_tasks, loop, hash, &prev_task) < 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this also use a lock held variant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it makes much of a difference, but we can. I'll add _PyDict_GetItemRef_KnownHash_LockHeld.

The lock held variants are important for the SetItem calls so that we don't try to re-lock the dict. But not so much for the GetItem calls, because those have fast paths that do not lock the dict.

@kumaraditya303 kumaraditya303 merged commit b5e6fb3 into python:main Aug 2, 2024
37 checks passed
@miss-islington-app
Copy link

Thanks @colesbury for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Aug 2, 2024
…ed build (pythonGH-122317)

* pythongh-120974: Make asyncio `swap_current_task` safe in free-threaded build
(cherry picked from commit b5e6fb3)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@bedevere-app
Copy link

bedevere-app bot commented Aug 2, 2024

GH-122612 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Aug 2, 2024
kumaraditya303 pushed a commit that referenced this pull request Aug 2, 2024
…ded build (GH-122317) (#122612)

gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (GH-122317)

* gh-120974: Make asyncio `swap_current_task` safe in free-threaded build
(cherry picked from commit b5e6fb3)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.13 has failed when building commit c1efeb3.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#builders/1386/builds/461) and take a look at the build logs.
  4. Check if the failure is related to this commit (c1efeb3) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#builders/1386/builds/461

Failed tests:

  • test_httplib

Failed subtests:

  • testHTTPConnectionSourceAddress - test.test_httplib.SourceAddressTest.testHTTPConnectionSourceAddress

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/0CD06645-FC15-4514-B5DC-AEF3ED24C912/data/Containers/Bundle/Application/1AE2CD3C-EBB6-482B-B920-80647D6C686A/iOSTestbed.app/python/lib/python3.13/test/test_httplib.py", line 1778, in testHTTPConnectionSourceAddress
    self.conn.connect()
    ~~~~~~~~~~~~~~~~~^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/0CD06645-FC15-4514-B5DC-AEF3ED24C912/data/Containers/Bundle/Application/1AE2CD3C-EBB6-482B-B920-80647D6C686A/iOSTestbed.app/python/lib/python3.13/http/client.py", line 1001, in connect
    self.sock = self._create_connection(
                ~~~~~~~~~~~~~~~~~~~~~~~^
        (self.host,self.port), self.timeout, self.source_address)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/0CD06645-FC15-4514-B5DC-AEF3ED24C912/data/Containers/Bundle/Application/1AE2CD3C-EBB6-482B-B920-80647D6C686A/iOSTestbed.app/python/lib/python3.13/socket.py", line 864, in create_connection
    raise exceptions[0]
  File "/Users/buildbot/Library/Developer/XCTestDevices/0CD06645-FC15-4514-B5DC-AEF3ED24C912/data/Containers/Bundle/Application/1AE2CD3C-EBB6-482B-B920-80647D6C686A/iOSTestbed.app/python/lib/python3.13/socket.py", line 848, in create_connection
    sock.bind(source_address)
    ~~~~~~~~~^^^^^^^^^^^^^^^^
OSError: [Errno 48] Address already in use

brandtbucher pushed a commit to brandtbucher/cpython that referenced this pull request Aug 7, 2024
…ed build (python#122317)

* pythongh-120974: Make asyncio `swap_current_task` safe in free-threaded build
blhsing pushed a commit to blhsing/cpython that referenced this pull request Aug 22, 2024
…ed build (python#122317)

* pythongh-120974: Make asyncio `swap_current_task` safe in free-threaded build
@colesbury colesbury deleted the gh-120974-asyncio-swap-current-task branch October 30, 2024 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants