Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-104372: Unlock the GIL before exec when using vfork(). #104515

Closed

Conversation

gpshead
Copy link
Member

@gpshead gpshead commented May 15, 2023

See code comments, this still needs work and likely should not be written this way.

Basically this solves the #104372 problem by unlocking the GIL before the child exec(). In the scariest way possible: it's a vforked child running in the parent vfork-calling thread address space, so the child "can" just do the unlock itself.

This proves the concept that it works when using a local test case involving an executable on a very slow filesystem.

But it doesn't feel clean to do it this way...

The proper approach would be a more traditional proper looking "all done in the parent" release the GIL before the vfork() call and reacquire in the parent afterwards code.

BUT we can't yet do that without some refactoring: child_exec() makes some Python C API calls that require the GIL to be held (!) the worst of them being in the existing make_inheritable() function which calls PyLong_AsLong() and PyErr_Occurred() (which asserts that the GIL is held).

We need to clean that code up anyways to have turned that tuple of ints into a C array of longs in the parent process before forking as while mostly async signal "safe" today, we shouldn't make that assumption as it is not guaranteed. After which I believe we'll be able to consider the more traditional drop GIL before vfork approach.

See comments, this still needs work and likely should not be written
this way.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant