gh-104372: Unlock the GIL before exec when using vfork(). #104515
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See code comments, this still needs work and likely should not be written this way.
Basically this solves the #104372 problem by unlocking the GIL before the child exec(). In the scariest way possible: it's a vforked child running in the parent vfork-calling thread address space, so the child "can" just do the unlock itself.
This proves the concept that it works when using a local test case involving an executable on a very slow filesystem.
But it doesn't feel clean to do it this way...
The proper approach would be a more traditional proper looking "all done in the parent" release the GIL before the vfork() call and reacquire in the parent afterwards code.
BUT we can't yet do that without some refactoring: child_exec() makes some Python C API calls that require the GIL to be held (!) the worst of them being in the existing
make_inheritable()
function which calls PyLong_AsLong() and PyErr_Occurred() (which asserts that the GIL is held).We need to clean that code up anyways to have turned that tuple of ints into a C array of longs in the parent process before forking as while mostly async signal "safe" today, we shouldn't make that assumption as it is not guaranteed. After which I believe we'll be able to consider the more traditional drop GIL before vfork approach.
vfork()
until the child processexec()
succeeds or fails. #104372