gh-117783: Immortalize objects that use deferred reference counting #118112
Conversation
…ting Deferred reference counting is not fully implemented yet. As a temporary measure, we immortalize objects that would use deferred reference counting to avoid multi-threaded scaling bottlenecks. This is only performed in the free-threaded build once the first non-main thread is started. Additionally, some tests, including refleak tests, suppress this behavior.
colesbury
added
the
🔨 test-with-refleak-buildbots
|
🤖 New build scheduled with the buildbot fleet by @colesbury for commit 0cab82c 🤖 If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again. |
bedevere-bot
removed
the
🔨 test-with-refleak-buildbots
colesbury
requested review from
rhettinger and
ericsnowcurrently
as code owners
Python/pylifecycle.c
Outdated
| // run much later than typical leading to attribute errors due to | ||
| // partially cleared modules. To avoid this, we copy the module dict | ||
| // if it was immortalized. | ||
| PyObject *copy = PyDict_Copy(mod->md_dict); |
There was a problem hiding this comment.
Do we need to clear the original dict here too? Otherwise it seems like the immortal dict will just hang around and keep everything alive anyway?
There was a problem hiding this comment.
My intention was to keep the fields alive, but maybe that's not the best way to address this.
Basically, test_concurrent_futures.test_interpreter_shutdown was failing because a weakref was cleared later than usual, after the module dictionary was partially cleared. I was concerned that the problem might not be limited to that test, but I don't really have evidence for that one way or the other.
I'll get rid of this swapping and make the weakref callback more robust.
Deferred reference counting is not fully implemented yet. As a temporary measure, we immortalize objects that would use deferred reference counting to avoid multi-threaded scaling bottlenecks.
This is only performed in the free-threaded build once the first non-main thread is started. Additionally, some tests, including refleak tests, suppress this behavior.