Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use deferred reference counting in some _PyInterpreterFrame fields #123923

Closed
colesbury opened this issue Sep 10, 2024 · 0 comments
Closed

Use deferred reference counting in some _PyInterpreterFrame fields #123923

colesbury opened this issue Sep 10, 2024 · 0 comments
Labels
topic-free-threading type-feature A feature request or enhancement

Comments

@colesbury
Copy link
Contributor

colesbury commented Sep 10, 2024

Feature or enhancement

The _PyInterpreterFrame struct contains a strong reference to:

  • f_executable - the currently executing code object
  • f_funcobj - the function object
  • f_locals - the locals object (often NULL)
  • frame_obj - the frame object (often NULL)

We should use deferred references (in the free-threaded build) for f_executable and f_funcobj because they are a common source of reference count contention. The pointed to objects (code and function) already support deferred reference counting, but the references in the frame are not deferred.

I think we don't need to bother with changing f_locals for now. I don't think it's used frequently enough to be a bottleneck, but we can revisit it later if necessary.

The frame_obj are typically unique -- not shared across threads -- so we don't need to bother with deferred reference counting for frame_obj.

Complications and hazards

_PyInterpreterFrame are also embedded in generators/coroutines and PyFrameObject, which are heap objects. We need to be careful that _PyStackRef fields are visible to the GC in order to be kept alive. Once an object is untracked, the _PyStackRef fields may no longer be valid: it's safe to call PyStackRef_CLOSE/CLEAR but not otherwise access or dereference those fields because the GC may have already collected them.

Linked PRs

@colesbury colesbury added type-feature A feature request or enhancement topic-free-threading labels Sep 10, 2024
colesbury added a commit to colesbury/cpython that referenced this issue Sep 10, 2024
…terFrame`

Use a `_PyStackRef` and defer the reference to `f_executable` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
colesbury added a commit to colesbury/cpython that referenced this issue Sep 10, 2024
…terFrame`

Use a `_PyStackRef` and defer the reference to `f_executable` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
colesbury added a commit that referenced this issue Sep 12, 2024
…me` (#123924)

Use a `_PyStackRef` and defer the reference to `f_executable` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
colesbury added a commit to colesbury/cpython that referenced this issue Sep 12, 2024
…Frame`

Use a `_PyStackRef` and defer the reference to `f_funcobj` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
colesbury added a commit to colesbury/cpython that referenced this issue Sep 12, 2024
…Frame`

Use a `_PyStackRef` and defer the reference to `f_funcobj` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
colesbury added a commit to colesbury/cpython that referenced this issue Sep 14, 2024
colesbury added a commit to colesbury/cpython that referenced this issue Sep 20, 2024
colesbury added a commit to colesbury/cpython that referenced this issue Sep 24, 2024
colesbury added a commit that referenced this issue Sep 24, 2024
…#124026)

Use a `_PyStackRef` and defer the reference to `f_funcobj` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-free-threading type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

1 participant