-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
python 3.13 support #442
python 3.13 support #442
Conversation
NumPy 2.1.0 added support for Python 3.13, but drops support for Python 3.9. See https://numpy.org/news/#numpy-210-released
…om the public API. We have to re-export it.
…ion to private API
…e function signature of the `_PyLong_AsByteArray` API See python/cpython#111140 and capi-workgroup/decisions#31
`_Py_IsFinalizing` becomes a stable API in Python 3.13, and is renamed to `Py_IsFinalizing` https://docs.python.org/3.13/c-api/init.html#c.Py_IsFinalizing
Since Python 3.13, `_PyDictView_New` function became an internal API.
Python 3.13 moved this undocumented function to private API. See python/cpython#108607
This reverts commit dc4442d.
This reverts commit d6e4cff.
…Keywords` instead
…Keywords` instead
…hread_state->asyncio_running_loop`
…o a dict Python 3.13 dramatically changed how the namespace in `exec`/`eval` works. See https://docs.python.org/3.13/whatsnew/3.13.html#defined-mutation-semantics-for-locals
56aed76
to
46909e3
Compare
46909e3
to
623d83e
Compare
`_PyErr_SetKeyError` is more complex than originally thought, use the provided API when possible See also python/cpython#101578
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed that only on python 3.13, the eval-test.simple
test says
"Exception ignored in: <_io.BufferedReader name='/home/runner/work/PythonMonkey/PythonMonkey/tests/js/resources/eval-test.js'>"
I looked into it briefly, looks to be something to do with the file being closed more than once somehow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks great, just some minor changes
python/pythonmonkey/require.py
Outdated
# Python 3.13 dramatically changed how the namespace in `exec`/`eval` works | ||
# See https://docs.python.org/3.13/whatsnew/3.13.html#defined-mutation-semantics-for-locals | ||
_locals = {} # keep the local variables inside `eval`/`exec` to a dict | ||
globalThis.python.eval = lambda x: eval(x, None, _locals) | ||
globalThis.python.exec = lambda x: exec(x, None, _locals) | ||
globalThis.python.getenv = os.getenv |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# Python 3.13 dramatically changed how the namespace in `exec`/`eval` works | |
# See https://docs.python.org/3.13/whatsnew/3.13.html#defined-mutation-semantics-for-locals | |
_locals = {} # keep the local variables inside `eval`/`exec` to a dict | |
globalThis.python.eval = lambda x: eval(x, None, _locals) | |
globalThis.python.exec = lambda x: exec(x, None, _locals) | |
globalThis.python.getenv = os.getenv | |
# Python 3.13 dramatically changed how the namespace in `exec`/`eval` works | |
# See https://docs.python.org/3.13/whatsnew/3.13.html#defined-mutation-semantics-for-locals | |
globalThis.python.eval = lambda x: eval(x, None, sys._getframe(1).f_locals) | |
globalThis.python.exec = lambda x: exec(x, None, sys._getframe(1).f_locals) |
Without the above change, pythonmonkey would lose the ability to mutate parent scope variables with pm.eval. On main the following assert passes, but it currently fails on this branch. We should add a test to ensure this behaviour as well.
import pythonmonkey as pm
a = 42
pm.eval("python.exec('a = 43')")
assert a == 43
While on the subject, the following currently fails on main, but probably shouldn't. The above change fixes that as well.
import pythonmonkey as pm
def foo():
b = 42
pm.eval("python.exec('b = 43')")
assert b == 43
foo()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://docs.python.org/3/library/functions.html#exec
In all cases, if the optional parts are omitted, the code is executed in the current scope. If only globals is provided, it must be a dictionary (and not a subclass of dictionary), which will be used for both the global and the local variables. If globals and locals are given, they are used for the global and local variables, respectively. If provided, locals can be any mapping object. Remember that at the module level, globals and locals are the same dictionary.
https://github.com/python/cpython/blob/v3.9.20/Python/bltinmodule.c#L979-L985
if (globals == Py_None) {
globals = PyEval_GetGlobals();
if (locals == Py_None) {
locals = PyEval_GetLocals();
if (locals == NULL)
return NULL;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On main
In your first example, a
is defined in the global scope, exec
can catch it and re-assign it without any problem.
However, in your second example, b
is defined in the local scope of the foo()
function. Since the exec
is executed inside of require.py
, it can only see the variables defined in require.py
as its "locals".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Without the above change, pythonmonkey would lose the ability to mutate parent scope variables with pm.eval. "
Question: is there a problem with losing that ability?
From a clean design POV, it's probably best to not let the languages mutate each other's scopes, except when those scoping objects are deliberately made available to this, i.e. in the case of globalThis
.
The fact that Python modules have a scoping semantic not understand in JS-land underscores this from my POV.
If we made python.eval
work more like indirect eval in JS, it would probably be a saner coding environment for our users, even though it might be a breaking change. And since the only code we really care about at this early stage is our test cases and BiFrost2, we control everything, so we can fix API and code in lock-step if necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i.e. to clarify: if we can change PythonMonkey to work with 3.13, and make it work the same in less than 3.13, I think that's probably okay. If a JS developer REALLY needs to mutate a Python scope, that dev can just pass in the scoping object, right?
…llOneArg` in all Python versions we support, so no need to do a shim on our own This commit reverts `bcfcbf35ce5ae10bb3542052cdccadf48e95cdca`
…llObject(func, NULL)`, which is available in all Python versions
…nction signatures
…exec`/`eval` Co-authored-by: Caleb Aikens <caleb@distributive.network>
Python 3.13 was finally released on October 7, 2024. GitHub Actions' `actions/setup-python` already added support for the final release version. See https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
Something is double-free-ed during the final finalization: in a debug build of Python, `./Include/object.h:602: _Py_NegativeRefcount: Assertion failed: object has negative ref count` In non-debug build of Python, it simply segfaults at the end during finalization.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
In order to make PythonMonkey support the upcoming (October 1, 2024) Python 3.13 release, the following changes/actions need to be made in the codebase:
Python 3.13 added a
PyLong_AsNativeBytes API
, but also changed the function signature of the_PyLong_AsByteArray
API( C API: Consider adding public PyLong_AsByteArray() and PyLong_FromByteArray() functions python/cpython#111140)
_Py_IsFinalizing
becomes a stable API in Python 3.13, and is renamed toPy_IsFinalizing
Python 3.13 moved several undocumented APIs to private and internal-only, i.e. removed from public use.
_PyDictView*
APIs are removed._PyArg_CheckPositional
API is removed._PyErr_SetKeyError
API is removed. gh-106320: Remove private _PyErr_SetKeyError() python/cpython#108607_PyThreadState_GetDict(tstate)
API gets removed.Closes #441