Injecting foreign stack frames

[encukou]

Language wrappers like JPype and Cython could use first-class support for injecting “foreign” stack frames/traceback entries.

See python/cpython#86784

[markshannon]

What does "injecting" mean?

Is the ability to add a frame to the top of the current stack sufficient, or do they need to inject frames into the middle of the stack?
Does the frame need to outlive the C code doing the injection?

[encukou]

(I haven't looked into this so don't know the answers myself.)

[davidhewitt]

Interaction between the native Rust stack and Python stack would be challenging to solve but net a decent win for developers.

I can think of at least exception tracebacks and profiling as two use cases where having Rust stack frames visible would be beneficial. I think it'd probably work with an API similar to Py_EnterRecursiveCall / Py_LeaveRecursiveCall (maybe just extended forms of those) which would append and pop frames.

I'm not sure what frame data would be desired other than a name.

[Thrameos]

There are two variants to this issue. The first is creating a frame on the stack in C so that the method name, file and line number appear when an exception is thrown. This can be used when enterring foreign code to indicate the location.

The second variation is one in which the user has an exception in another language with a backtrace in the form of a list of object that can be converted to method name, file and line numbers. Jpype has code for this case in which we convert the objects into a python exception that is the added as a cause. Currently this is very inefficient because python frames are concrete so we have to convert even if the user never traverses the list (as opposed to when the user accesses the list). I understand there would be a speed penality to lazy allocation so not much we can do here. But if you assume exceptions should be exceptional it is fine.