Add user data context to base callbacks #1105
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
This commit adds support for passing user data through the callback system. When registering a callback you can optionally pass an arbitrary
void*
which can then be used by your callback, as it will always be passed as the first argument to your callback. This allows storing callback state in a manner which doesn't require global state.Also included, primarily for backwards-compatibility purposes, is a set of trampolines which allows the old APIs (which (a) have incompatible callback function types and (b) don't have users pass a context variable) to use the new callback system. The way this works is when an old style of callback is registered, the trampoline is installed as the callback, then the user's callback is passed as the context variable. Then when the trampoline is called it calls the user callback by pulling it from the context variable.
Motivating Usecase
In pypanda it is typical to use functions which capture variables in the surrounding scope as callbacks in order to access the variables in the callback itself. This reduces (and often removes) the need for excessive global variables in order to keep track of callback state. The equivalent feature in Rust is closures, which are local functions which can capture state, however using closures for callbacks isn't possible as callbacks provide neither a way to pass user data nor any other means of identifying the callback. Being able to pass user data to callbacks would enable a closure-based callback API in Rust.
The reason pypanda can have this behavior without a context variable is due to the fact that pypanda uses
cffi
to generate trampolines to callbacks, enabling the ability to uniquely identify instances of variable capturing by function pointer. Such an approach would be quite undesirable in the context of Rust due to this style of JIT not being idiomatic for compiled languages, as well as being harder to maintain due to reduced runtime flexibility limiting what possible implementations would look like.Potential Future Use
While this API was primarily designed with Rust in mind, this API could also be used in C plugins to simplify logic which otherwise needs to be encoded as an ad-hoc global state machine. It could also make it easier to alleviate the need for callback trampoline JIT if pypanda is ever moved to an FFI library which is no longer pycparser-based, which could allow for less fragile python bindings.