Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add user data context to base callbacks #1105

Merged
merged 5 commits into from
Oct 19, 2021
Merged

Add user data context to base callbacks #1105

merged 5 commits into from
Oct 19, 2021

Conversation

jamcleod
Copy link
Member

@jamcleod jamcleod commented Oct 15, 2021

Changes

This commit adds support for passing user data through the callback system. When registering a callback you can optionally pass an arbitrary void* which can then be used by your callback, as it will always be passed as the first argument to your callback. This allows storing callback state in a manner which doesn't require global state.

Also included, primarily for backwards-compatibility purposes, is a set of trampolines which allows the old APIs (which (a) have incompatible callback function types and (b) don't have users pass a context variable) to use the new callback system. The way this works is when an old style of callback is registered, the trampoline is installed as the callback, then the user's callback is passed as the context variable. Then when the trampoline is called it calls the user callback by pulling it from the context variable.

Motivating Usecase

In pypanda it is typical to use functions which capture variables in the surrounding scope as callbacks in order to access the variables in the callback itself. This reduces (and often removes) the need for excessive global variables in order to keep track of callback state. The equivalent feature in Rust is closures, which are local functions which can capture state, however using closures for callbacks isn't possible as callbacks provide neither a way to pass user data nor any other means of identifying the callback. Being able to pass user data to callbacks would enable a closure-based callback API in Rust.

The reason pypanda can have this behavior without a context variable is due to the fact that pypanda uses cffi to generate trampolines to callbacks, enabling the ability to uniquely identify instances of variable capturing by function pointer. Such an approach would be quite undesirable in the context of Rust due to this style of JIT not being idiomatic for compiled languages, as well as being harder to maintain due to reduced runtime flexibility limiting what possible implementations would look like.

Potential Future Use

While this API was primarily designed with Rust in mind, this API could also be used in C plugins to simplify logic which otherwise needs to be encoded as an ad-hoc global state machine. It could also make it easier to alleviate the need for callback trampoline JIT if pypanda is ever moved to an FFI library which is no longer pycparser-based, which could allow for less fragile python bindings.

Copy link
Member

@AndrewFasano AndrewFasano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants