Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a feature to allow creating Buffers from Vec with custom allocators (allocator_api) #3960

Open
jhorstmann opened this issue Mar 27, 2023 · 0 comments
Labels
enhancement Any new improvement worthy of a entry in the changelog

Comments

@jhorstmann
Copy link
Contributor

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

in #3920 and #3917 support was added to to create buffers from standard Rust vectors. The currently instable allocator_api feature extends Vec to support custom allocators, using functions such as new_in. In our product we are using such custom allocators to track the memory usage of individual queries. I'd like to add a similarly named feature to arrow-rs which would generalize the Buffer::from_vec and MutableBuffer::from_vec functions.

Describe the solution you'd like

  • Add a feature named allocator-api to arrow-buffer
  • If that feature is active, a different impl of from_vec would be enabled (via cfg attribute), which has a generic parameter for the allocator
  • Deallocation::Standard would additionally store the allocator (note the default Global allocator is zero-sized)
  • The feature would require a nightly rust compiler, just as the feature of the standard library

Describe alternatives you've considered

Something similar can be achieved using Buffer::from_custom_allocation but requires unsafe and dealing with pointers.

Additional context

@jhorstmann jhorstmann added the enhancement Any new improvement worthy of a entry in the changelog label Mar 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

No branches or pull requests

1 participant