feat: add to_buffer and helper methods in layout builder
#2766
Conversation
There was a problem hiding this comment.
I can see how this will make it much more convenient to fill buffers from a C interface, which has to pass each buffer individually. This implementation puts the recursion in the LayoutBuilder classes themselves, rather than outside of them.
Is our C/C++ tidy formatter removing the curly brackets from single-statement if/for bodies? I'm surprised it's not enforcing them. According to https://stackoverflow.com/q/26111162/1623645, it's either named readability-braces-around-statements in clang-tidy or InsertBraces in clang-format.
There was a problem hiding this comment.
@ManasviGoyal - good progress, thanks!
Let's combine these helper functions to fill one map to avoid a potential miss-match between the map key and its value.
Here is how ChatGPT suggests the std::map handling :-)
please, write a Python code and a C function using ctypes that wraps C++ to pass std::map to python
Certainly! To wrap a C++ function using ctypes that passes a std::map to Python, you'll need to create a C interface that can be accessed from Python. Here's an example:
- C++ Code (
example.cpp):
#include <iostream>
#include <map>
extern "C" {
typedef std::map<int, int> IntMap;
IntMap* create_map() {
return new IntMap();
}
void insert_to_map(IntMap* map, int key, int value) {
(*map)[key] = value;
}
void print_map(IntMap* map) {
for (const auto& entry : *map) {
std::cout << entry.first << ": " << entry.second << std::endl;
}
}
void delete_map(IntMap* map) {
delete map;
}
}- C++ Compilation:
g++ -shared -o example.so -fPIC example.cppThis will create a shared library example.so that can be loaded by ctypes in Python.
- Python Code (
example.py):
import ctypes
# Load the shared library
example = ctypes.CDLL('./example.so')
# Define types for C functions
example.create_map.restype = ctypes.c_void_p
example.create_map.argtypes = []
example.insert_to_map.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_int]
example.print_map.argtypes = [ctypes.c_void_p]
example.delete_map.argtypes = [ctypes.c_void_p]
# Create a map
map_ptr = example.create_map()
# Insert some values
example.insert_to_map(map_ptr, 1, 10)
example.insert_to_map(map_ptr, 2, 20)
example.insert_to_map(map_ptr, 3, 30)
# Print the map
example.print_map(map_ptr)
# Clean up memory
example.delete_map(map_ptr)- Run the Python Code:
python example.pyThis code will create a std::map in C++, insert some values, print the map, and then clean up the memory.
Please make sure to adjust the compilation and library loading paths as needed. Additionally, this example assumes a Linux-like environment; if you're on a different OS, the compilation and library loading process may be slightly different.
I think the clang-tidy is reinforcing it :-) |
|
Not to be down on ChatGPT, but I prefer Manasvi's solution. Instead of creating an temporary object that gets filled and later has to be deleted, Manasvi's solution goes directly to the thing we ultimately want to do: get the buffers to the right depth of the LayoutBuilder tree so that they can be filled. I'll admit that there are performance benefits to making the temporary But if we go with the temporary # Create a map
map_ptr = example.create_map()
try:
# Insert some values
example.insert_to_map(map_ptr, 1, 10)
example.insert_to_map(map_ptr, 2, 20)
example.insert_to_map(map_ptr, 3, 30)
# There's no reason to ever print the map
finally:
# Clean up memory
example.delete_map(map_ptr)That (This is the kind of complexity I wanted to avoid by not having intermediate objects.) |
|
We have both a |
No, I don't think so |
I'm not worried about the performance, but rather a potential issues when a user loads multiple files, gets the sizes first for all files, then gets the names, etc. The results could be out of order. The helpers should be hidden. And, yes, I think the anonymous helpers would make more sense in an auto-generated part, or outside of the LayoutBuilder header. |
Yes, you are right. The anonymous helpers don't need to be in the |
It's a bit uneven. There are few places where it is enforced while in other places it's not. |
It's already the case that a single LayoutBuilder instance can't be used with multiple files. It's a stateful object that is created when a file is opened, goes through all the sequential steps of snapshot-generation when it's done reading, and then is deleted when the file is closed. That's true in both its C++ and its C interface—even in the C++ interface, the snapshot process involved multiple steps that have to go in order.
I would expect to find But Manasvi's solution of passing each buffer and name down through the LayoutBuilder tree is most easily expressed in the Awkward header-only code, so if you're going with that implementation, I agree with Manasvi's original idea that it should be here. I'm in favor of merging this PR if all of the curly brackets are put in. |
I have removed the helper functions for now and fixed the curly braces (there were more in the code). Maybe we can this PR open and discuss further about this in tomorrow. I do have a few questions regarding what should go the the Maybe we can have something like |
|
@jpivarski - I'm happy with the PR. Please, feel free to merge it anytime. Thanks! |
Added some methods to be able to the features of
LayoutBuilder.hinCwherestd::mapcan't be used.to_buffermethod for each builder to copy and concatenate the data of each builder buffer to user-defined pointer based on the given node name instead of passing a map.