Does arquero serialize apache arrow tables to do any/all operations?

[deanm0000]

Does arquero serialize arrow tables or is it able to do filters or other computations without serializing to js objects under the hood?

[jheer]

It depends. Arquero uses an internal column representation that is generally compatible with Apache Arrow columns, so Arquero can process many Arrow data types without data copying or serialization. However, in some cases Arquero will try to optimize access either by caching or (e.g., for nested data types like structs and arrays) directly extracting the Arrow data into JS types.

When creating new derived columns or tables (e.g., rollups) from Arrow data, Arquero will create new JS arrays, but can later map those values to Arrow data using toArrow or toArrowBuffer.