v7.0.0
Changes from v6.0.1:
-
Breaking: Change the default handler of Apache Arrow data from the
apache-arrow
JS reference implementation to the smaller, faster, and more flexible@uwdata/flechette
.This change improves performance and simplifies Arquero internals, as it obviates the need for internal custom Arrow data conversion steps and a variety of earlier performance and correctness workarounds. Arrow IPC binary decoding (in
fromArrow
) and building/encoding (intoArrow
andtoArrowIPC
) now use Flechette.If passing an already instantiated Arrow table to
fromArrow
, Arquero supports bothapache-arrow
and@uwdata/flechette
tables, including optimizations for groupby and filter operations over dictionary types. However, special extraction of decimals, dates, and nested types (lists, structs) is no longer provided by Arquero; instead we default to whatever the Arrow implementation provides. For Flechette, this includes customizable extraction options and efficient serialization to native JS array and object types. For Arrow-JS, decimals are not well-supported and lists are returned as Arrow Vector types and structs as slow Proxy row objects.If loading data directly from IPC bytes, Arquero results should be consistent. Moreover, as Flechette is small, it is bundled directly with Arquero, so (unlike before) no additional imports are needed to process Arrow data. However, when passing-in already-parsed
apache-arrow
tables, results will differ for some data types.
Full Changelog: v6.0.1...v7.0.0