Polars dataframe / arrow table

[aduarterengifo]

Feature Request

Hi, I was wondering how much work it would be to add support for polars.DataFrame, I saw a pull request for to_arrow() that seemed to have stalled.

[timkpaine]

In python and JS we already support instantiation from arrow (see python tests here). We also support pulling in data in python from numpy or pandas. For export in python, you can go to arrow or numpy / pandas via numpy.

I think it would be fine to support importing/exporting to/from flat polars dataframes. We wouldn't want to introduce an extra required dependency, and it may be difficult to deconstruct a pivoted polars dataframe (as it is for pandas). It also might be straightforward to just import/export polars from the existing numpy/arrow APIs without having any first party support.

[aduarterengifo]

sorry for the duplicate.

this is the same as #1157.

I want to do this

arrow_table = polars_df.to_arrow()
perspective_table = Table(arrow_table)

instead of

arrow_table = polars_df.to_arrow()
stream = pa.BufferOutputStream()
writer = pa.RecordBatchStreamWriter(stream, arrow_table.schema)
writer.write_table(arrow_table)
writer.close()
perspective_table = Table(stream.getvalue().to_pybytes())