Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql_lab.py uses PyArrow API that was removed in pyarrow 12.0.0 #24030

Closed
wjones127 opened this issue May 11, 2023 · 2 comments · Fixed by #24135
Closed

sql_lab.py uses PyArrow API that was removed in pyarrow 12.0.0 #24030

wjones127 opened this issue May 11, 2023 · 2 comments · Fixed by #24135
Labels
#bug Bug report

Comments

@wjones127
Copy link
Member

This:

data = (
pa.default_serialization_context()
.serialize(result_set.pa_table)
.to_buffer()
.to_pybytes()
)

Should be replaced with

sink = pa.BufferOutputStream()
with pa.ipc.new_stream(sink, result_set.pa_table.schema) as writer:
    writer.write_table(result_set.pa_table)

data = sink.getvalue().to_pybytes()

Will likely need to update the deserialization code as well, where ever that is.

Issue reported on Arrow mailing list here: https://lists.apache.org/thread/93sj6hzgnzz3j0gt5sw0l8pxg6b546ho

@wjones127 wjones127 added the #bug Bug report label May 11, 2023
@wjones127
Copy link
Member Author

wjones127 commented May 11, 2023

@robdiciuccio it looks like you implemented this in #8069, so you might be in the best position to update this. (I haven't yet worked on superset myself.) I'm happy to provide more guidance on the use of PyArrow if needed, though.

@EugeneTorap
Copy link
Contributor

Hi @wjones127! Thanks for this issue!
@sebastianliebscher Can we prepare a PR for upgrading pyarrow to 12.0.0 and replace that sql lab code?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
#bug Bug report
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants