You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #470, we made the ResultSet object lazy, which improves performance when running queries that fetch a lot of data.
However, there's still a bottleneck when converting DuckDB results into pandas dataframes. DuckDB offers a .df() method that can efficiently convert DuckDB's results into a pandas data frame. When the autopandas option is turned on, the .df() is used (see #469); however when autopandas is off, and users call:
%sql df << SELECT * FROM ...
df.DataFrame()
.DataFrame() won't use DuckDB's .df() method. We need to change that and ensure that .df() is always used whenever we are using DuckDB (as this is a DuckDB-specific feature). Once this is implemented, we can get rid of the logic introduced by #469 (here), and just call .DataFrame()
The text was updated successfully, but these errors were encountered:
In #470, we made the ResultSet object lazy, which improves performance when running queries that fetch a lot of data.
However, there's still a bottleneck when converting DuckDB results into pandas dataframes. DuckDB offers a
.df()
method that can efficiently convert DuckDB's results into a pandas data frame. When the autopandas option is turned on, the.df()
is used (see #469); however when autopandas is off, and users call:.DataFrame()
won't use DuckDB's.df()
method. We need to change that and ensure that.df()
is always used whenever we are using DuckDB (as this is a DuckDB-specific feature). Once this is implemented, we can get rid of the logic introduced by #469 (here), and just call.DataFrame()
The text was updated successfully, but these errors were encountered: