-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Raise an error when constructing a Series or DataFrame with mixed types (e.g. string + number) #11156
Comments
I think this is a very similar issue to this: #11009. We really should do a pass on the Python -> Polars parsing to make it more restrictive by default, instead of silently casting/nulling/truncating values. |
@stinodego thoughts on polars's behavior of auto-converting |
It should either raise or cast to string, not sure which. |
@stinodego I would be in favour of raising an error. |
I'm also in favor of raising an error. If the developers are in agreement, could you accept this issue? |
Closing in favor of #14427 |
Description
I recently found a bug in my own code where I constructed a DataFrame with a mix of integers and strings, and the integers got set to
null
. Here's a simple illustration:Three other options here are to 1) convert everything to
dtype=object
(pandas's solution, but highly inefficient), 2) automatically upcast everything to a string, and 3) raise an error. I'm a big fan of raising an error here and letting the user decide whether they want to convert the integers to strings, set them tonull
, or take some other action.One of the beautiful things about polars is that it makes it much harder to accidentally introduce missing values than pandas, where pretty much every operation does an implicit outer join! Avoiding implicit conversions to
null
during Series/DataFrame construction would further reduce the potential for missing value-related bugs.Edit: this also happens here:
pandas converts to string in this situation:
The text was updated successfully, but these errors were encountered: