Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use nil_values instead of null_character in IO operations #667

Merged
merged 3 commits into from
Jul 28, 2023

Conversation

cnpryer
Copy link
Contributor

@cnpryer cnpryer commented Jul 27, 2023

Closes #666

Updates null_character defaults used in csv reading operations to use nil instead of "NA". See #667 (comment)

Updates usage of null_character to null_values allowing multiple missing value interpretation in IO operations. This also changes the default from "NA" to nil.

Result:

iex(8)> Explorer.DataFrame.load_csv("name,abbreviation\nNamibia,NA")
{:ok,
 #Explorer.DataFrame<
   Polars[1 x 2]
   name string ["Namibia"]
   abbreviation string ["NA"]
 >}

Copy link
Member

@philss philss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense to me 👍

I added some suggestions.

lib/explorer/data_frame.ex Outdated Show resolved Hide resolved
notebooks/exploring_explorer.livemd Outdated Show resolved Hide resolved
native/explorer/src/dataframe/io.rs Outdated Show resolved Hide resolved
@cnpryer cnpryer changed the title Use nil for null_character default in IO operations Use nil_values instead of null_character in IO operations Jul 27, 2023
@cnpryer
Copy link
Contributor Author

cnpryer commented Jul 27, 2023

I didn't use "nil" in Rust since I figured it's Elixir-specific (I'm very new to Elixir). I can change that as well if you'd like.

@josevalim josevalim merged commit dab4941 into elixir-explorer:main Jul 28, 2023
@josevalim
Copy link
Member

💚 💙 💜 💛 ❤️

@cnpryer cnpryer deleted the null-character-default branch July 28, 2023 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DataFrame.from_csv incorrectly reads "NA" as nil
3 participants