Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(python): Selectors by_name and by_dtype should allow empty list as input #11024

Merged
merged 2 commits into from
Jan 15, 2024

Conversation

29antonioac
Copy link
Contributor

Closes #10855

Currently the selectors are being created with an unpacked list F.col(*all_names), but since F.col can manage iterables, by not unpacking we allow selecting nothing by inputing an empty list.

import polars as pl
import polars.selectors as cs

cols_to_process = [] # This could come from an input file

# This works now
df = pl.DataFrame({"a": [1,2,3]}) 
df_processed = df.with_columns(cs.by_name(cols_to_process) * 2)

Fixing this in my opinion gives more consistency on how some functions work: with_columns also works with empty lists for example.

@github-actions github-actions bot added fix Bug fix python Related to Python Polars rust Related to Rust Polars labels Sep 9, 2023
@alexander-beedie
Copy link
Collaborator

alexander-beedie commented Sep 9, 2023

Hmm... not quite sure about this; I can see why it would work for this case, but may need some other/related tweaks if we want this behaviour. I'll be able to review it properly tomorrow night though 👍

@29antonioac
Copy link
Contributor Author

Thanks! Happy to try to improve this if you think this could be useful!

@29antonioac 29antonioac changed the title fix: selectors by_name and by_dtype should allow empty list as input fix(python): selectors by_name and by_dtype should allow empty list as input Sep 14, 2023
@29antonioac
Copy link
Contributor Author

Hi, I just had another look at the code of the selectors and F.col and I can't see a reason why this could be a problem. Probably since I'm not used to this codebase I'm missing a lot of stuff so feel free to correct me 😄

Looks like the empty list is fully covered in F.col so I think it shouldn't be an issue. It looks every case is controlled in that function.

This could increase the consistency across other selectors too like datetime() (it uses a list to create the selector instead of an unpacked iterable), and consistency with the use of different ones as numeric() (if there's no numeric columns in the dataframe, it returns nothing). Any thoughts?

Thanks a lot!

@alexander-beedie
Copy link
Collaborator

alexander-beedie commented Sep 27, 2023

Apologies, got swamped by a few things; will get this sorted out today tomorrow 👌

Copy link
Member

@stinodego stinodego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into this, and it's a good change 👍 there's no need to unpack the lists here. Thanks!

@stinodego stinodego merged commit 42111c4 into pola-rs:main Jan 15, 2024
13 checks passed
@stinodego stinodego changed the title fix(python): selectors by_name and by_dtype should allow empty list as input fix(python): Selectors by_name and by_dtype should allow empty list as input Jan 15, 2024
r-brink pushed a commit to r-brink/polars that referenced this pull request Jan 15, 2024
…s input (pola-rs#11024)

Co-authored-by: Antonio Caballero <antonio.caballero@concirrus.com>
r-brink pushed a commit to r-brink/polars that referenced this pull request Jan 24, 2024
…s input (pola-rs#11024)

Co-authored-by: Antonio Caballero <antonio.caballero@concirrus.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix Bug fix python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Selectors by_name and by_dtype don't work with empty input
3 participants