Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(rust): More refactor for PlSmallStr #18456

Merged
merged 5 commits into from
Aug 30, 2024

Conversation

nameexhaustion
Copy link
Collaborator

@nameexhaustion nameexhaustion commented Aug 29, 2024

  • Organize the impls
  • Some remaining cleanup of .as_ref().into() -> .clone()
  • Removes From<&&str>: *breaks a lot of things - will do separately
    • Existing code calling with constant arrays are changed from e.g. Series::new(_, &["a", ...]) to Series::new(_, ["a", ...])
    • &[&str] can no longer be passed to most API functions accepting impl IntoIterator<Item = impl Into<PlSmallStr>>
  • Add AsRef<Path>, AsRef<[u8]>, AsRef<OsStr>

@github-actions github-actions bot added internal An internal refactor or improvement rust Related to Rust Polars labels Aug 29, 2024
}
}

impl core::fmt::Display for PlSmallStr {
#[inline(always)]
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.0.fmt(f)
self.as_str().fmt(f)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure we use the Display impl from &str

@nameexhaustion nameexhaustion changed the title refactor(rust): Add AsRef impls to PlSmallStr, and some cleanup refactor(rust): Fix Display for PlSmallStr, organize impls Aug 29, 2024
@nameexhaustion nameexhaustion marked this pull request as ready for review August 29, 2024 10:46
impl core::fmt::Debug for PlSmallStr {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.as_str().fmt(f)
impl From<&&str> for PlSmallStr {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When do we have double indirection in &&str?, I think we should just deref.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please don't add this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have some functions that take impl IntoIterator<Item = impl Into<PlSmallStr>> -

fn join<I, S>(
&self,
other: &DataFrame,
left_on: I,
right_on: I,
args: JoinArgs,
) -> PolarsResult<DataFrame>
where
I: IntoIterator<Item = S>,
S: Into<PlSmallStr>,

and they get passed &[&str] -

let df_inner_join = df_left
.join(
&df_right,
&["col1", "join_col2"],
&["join_col1", "col2"],
JoinType::Inner.into(),
)
.unwrap();

Whose IntoIterator then becomes &&str, so we needed the From<&&str>.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just pass ["col1", "join_col2"] instead of &["col1", "join_col2"].

Copy link
Collaborator Author

@nameexhaustion nameexhaustion Aug 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That still gives &&str for the IntoIter *nvm: this has changed since a while ago in Rust

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ritchie46 removing this causes a huge amount of breakage, I will do it in a later PR

Copy link

codecov bot commented Aug 29, 2024

Codecov Report

Attention: Patch coverage is 78.50467% with 23 lines in your changes missing coverage. Please review.

Project coverage is 79.83%. Comparing base (915f164) to head (96ac672).
Report is 32 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-utils/src/pl_str.rs 57.14% 12 Missing ⚠️
crates/polars-sql/src/context.rs 50.00% 4 Missing ⚠️
crates/polars-expr/src/expressions/column.rs 0.00% 1 Missing ⚠️
crates/polars-io/src/ipc/ipc_reader_async.rs 0.00% 1 Missing ⚠️
crates/polars-io/src/partition.rs 50.00% 1 Missing ⚠️
crates/polars-plan/src/plans/ir/tree_format.rs 50.00% 1 Missing ⚠️
...rates/polars-python/src/lazyframe/visitor/nodes.rs 0.00% 1 Missing ⚠️
crates/polars-sql/src/functions.rs 0.00% 1 Missing ⚠️
crates/polars-sql/src/sql_expr.rs 90.90% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #18456      +/-   ##
==========================================
- Coverage   79.84%   79.83%   -0.01%     
==========================================
  Files        1497     1498       +1     
  Lines      201828   201803      -25     
  Branches     2867     2867              
==========================================
- Hits       161141   161115      -26     
- Misses      40141    40142       +1     
  Partials      546      546              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nameexhaustion nameexhaustion marked this pull request as draft August 30, 2024 04:51
@nameexhaustion nameexhaustion changed the title refactor(rust): Fix Display for PlSmallStr, organize impls refactor(rust): More refactor for PlSmallStr Aug 30, 2024
&self,
other: &DataFrame,
left_on: impl IntoIterator<Item = impl Into<PlSmallStr>>,
right_on: impl IntoIterator<Item = impl Into<PlSmallStr>>,
Copy link
Collaborator Author

@nameexhaustion nameexhaustion Aug 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pre-work for removing From<&&str> - as then calling join(left_on=["a"], right_on=["a", "b"]) becomes 2 distinct generics <[&str; 1] as IntoIterator>, <[&str; 2] as IntoIterator>

*although also in general, allows for 2 distinct types

@nameexhaustion nameexhaustion marked this pull request as ready for review August 30, 2024 08:06
@ritchie46 ritchie46 merged commit 910fc03 into pola-rs:main Aug 30, 2024
22 checks passed
@c-peters c-peters added the accepted Ready for implementation label Sep 3, 2024
@nameexhaustion nameexhaustion deleted the pl-str branch September 4, 2024 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation internal An internal refactor or improvement rust Related to Rust Polars
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants