Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rust, python): avoid panic error in strftime with invalid format #6810

Merged
merged 8 commits into from
Feb 12, 2023

Conversation

MarcoGorelli
Copy link
Collaborator

@MarcoGorelli MarcoGorelli commented Feb 11, 2023

On the latest release it looks like this:

In [32]: pl.Series(["2020-01-01"]).str.strptime(pl.Datetime).dt.strftime('%z')
thread '<unnamed>' panicked at 'a formatting trait implementation returned an error: Error', library/alloc/src/fmt.rs:612:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---------------------------------------------------------------------------
PanicException                            Traceback (most recent call last)
Cell In [32], line 1
----> 1 pl.Series(["2020-01-01"]).str.strptime(pl.Datetime).dt.strftime('%z')

File ~/tmp/.venv/lib/python3.8/site-packages/polars/internals/series/utils.py:98, in call_expr.<locals>.wrapper(self, *args, **kwargs)
     96     expr = getattr(expr, namespace)
     97 f = getattr(expr, func.__name__)
---> 98 return s.to_frame().select(f(*args, **kwargs)).to_series()

File ~/tmp/.venv/lib/python3.8/site-packages/polars/internals/dataframe/frame.py:5732, in DataFrame.select(self, exprs, *more_exprs, **named_exprs)
   5616 def select(
   5617     self,
   5618     exprs: (
   (...)
   5627     **named_exprs: str | PolarsExprType | PythonLiteral | pli.Series | None,
   5628 ) -> Self:
   5629     """
   5630     Select columns from this DataFrame.
   5631
   (...)
   5729
   5730     """
   5731     return self._from_pydf(
-> 5732         self.lazy()
   5733         .select(exprs, *more_exprs, **named_exprs)
   5734         .collect(no_optimization=True)
   5735         ._df
   5736     )

File ~/tmp/.venv/lib/python3.8/site-packages/polars/internals/lazyframe/frame.py:1145, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, no_optimization, slice_pushdown, common_subplan_elimination, streaming)
   1134     common_subplan_elimination = False
   1136 ldf = self._ldf.optimization_toggle(
   1137     type_coercion,
   1138     predicate_pushdown,
   (...)
   1143     streaming,
   1144 )
-> 1145 return pli.wrap_df(ldf.collect())

PanicException: a formatting trait implementation returned an error: Error

Here:

In [1]: pl.Series(["2020-01-01"]).str.strptime(pl.Datetime).dt.strftime('%z')
thread '<unnamed>' panicked at 'a formatting trait implementation returned an error: Error', library/alloc/src/fmt.rs:612:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---------------------------------------------------------------------------
ComputeError                              Traceback (most recent call last)
Cell In[1], line 1
----> 1 pl.Series(["2020-01-01"]).str.strptime(pl.Datetime).dt.strftime('%z')

File ~/polars-dev/py-polars/polars/internals/series/utils.py:98, in call_expr.<locals>.wrapper(self, *args, **kwargs)
     96     expr = getattr(expr, namespace)
     97 f = getattr(expr, func.__name__)
---> 98 return s.to_frame().select(f(*args, **kwargs)).to_series()

File ~/polars-dev/py-polars/polars/internals/dataframe/frame.py:5766, in DataFrame.select(self, exprs, *more_exprs, **named_exprs)
   5650 def select(
   5651     self,
   5652     exprs: (
   (...)
   5661     **named_exprs: str | PolarsExprType | PythonLiteral | pli.Series | None,
   5662 ) -> Self:
   5663     """
   5664     Select columns from this DataFrame.
   5665 
   (...)
   5763 
   5764     """
   5765     return self._from_pydf(
-> 5766         self.lazy()
   5767         .select(exprs, *more_exprs, **named_exprs)
   5768         .collect(no_optimization=True)
   5769         ._df
   5770     )

File ~/polars-dev/py-polars/polars/internals/lazyframe/frame.py:1145, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, no_optimization, slice_pushdown, common_subplan_elimination, streaming)
   1134     common_subplan_elimination = False
   1136 ldf = self._ldf.optimization_toggle(
   1137     type_coercion,
   1138     predicate_pushdown,
   (...)
   1143     streaming,
   1144 )
-> 1145 return pli.wrap_df(ldf.collect())

ComputeError: Cannot format NaiveDateTime with format '%z'.

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels Feb 11, 2023
@MarcoGorelli MarcoGorelli marked this pull request as draft February 11, 2023 14:42
@MarcoGorelli MarcoGorelli marked this pull request as ready for review February 11, 2023 14:56
"{}",
Utc.from_local_datetime(&dt).earliest().unwrap().format(fmt)
),
_ => format!("{}", dt.format(fmt)),
) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is cleaner to write something like this:

write!().map_err(|_| PolarsError::ComputeError(..))?;

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, that's better - thanks!

));
}
},
_ => match write!(fmted, "{}", dt.format(fmt)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above.

Copy link
Member

@ritchie46 ritchie46 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MarcoGorelli. I have a small clippy like comment. Other than than that great additon. Errors > panics!

@ritchie46 ritchie46 merged commit 194d9b8 into pola-rs:master Feb 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants