Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when writing to csv when DataFrame contains both naive and aware datetimes and no datetime_format is specified #6827

Closed
2 tasks done
MarcoGorelli opened this issue Feb 12, 2023 · 1 comment · Fixed by #6828
Labels
bug Something isn't working python Related to Python Polars

Comments

@MarcoGorelli
Copy link
Collaborator

MarcoGorelli commented Feb 12, 2023

Polars version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of Polars.

Issue description

I accidentally introduced in #6722 so sorry!

Reproducible example

In [3]: from datetime import datetime
   ...: import datetime as dt
   ...:
   ...: import polars as pl
   ...:
   ...: ts = pl.date_range(datetime(2000, 1, 1), datetime(2000, 1, 2))
   ...: df = pl.DataFrame({"naive": ts, "aware": ts.dt.replace_time_zone("UTC")})
   ...: print(df.write_csv())
   ...:
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Uncategorized, message: "formatter error" }', /home/runner/work/polars/polars/polars/polars-io/src/csv/write_impl.rs:128:6
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---------------------------------------------------------------------------
PanicException                            Traceback (most recent call last)
Cell In [3], line 9
      7 ts = pl.date_range(datetime(2000, 1, 1), datetime(2001, 1, 1))
      8 df = pl.DataFrame({"naive": ts, "aware": ts.dt.replace_time_zone("UTC")})
----> 9 print(df.write_csv())

File ~/tmp/.venv/lib/python3.8/site-packages/polars/internals/dataframe/frame.py:2295, in DataFrame.write_csv(self, file, has_header, sep, quote, batch_size, datetime_format, date_format, time_format, float_precision, null_value)
   2293 if file is None:
   2294     buffer = BytesIO()
-> 2295     self._df.write_csv(
   2296         buffer,
   2297         has_header,
   2298         ord(sep),
   2299         ord(quote),
   2300         batch_size,
   2301         datetime_format,
   2302         date_format,
   2303         time_format,
   2304         float_precision,
   2305         null_value,
   2306     )
   2307     return str(buffer.getvalue(), encoding="utf-8")
   2309 if isinstance(file, (str, Path)):

PanicException: called `Result::unwrap()` on an `Err` value: Error { kind: Uncategorized, message: "formatter error" }

Expected behavior

naive,aware
2000-01-01T00:00:00.000000,2000-01-01T00:00:00.000000+0000
2000-01-02T00:00:00.000000,2000-01-02T00:00:00.000000+0000

Installed versions

---Version info---
Polars: 0.16.4
Index type: UInt32
Platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python: 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
---Optional dependencies---
pyarrow: 11.0.0
pandas: 1.5.3
numpy: 1.24.1
fsspec: <not installed>
connectorx: <not installed>
xlsx2csv: 0.8.1
deltalake: <version not detected>
matplotlib: <not installed>
None

@MarcoGorelli MarcoGorelli added bug Something isn't working python Related to Python Polars labels Feb 12, 2023
@MarcoGorelli
Copy link
Collaborator Author

PR on its way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working python Related to Python Polars
Projects
None yet
1 participant