feat(rust, python): support timezone in csv writer

[MarcoGorelli]

closes #6718

[MarcoGorelli]

I tried to, instead do

Suggested change

	dtype-datetime = ["polars-core/dtype-datetime", "polars-core/temporal", "polars-time/dtype-datetime", "chrono-tz", "chrono"]
	dtype-datetime = ["polars-core/dtype-datetime", "polars-core/temporal", "polars-time/dtype-datetime"]
	timezones = ["chrono-tz", "chrono"]

and then, in write_impl.rs, applying the following:

diff --git a/polars/polars-io/src/csv/write_impl.rs b/polars/polars-io/src/csv/write_impl.rs
index d2aaab6b5..f855dc97c 100644
--- a/polars/polars-io/src/csv/write_impl.rs
+++ b/polars/polars-io/src/csv/write_impl.rs
@@ -1,9 +1,9 @@
 use std::io::Write;
 
 use arrow::temporal_conversions;
-#[cfg(feature = "dtype-datetime")]
+#[cfg(feature = "timezones")]
 use chrono::TimeZone;
-#[cfg(feature = "dtype-datetime")]
+#[cfg(feature = "timezones")]
 use chrono_tz::Tz;
 use lexical_core::{FormattedSize, ToLexical};
 use memchr::{memchr, memchr2};
@@ -95,6 +95,7 @@ fn write_anyvalue(f: &mut Vec<u8>, value: AnyValue, options: &SerializeOptions)
                 TimeUnit::Milliseconds => temporal_conversions::timestamp_ms_to_datetime(v),
             };
             match tz {
+                #[cfg(feature = "timezones")]
                 Some(tz) => match tz.parse::<Tz>() {
                     Ok(parsed_tz) => match &options.datetime_format {
                         None => write!(f, "{}", PlTzAware::new(ndt, tz)),

However, I then get

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: Uncategorized, message: "formatter error" }', /home/marcogorelli/polars-dev/polars/polars-io/src/csv/write_impl.rs:132:6
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "/home/marcogorelli/polars-dev/py-polars/t.py", line 19, in <module>
    print(df.write_csv())
  File "/home/marcogorelli/polars-dev/py-polars/polars/internals/dataframe/frame.py", line 2237, in write_csv
    self._df.write_csv(
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: Error { kind: Uncategorized, message: "formatter error" }

Looking into it, but am a bit confused

[ritchie46]

How does the panic relate with the dependencies?

[MarcoGorelli]

I don't know, that's what I'm confused about - there's no panic currently, but it appears if I move chrono-tz and chrono behind the timezones feature

[ritchie46]

I can take a look later before merging.

[MarcoGorelli]

thanks! 🙌 I'll take another look too, perhaps I can figure it out

[MarcoGorelli]

OK, I see what happens (kinda): if I put the Some arm under the timezones feature, then it'll be skipped, and the other arm will be hit

            let formatted = match tz {
                #[cfg(feature = "timezones")]
                Some(tz) => match tz.parse::<Tz>() {
                    Ok(parsed_tz) => parsed_tz.from_utc_datetime(&ndt).format(datetime_format),
                    Err(_) => match temporal_conversions::parse_offset(tz) {
                        Ok(parsed_tz) => parsed_tz.from_utc_datetime(&ndt).format(datetime_format),
                        Err(_) => unreachable!(),
                    },
                },
                _ => ndt.format(datetime_format),
            };

And then, if date_format contains '%z', then ndt.format(datetime_format) fails

So, I need to get the timezones feature activated when match tz is reached with a tz-aware input

[MarcoGorelli]

Hi @alexander-beedie - do you have any thoughts on this? On the one hand, parsing tz for each element slows things down for tz-aware columns, and could be done beforehand in a per-column fashion

On the other hand, in #4724 it looks like you intentionally tried to avoid per-column inference

[alexander-beedie]

Yup - though it wasn't about not wanting per-column inference (which would actually be great) as ensuring that any such inference was done outside the hot loop (per-element inference would be bad).

I took a minimal approach in the end as my initial attempt to reshuffle things on a per-column basis became unnecessarily convoluted - treat my earlier commit as a mere first step towards a more flexible/better per-column future... ;)

[ritchie46]

@MarcoGorelli let me know when I can finish this!

[MarcoGorelli]

Sure, feel free to take over, thanks!

[MarcoGorelli]

ah looks like this is what I was missing - thanks for fixing it up!

looks like this needs a rebase?

[ritchie46]

Nope, a squash will do. 👍