feat(python): Warn on inefficient use of map_elements for temporal attributes/methods

[alexander-beedie]

Ref: #9968.
(Implements the ".dt.month and other stdlib datetime functions" suggestion).

Saw somebody do this in our own codebase today (shame on that person 🤣), so have added new bytecode detection logic for temporal attrs/methods, along with the appropriate native expression mapping.

Covers the following temporal attributes...

date
day
hour
microsecond
minute
month
second
year

...and temporal methods:

isoweekday()
date()
time()

Before

import polars as pl

df = pl.DataFrame({"dtm": [datetime(2024,2,16,10,30,45)]})
df.with_columns(
    # shameful (and wildly inefficient) use of a lambda :)
    time = pl.col("dtm").map_elements(lambda d: d.time())
)
# shape: (1, 2)
# ┌─────────────────────┬──────────┐
# │ dtm                 ┆ time     │
# │ ---                 ┆ ---      │
# │ datetime[μs]        ┆ time     │
# ╞═════════════════════╪══════════╡
# │ 2024-02-16 10:30:45 ┆ 10:30:45 │
# └─────────────────────┴──────────┘

After

Can still do it, but now triggers a PolarsInefficientMapWarning:

# PolarsInefficientMapWarning: 
# Expr.map_elements is significantly slower than the native expressions API.
# Only use if you absolutely CANNOT implement your logic otherwise.
# Replace this expression...
#   - pl.col("dtm").map_elements(lambda d: ...)
# with this one instead:
#   + pl.col("dtm").dt.time()

Also

Now optimises out unnecessary implicit existence checks from the final suggestion, eg:

lambda x: x and x and x and x.date()
...becomes:
pl.col("d").dt.date()

[alexander-beedie]

Looks like we should better-handle the form lambda x: x and x.time(); will get on it ;)

Update: done 👌

[MarcoGorelli]

thanks for doing this

looks like 3.12 is failing?

[alexander-beedie]

looks like 3.12 is failing?

Indeed; and yet... it isn't failing locally here on an M2 running on 3.12, hmm. Will dig in at home later; got access to a Ubuntu build running on my NAS - hopefully it replicates!

Ah! Tweaked my env and was able to replicate locally; should be a quick fix after all 🤞

[nameexhaustion]

This is some nice stuff. I took this for a spin and and came up with a more refined evil 😄

df.with_columns(time=pl.col("dtm").map_elements(datetime.time))

[MarcoGorelli]

nice work!

just out of interest, why would someone write lambda x: x and x and x.date()?

[alexander-beedie]

just out of interest, why would someone write lambda x: x and x and x.date()?

They likely wouldn't, but x and x.date() was spotted in our unit tests, so it primarily handles that. I made it handle the general case on the off-chance that some compound statement may one day collapse to form more than one x and in a row. The test is written that way just to prove that it'll handle the edge-case.

[alexander-beedie]

df.with_columns(time=pl.col("dtm").map_elements(datetime.time))

Devious; I can tackle that in a follow-up ;)

[MarcoGorelli]

nice, always happy to do more "butt-kicking" if people use map_elements unnecessarily :)