depr(python): Rename map_dict to replace and change default behavior

[stinodego]

Closes #10755

This has come up many times (e.g. #10744, #12537, #12533, ...) and I've been putting off making a decision on this. I think this is the way to go.
See #10755 (comment) for justification.

Changes

• Rename map_dict to replace
• Change the default behavior to keep any values not mentioned in the mapping unchanged. To preserve existing behavior, pass default=None.

[ritchie46]

Nice, can we take this opportunity to sometimes decide to go to a when -> then -> otherwise?

[stinodego]

Nice, can we take this opportunity to sometimes decide to go to a when -> then -> otherwise?

When would it be preferable, though? I guess for a single value you'd generally want to use when/then/otherwise. But I'm not sure where the threshold is at which replace becomes faster.

I'd like to just re-implement this whole function on the Rust side regardless. We can pick that up in a separate PR and include performance improvements then?

[ritchie46]

For a single value indeed.

[alexander-beedie]

When would it be preferable, though? I guess for a single value you'd generally want to use when/then/otherwise. But I'm not sure where the threshold is at which replace becomes faster.

Essentially immediately... I was timing this at work the other day and map_dict is significantly faster than when/then as soon as you go beyond ONE value (when/then is ~20% faster in that case though). With two values map_dict is ~2x faster, and the difference just gets more pronounced from there 😅 (which makes sense: O(1) vs O(n))

With a release build from current HEAD:

from codetiming import Timer
import polars as pl

df = pl.DataFrame({"x": range(10_000_000)})

with Timer("when/then"):
    res = df.with_columns(
        y = pl.when(pl.col("x") == 3).then("!!").when(pl.col("x") == 9_999_998).then("##")
    )
    # Elapsed time: 0.1122 seconds

with Timer("map_dict"):
    res = df.with_columns(
        y = pl.col("x").map_dict({3:"!!", 9_999_998:"##"})
    )
    # Elapsed time: 0.0581 seconds

shape: (10_000_000, 2)
┌─────────┬──────┐
│ x       ┆ y    │
│ ---     ┆ ---  │
│ i64     ┆ str  │
╞═════════╪══════╡
│ 0       ┆ null │
│ 1       ┆ null │
│ 2       ┆ null │
│ 3       ┆ !!   │
│ …       ┆ …    │
│ 9999996 ┆ null │
│ 9999997 ┆ null │
│ 9999998 ┆ ##   │
│ 9999999 ┆ null │
└─────────┴──────┘

We'd probably get more mileage from an optimisation that goes the other way, detecting patterns like this in when/then and converting to map_dict (or, as it will soon be known, replace) under the hood.

[ritchie46]

Yeap, that's what I expect. A single value to when then otherwise, the rest via map_dict.