min/max operations on list with empty and/or None elements

[FBruzzesi]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl

print(
    pl.DataFrame({"lst": [[], [1,2,3]]}).select(pl.col("lst").list.max()),
    pl.DataFrame({"lst": [[], [None, 1], [1,2,3]]}).select(pl.col("lst").list.max()),
    sep="\n"
)

Log output

shape: (2, 1)
┌──────────────────────┐
│ lst                  │
│ ---                  │
│ i64                  │
╞══════════════════════╡
│ -9223372036854775808 │
│ 3                    │
└──────────────────────┘
shape: (3, 1)
┌──────┐
│ lst  │
│ ---  │
│ i64  │
╞══════╡
│ null │
│ 1    │
│ 3    │
└──────┘

Issue description

Breaking down to two cases:

• If all list elements are either empty or populated with integer values, then min/max output the type precision
• The behavior is different if there is one list of integers, one empty and one with None's

Expected behavior

I would expect the first case to behave as the second, namely output null for empty lists

Installed versions

--------Version info---------
Polars:               0.20.5
Index type:           UInt32
Platform:             Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python:               3.10.11 (main, May 16 2023, 00:28:57) [GCC 11.2.0]

----Optional dependencies----
adbc_driver_manager:  <not installed>
cloudpickle:          2.2.1
connectorx:           <not installed>
deltalake:            <not installed>
fsspec:               2023.1.0
gevent:               23.9.0.post1
hvplot:               <not installed>
matplotlib:           3.7.1
numpy:                1.24.4
openpyxl:             <not installed>
pandas:               2.0.3
pyarrow:              12.0.1
pydantic:             2.4.0
pyiceberg:            <not installed>
pyxlsb:               <not installed>
sqlalchemy:           2.0.17
xlsx2csv:             <not installed>
xlsxwriter:           <not installed>

[Julian-J-S]

can reproduce. Interesting and weird bug 😮

pl.DataFrame(
    {"lst": [[1, 2, 3], []]},
).with_columns(max=pl.col("lst").list.max())
# shape: (2, 2)
# ┌───────────┬──────────────────────┐
# │ lst       ┆ max                  │
# │ ---       ┆ ---                  │
# │ list[i64] ┆ i64                  │
# ╞═══════════╪══════════════════════╡
# │ [1, 2, 3] ┆ 3                    │
# │ []        ┆ -9223372036854775808 │ <<<<<<<<<<<< whoooooopsi 😱 
# └───────────┴──────────────────────┘

pl.DataFrame(
    {"lst": [[], [None, 1], [1, 2, 3]]},
).with_columns(max=pl.col("lst").list.max())
# shape: (3, 2)
# ┌───────────┬──────┐
# │ lst       ┆ max  │
# │ ---       ┆ ---  │
# │ list[i64] ┆ i64  │
# ╞═══════════╪══════╡
# │ []        ┆ null │ <<<<<<<<<<<<<<<<<<<<<< same value but correct? 🤔 
# │ [null, 1] ┆ 1    │
# │ [1, 2, 3] ┆ 3    │
# └───────────┴──────┘

[taki-mekhalfa]

I can reproduce in python, but not in rust:

fn main() {
    let file = fs::File::open("list.parquet").unwrap();

    let df = ParquetReader::new(file).finish().unwrap();
    println!("{df}");
    // ==>
   shape: (2, 1)
    ┌───────────┐
    │ c         │
    │ ---       │
    │ list[i64] │
    ╞═══════════╡
    │ [1, 2, 3] │
    │ []        │
    └───────────┘
    // <==
    let df = df.lazy().select([col("c").list().min()]).collect().unwrap();
    println!("{df}");
    // ==>
    shape: (2, 1)
    ┌──────┐
    │ c    │
    │ ---  │
    │ i64  │
    ╞══════╡
    │ 1    │
    │ null │
    └──────┘
    // <==
}

[cmdlineluser]

There seems to be different codepaths if there are nulls:

polars/crates/polars-ops/src/chunked_array/list/min_max.rs

Lines 206 to 212 in 9381381

	if has_inner_nulls(ca) {
	return inner(ca);
	};
	match ca.inner_dtype() {
	dt if dt.is_numeric() => Ok(max_list_numerical(ca, &dt)),
	_ => inner(ca),

>>> pl.Series([[]], dtype=pl.List(pl.Int64)).list.max()
shape: (1,)
Series: '' [i64]
[
	-9223372036854775808
]

>>> pl.Series([[]], dtype=pl.List(pl.Float64)).list.max()
shape: (1,)
Series: '' [f64]
[
	-inf
]

[reswqa]

Thanks for reporting this, will take a look.