Sort in Aggregation causes incorrect results

[syntonym]

Polars version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of Polars.

Issue description

Using a sort expression in an aggregation leads to random and incorrect results.

This sounds similar to this issue, but as far as I see the fix is not yet published on pypi. Also the fix is titled fix[rust]: unset sorted flag on mutation, which intuitively seems to not be the issue I've run in here, as in my case the dataframe to be aggregated is not sorted.

Reproducible example

df = pl.DataFrame({"A": ["a", "a", "a", "b", "b", "a"], "B": [1, 2, 3, 4, 5, 6]})
df.groupby("A").agg(pl.col("B").list().sort(reverse=True))

This results sometimes in

A	B
a	[5, 3, 2, 1]
b	[6, 4]

and sometimes in

A	B
b	[3, 2]
a	[6, 5, 4, 1]

Expected behavior

The result should be

A	B
b	[5, 4]
a	[6, 3, 2, 1]

Installed versions

---Version info---
Polars: 0.14.22
Index type: UInt32
Platform: Linux-5.19.12-arch1-1-x86_64-with-glibc2.36
Python: 3.10.7 (main, Sep  6 2022, 21:22:27) [GCC 12.2.0]
---Optional dependencies---
pyarrow: 6.0.1
pandas: 1.4.1
numpy: 1.22.4
fsspec: 2022.5.0
connectorx: <not installed>
xlsx2csv: <not installed>
matplotlib: 3.5.1

[syntonym]

As pointed out on discord by ghuls, the line df.groupby("A").agg(pl.col("B").sort(reverse=True)) produces the correct result.

[ghuls]

There might still be an issue with the list() operator anyway. Also depending on the optimizations it produces different results.

In [53]: df = pl.DataFrame({"A": ["a", "a", "a", "b", "b", "a"], "B": [1, 2, 3, 4, 5, 6]})

In [54]: df
Out[54]: 
shape: (6, 2)
┌─────┬─────┐
│ A   ┆ B   │
│ --- ┆ --- │
│ str ┆ i64 │
╞═════╪═════╡
│ a   ┆ 1   │
├╌╌╌╌╌┼╌╌╌╌╌┤
│ a   ┆ 2   │
├╌╌╌╌╌┼╌╌╌╌╌┤
│ a   ┆ 3   │
├╌╌╌╌╌┼╌╌╌╌╌┤
│ b   ┆ 4   │
├╌╌╌╌╌┼╌╌╌╌╌┤
│ b   ┆ 5   │
├╌╌╌╌╌┼╌╌╌╌╌┤
│ a   ┆ 6   │
└─────┴─────┘

In [55]: df.lazy().groupby("A").agg([pl.col("B").alias("b"), pl.col("B").sum().alias("b_sum"), pl.col("B").min().alias("b_min"), pl.col("B").max().alias("b_max"), pl.col("B").list().alias("b_list"), pl.col("B").list().sum().alias("b_list_sum"), pl.col("B
    ...: ").list().min().alias("b_list_min"), pl.col("B").list().max().alias("b_list_max"),]).collect()
Out[55]: 
shape: (2, 9)
┌─────┬───────────────┬───────┬───────┬───────┬───────────────┬────────────┬────────────┬────────────┐
│ A   ┆ b             ┆ b_sum ┆ b_min ┆ b_max ┆ b_list        ┆ b_list_sum ┆ b_list_min ┆ b_list_max │
│ --- ┆ ---           ┆ ---   ┆ ---   ┆ ---   ┆ ---           ┆ ---        ┆ ---        ┆ ---        │
│ str ┆ list[i64]     ┆ i64   ┆ i64   ┆ i64   ┆ list[i64]     ┆ i64        ┆ i64        ┆ i64        │
╞═════╪═══════════════╪═══════╪═══════╪═══════╪═══════════════╪════════════╪════════════╪════════════╡
│ b   ┆ [4, 5]        ┆ 9     ┆ 4     ┆ 5     ┆ [4, 5]        ┆ 5          ┆ 2          ┆ 3          │
├╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┤
│ a   ┆ [1, 2, ... 6] ┆ 12    ┆ 1     ┆ 6     ┆ [1, 2, ... 6] ┆ 16         ┆ 1          ┆ 6          │
└─────┴───────────────┴───────┴───────┴───────┴───────────────┴────────────┴────────────┴────────────┘

In [56]: df.lazy().groupby("A").agg([pl.col("B").alias("b"), pl.col("B").sum().alias("b_sum"), pl.col("B").min().alias("b_min"), pl.col("B").max().alias("b_max"), pl.col("B").list().alias("b_list"), pl.col("B").list().sum().alias("b_list_sum"), pl.col("B
    ...: ").list().min().alias("b_list_min"), pl.col("B").list().max().alias("b_list_max"),]).collect(type_coercion=False, predicate_pushdown=False, projection_pushdown=False, no_optimization=True, slice_pushdown=False, common_subplan_elimination=False)
Out[56]: 
shape: (2, 9)
┌─────┬───────────────┬───────┬───────┬───────┬───────────────┬────────────┬────────────┬────────────┐
│ A   ┆ b             ┆ b_sum ┆ b_min ┆ b_max ┆ b_list        ┆ b_list_sum ┆ b_list_min ┆ b_list_max │
│ --- ┆ ---           ┆ ---   ┆ ---   ┆ ---   ┆ ---           ┆ ---        ┆ ---        ┆ ---        │
│ str ┆ list[i64]     ┆ i64   ┆ i64   ┆ i64   ┆ list[i64]     ┆ i64        ┆ i64        ┆ i64        │
╞═════╪═══════════════╪═══════╪═══════╪═══════╪═══════════════╪════════════╪════════════╪════════════╡
│ a   ┆ [1, 2, ... 6] ┆ 12    ┆ 1     ┆ 6     ┆ [1, 2, ... 6] ┆ 11         ┆ 1          ┆ 5          │
├╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┤
│ b   ┆ [4, 5]        ┆ 9     ┆ 4     ┆ 5     ┆ [4, 5]        ┆ 10         ┆ 4          ┆ 6          │
└─────┴───────────────┴───────┴───────┴───────┴───────────────┴────────────┴────────────┴────────────┘