categorical arrays fail with PanicException

[khusmann]

Polars version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of Polars.

Issue description

When building arrays of categorical vars, I get PanicExceptions if not all the levels are present in the first array.

pl.Series("foo", [["a"], ["a", "b"]]).arr.eval(pl.element().cast(pl.Categorical))

fails with:

thread '<unnamed>' panicked at 'assertion failed: i < self.len()', /root/.cargo/git/checkouts/arrow2-945af624853845da/baa2618/src/array/utf8/mod.rs:155:9

but this works just fine:

pl.Series("foo", [["a", "b"], ["a", "b"]]).arr.eval(pl.element().cast(pl.Categorical))

I tried using the global string cache, and it still fails:

with pl.StringCache():
   pl.Series("foo", [["a"], ["a", "b"]]).arr.eval(pl.element().cast(pl.Categorical))

but with a slightly different error:

thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /home/runner/work/polars/polars/polars/polars-core/src/chunked_array/logical/categorical/builder.rs:91:28

Reproducible example

pl.Series("foo", [["a"], ["a", "b"]]).arr.eval(pl.element().cast(pl.Categorical))

Expected behavior

As I mention in #6003, it'd be best to be able to directly cast to pl.List(pl.Categorical), without needing .arr.eval:

pl.Series("foo", [["a"], ["a", "b"]]).cast(pl.List(pl.Categorical))

But this just results in nulls:

shape: (2,)
Series: 'foo' [list]
[
	[null]
	[null, null]
]

Installed versions

---Version info---
Polars: 0.15.13
Index type: UInt32
Platform: Linux-5.15.0-53-generic-x86_64-with-glibc2.31
Python: 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:26:04) [GCC 10.4.0]
---Optional dependencies---
pyarrow: 10.0.1
pandas: 1.4.0
numpy: 1.23.4
fsspec: <not installed>
connectorx: <not installed>
xlsx2csv: <not installed>
matplotlib: 3.5.1

[ritchie46]

Thanks for the report. I shall ensure we support : pl.Series("foo", [["a"], ["a", "b"]]).cast(pl.List(pl.Categorical))

[khusmann]

Awesome, thanks! I also see you prevent categorical casts in arr.eval now -- nice touch.

[ritchie46]

Awesome, thanks! I also see you prevent categorical casts in arr.eval now -- nice touch.

Yes, that kind of dark arts can lead to all kinds of problems. Better stop it early. :)