Polars 1.3.0 fails to collect scanned Parquet containing struct columns

[danielgafni]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
from hypothesis import given, settings
from polars.testing.parametric import dataframes


# this test fails 

@given(df=dataframes(min_size=5))
@settings(max_examples=100, deadline=None)
def test_polars_parquet_write_read_with_structs(
    df: pl.DataFrame, tmp_path_factory
):
    path = tmp_path_factory.mktemp("test1") / "df.parquet"

    df.write_parquet(path)
    pl.read_parquet(path)  # this works

    ldf = pl.scan_parquet(path)
    ldf.collect()  # this fails!


# this test passes because it's not running over `pl.Struct` types

@given(df=dataframes(excluded_dtypes=[pl.Struct], min_size=5))
@settings(max_examples=100, deadline=None)
def test_polars_parquet_write_read_without_structs(
    df: pl.DataFrame, tmp_path_factory
):
    path = tmp_path_factory.mktemp("test2") / "df.parquet"

    df.write_parquet(path)
    pl.read_parquet(path)  # this works

    ldf = pl.scan_parquet(path)
    ldf.collect()  # this passes!

Log output

>       return wrap_df(ldf.collect(callback))
E       pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: ComputeError(ErrString("validity mask length must match the number of values"))

Issue description

Might be related:

• Polars 1.3.0 throws parquet errors when reading (I think) categorical columns in settings that 1.1.0 does not. #17931
• OOM on a trivial LazyFrame.head() #17932

Expected behavior

Writing and reading a Parquet file should always work

Installed versions

--------Version info---------
Polars:               1.3.0
Index type:           UInt32
Platform:             Linux-6.7.6-arch1-1-x86_64-with-glibc2.39
Python:               3.10.12 (main, Jul 26 2023, 13:14:21) [Clang 16.0.3 ]

----Optional dependencies----
adbc_driver_manager:  <not installed>
cloudpickle:          <not installed>
connectorx:           <not installed>
deltalake:            <not installed>
fastexcel:            <not installed>
fsspec:               <not installed>
gevent:               <not installed>
great_tables:         <not installed>
hvplot:               <not installed>
matplotlib:           <not installed>
nest_asyncio:         <not installed>
numpy:                2.0.1
openpyxl:             <not installed>
pandas:               2.2.2
pyarrow:              <not installed>
pydantic:             <not installed>
pyiceberg:            <not installed>
sqlalchemy:           <not installed>
torch:                <not installed>
xlsx2csv:             <not installed>
xlsxwriter:           <not installed>

[danielgafni]

oh, I found the issue: it doesn't work with pl.Struct anymore. Excluding this type from dataframes strategy fixes this test.