Multiple enumeration while serializing

[raminrahimzada]

Library Version

4.25.0

OS

Windows

OS Architecture

64 bit

How to reproduce?

Hi, I have the following example code to reproduce:

public sealed record class MyClass(int Id, Guid Hash);

static IEnumerable<MyClass> YieldAll()
{
    Console.WriteLine("I am here");
    yield return new MyClass(1, Guid.NewGuid());
    yield return new MyClass(2, Guid.NewGuid());
}
static async Task Main()
{
    const string path = "my_file.parquet";
    await ParquetSerializer.SerializeAsync(YieldAll(), path);
}

And the output was:

I am here
I am here

I think it should not enumerate my items twice and it is an issue because in my case I am connecting to the sql table containing millions of rows, so serializing fetches them twice

Failing test

No response

[aloneguid]

This is by design. Each property will incur full collection enumeration. It's a consequence of parquet being columnar format, so to fill one column we need to enumerate it.