Extremely large memory consumption in "ToListAsync()" when using cancellation token

[Stocki916]

Dear community,

i faced a memory issue and a big difference in selecting data from a microsoft sql database when using an async "ToListAsync()" with and without a cancellation token.
Im selecting "ids" of type "bigint / long" from a table in the database by using this statement which returns 140 mio ids:

return await _context.Items.AsNoTracking().Select(s => s.Id.GetValueOrDefault(0)).Distinct().ToListAsync();

The memory consumption of the app is round about 2 GB of memory.

The largest object in memory monitor is the list with "Ids". That's clear so far :-)

If i modify the statement and pass a cancellation token to the "ToListAsync(ct)", then the memory consumption increases to > 15 GB

return await _context.Items.AsNoTracking().Select(s => s.Id.GetValueOrDefault(0)).Distinct().ToListAsync(ct);

Now there are really large "CancellationTokenSource" and "CallBacknode" objects, which cause the high memory usage:

In general i could just remove the cancellation token but i would like to understand how i can still use a cancellation token and prevent such a large memory consumption.

[ErikEJ]

Try to update to Microsoft.Data.SqlClient 5.0.2 or 5.1.1

[Stocki916]

Wow that's strange! Im also using "Microsoft.Data.SqlClient" in version 5.1.1 but not directly included in the app, but using another nuget package which contains this.
So i added "Microsoft.Data.SqlClient" also directly to the app, and now the memory usage is also at 2GB WITH cancellation Token!

Thanks for your help - now i need to understand what happened here :-D

[ErikEJ]

@roji Any plans to adopt 5.0.2 for EF Core 7.0.x ?

[roji]

@ErikEJ yeah, I think we'll be doing that - though it's probably wise to wait a little bit to see that 5.0.2 is stable...

[roji]

Duplicate of dotnet/SqlClient#1810

[ErikEJ]

@roji agree, issue is that it will have small adoption now that higher versions are out (5.1.x)