OOM copying a file to memory in Blazor

[verdie-g]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

In a Blazor Wasm app, copying a 1GB file to memory will cause a OOM exception.

I'm aware that, according to https://learn.microsoft.com/en-us/aspnet/core/blazor/file-uploads?view=aspnetcore-8.0#file-uploads, copying to file in memory is a bad practice. In my exact use-case, I'm not directly copying the bytes, but I'm parsing a very large nettrace file for this project https://verdie-g.github.io/dotnet-event-viewer.

Expected Behavior

I would expect the browser to allow allocating 1GB on a 16GB machine.

Steps To Reproduce

1. dotnet new blazorwasm -n BlazorTest
2. Replace the content of Home.razor with

@page "/"

<PageTitle>Home</PageTitle>

<h1>Hello, world!</h1>

<InputFile OnChange="LoadFiles" />

@code {
    private async Task LoadFiles(InputFileChangeEventArgs e)
    {
        List<byte[]> list = new();
        await using var s = e.File.OpenReadStream(maxAllowedSize: long.MaxValue);
        while (true)
        {
            byte[] buffer;
            int read;
            try
            {
                buffer = new byte[64 * 1024];
                read = await s.ReadAsync(buffer);
            }
            catch
            {
                Console.WriteLine("Read " + list.Sum(x => x.Length));
                throw;
            }

            if (read == 0)
            {
                break;
            }

            list.Add(buffer);
        }

        Console.WriteLine("Done");
    }
}

3. cd BlazorTest && dotnet run
4. Navigate to the blazor app
5. Upload a 1GB file (created for example with File.WriteAllBytes("1gb.bin", new byte[1_000_000_000]))
6. Wait
7. Observe the exception in the console

Exceptions (if any)

When the OOM occurs in BrowserFileStream.ReadAsync

Microsoft.JSInterop.JSException: Out of memory
   at System.Runtime.InteropServices.JavaScript.JSMarshalerArgument.ToManaged(Byte[]& value)
   at Microsoft.AspNetCore.Components.WebAssembly.Services.DefaultWebAssemblyJSRuntime.__Wrapper_ReceiveByteArrayFromJS_980098407(JSMarshalerArgument* __arguments_buffer)
Error: Out of memory
    at Jn (http://localhost:5140/_framework/dotnet.runtime.8.0.2.io1j2tel8l.js:3:31614)
    at kr (http://localhost:5140/_framework/dotnet.runtime.8.0.2.io1j2tel8l.js:3:35529)
    at Object.<anonymous> (http://localhost:5140/_framework/dotnet.runtime.8.0.2.io1j2tel8l.js:3:180123)
    at Object.sendByteArray (http://localhost:5140/_framework/blazor.webassembly.js:1:45278)
    at Array.R (http://localhost:5140/_framework/blazor.webassembly.js:1:6844)
    at JSON.stringify (<anonymous>)
    at _ (http://localhost:5140/_framework/blazor.webassembly.js:1:6695)
    at http://localhost:5140/_framework/blazor.webassembly.js:1:2907
   at Microsoft.JSInterop.JSRuntime.<InvokeAsync>d__16`1[[System.Byte[], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Microsoft.AspNetCore.Components.PullFromJSDataStream.RequestDataFromJSAsync(Int32 numBytesToRead)
   at Microsoft.AspNetCore.Components.PullFromJSDataStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Components.Forms.BrowserFileStream.CopyFileDataIntoBuffer(Memory`1 destination, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Components.Forms.BrowserFileStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at BlazorTest.Pages.Home.LoadFiles(InputFileChangeEventArgs e) in C:\Users\grego\Documents\BlazorTest\Pages\Home.razor:line 26
   at BlazorTest.Pages.Home.LoadFiles(InputFileChangeEventArgs e) in C:\Users\grego\Documents\BlazorTest\Pages\Home.razor:line 37
   at Microsoft.AspNetCore.Components.ComponentBase.CallStateHasChangedOnAsyncCompletion(Task task)
    at b.endInvokeDotNetFromJS (http://localhost:5140/_framework/blazor.webassembly.js:1:3136)
    at Object.gn [as endInvokeDotNetFromJS] (http://localhost:5140/_framework/blazor.webassembly.js:1:58943)
    at http://localhost:5140/_framework/dotnet.runtime.8.0.2.io1j2tel8l.js:3:177853
    at Ul (http://localhost:5140/_framework/dotnet.runtime.8.0.2.io1j2tel8l.js:3:178687)
    at http://localhost:5140/_framework/dotnet.native.wasm:wasm-function[350]:0x1faef
    at http://localhost:5140/_framework/dotnet.native.wasm:wasm-function[246]:0x1bf8c
    at http://localhost:5140/_framework/dotnet.native.wasm:wasm-function[239]:0xf173
    at http://localhost:5140/_framework/dotnet.native.wasm:wasm-function[307]:0x1e7e5
    at http://localhost:5140/_framework/dotnet.native.wasm:wasm-function[328]:0x1efdb
    at http://localhost:5140/_framework/dotnet.native.wasm:wasm-function[218]:0xcfed

but it can also occur in the byte array allocation. In that case it's a System.OOM exception.

.NET Version

9.0.100-preview.3.24204.13

[mkArtakMSFT]

@lewing do you know what may prevent a file size of 1GB be a problem? Any limitations in the runtime layer?

@verdie-g what browser are you using?

[verdie-g]

what browser are you using?

Google Chrome.

[lewing]

The wasm address space for the MVP is 32 bits, that means that all of your assemblies, your runtime, any allocations, and all I/O buffering needs to fit into 2GB. To process large nettrace files you will likely need to parse the nettrace file incrementally and stay within the 2GB rss for Wasm data. There are wasm64 extensions that remove this limit but they are not broadly implemented at this time so we'd love to hear more about your use case.

[verdie-g]

Ah I should have read more about wasm before starting this project :(

The project is https://verdie-g.github.io/dotnet-event-viewer. A nettrace analyzer tool that could be an alternative to Perfview. Nettrace files can be quite large (e.g. a 20 minutes trace of GCAllocationTick on one my service weights 10 GB). When, a nettrace is loaded on that tool, it's indeed parsed incrementally, but the thing is that decompressing the data can do a 5x on the size. So obviously that 10GB trace wouldn't even fit in memory but that tool would still cover most of the use-cases if it could use all the available memory.

[verdie-g]

Also I'm surprised the 2GB limit is reached there. The file is 1GB but what else could use the other GB?

[verdie-g]

stay within the 2GB rss for Wasm data

If wasm uses 32-bits addressing wouldn't the limit be 4 GiB ? 🤔

[verdie-g]

Apologies for the spam but I noticed another issue:

1. Reuse the same code snippet as OOM copying a file to memory in Blazor #55694
2. Upload a 1 GB file
3. Wait for the OOM
4. Upload a 50 MB
5. Observe an instant OOM

In a non-wasm environment I would expect the GC to be super aggressive to collect objects when the available memory is low. I'm concerned this is not the case here.

[halter73]

stay within the 2GB rss for Wasm data

If wasm uses 32-bits addressing wouldn't the limit be 4 GiB ? 🤔

Maybe we're not using the right Emscripten flags for 4 GiB support. Historically, Emscripten didn't support 4 GiB because it did things like signed bitshifts of pointers. Once Emscripten fixed those issues, they added a flag (emcc -s MAXIMUM_MEMORY=4GB) to opt-in to 4 GiB support.

https://v8.dev/blog/4gb-wasm-memory

Edit: It looks like we did add an EmccMaximumHeapSize msbuild property in .NET 8 that controls the MAXIMUM_MEMORY flag as part of dotnet/runtime#91256, but that was to help address dotnet/runtime#84638 which asks to change the default from 2 GiB to 256 MiB rather than 4 GiB to better support Mobile Safari. The plan appears to only change the default for Mobile Safari though. Hopefully that can be done at runtime. @maraf @lambdageek @pavelsavara

[lewing]

Looking closer, there are additional issues. First it appears that the sample code assumes that every read fills the 64k buffer completely, with stream apis you always want to assume the amount read can be less than the entire buffer and track the position in the buffer until it is full. In your sample if the amount read is smaller than 64k you end up allocating another 64k buffer and appending the partially full buffer to the list. Second the API in question here BrowserFileStream is part of aspnetcore not runtime and not directly familiar with how it buffers the I/O.

[verdie-g]

In your sample if the amount read is smaller than 64k you end up allocating another 64k buffer and appending the partially full buffer to the list

The file upload is just an example. The issue can be reproduced by just allocating empty buffers

const int bufferSize = 64 * 1024;
const int oneGig = 1024 * 1024 * 1024;
List<byte[]> list = new();
for (int i = 0; i < oneGig / bufferSize; i += 1)
{
    list.Add(new byte[bufferSize]);
}

Second the API in question here BrowserFileStream is part of aspnetcore not runtime and not directly familiar with how it buffers the I/O.

The issue was moved from dotnet/aspnetcore.

[verdie-g]

@mkArtakMSFT the issue is not solved, could you please reopen the thread

[verdie-g]

or @lewing?