Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wasm][aot] Windows: mono-aot-cross crashes with 0xC0000374 compiling MessagePack.dll #53546

Closed
radical opened this issue Jun 1, 2021 · 8 comments · Fixed by #56535
Closed
Assignees
Labels
arch-wasm WebAssembly architecture area-Codegen-AOT-mono
Milestone

Comments

@radical
Copy link
Member

radical commented Jun 1, 2021

Based off #53314 .

Steps to reproduce:

  • install dotnet preview 4 (dotnet.microsoft.com/download/dotnet/6.0), into a system location
  • install the workload packs with: dotnet workload install microsoft-net-sdk-blazorwebassembly-aot. You would need to run that as root
  • create a new blazor project with dotnet new blazorwasm
  • Add <RunAOTCompilation>true</RunAOTcompilation>, and <PackageReference Include="Microsoft.AspNetCore.SignalR.Protocols.MessagePack" Version="6.0.0-preview.4.21253.5" /> to the project
  • build with dotnet publish -c:release -p:RunAOTCompilation=true -bl

Fails while running mono-aot-cross for MessagePack.dll:

Running: C:\Program Files\dotnet\packs\microsoft.netcore.app.runtime.aot.win-x64.cross.browser-wasm\6.0.0-preview.4.21222.10\Sdk\..\tools\mono-aot-cross.exe --debug
--llvm "--aot=no-opt,static,direct-icalls,deterministic,nodebug,llvm-path=C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\bin\,llvmonly,interp,asmonly,llvm-outfile=C:\dev\bl0\obj\release\net6.0\wasm\MessagePack.dll.bc" MessagePack.dll (TaskId:134)
       Using working directory: C:\dev\bl0\obj\release\net6.0\linked (TaskId:134)
       Setting environment variables for execution: (TaskId:134)
           MONO_PATH = C:\dev\bl0\obj\release\net6.0\linked;C:\Users\Ankit Jain\.nuget\packages\microsoft.netcore.app.runtime.mono.browser-wasm\6.0.0-preview.4.21253.7\runtimes\browser-wasm\native\;C:\Users\Ankit Jain\.nuget\packages\microsoft.netcore.app.runtime.mono.browser-wasm\6.0.0-preview.4.21253.7\runtimes\browser-wasm\\lib\net6.0 (TaskId:134)
           MONO_ENV_OPTIONS =  (TaskId:134)

Exit code: -1073740940 (TaskId:134)

System.Exception: Error:

  at Utils.RunProcess(String path, String args, IDictionary`2 envVars, String workingDir, Boolean ignoreErrors, Boolean silent, MessageImportance outputMessageImportance, MessageImportance debugMessageImportance) in MonoAOTCompiler.dll:token 0x600002f+0x1fd
  at MonoAOTCompiler.PrecompileLibrary(ITaskItem assemblyItem, String monoPaths) in MonoAOTCompiler.dll:token 0x600002b+0x436 (TaskId:134)

exit code -1073740940(0xC0000374, heap corruption?)

cc @vargaz @lewing @radekdoulik @steveisok

@radical radical added arch-wasm WebAssembly architecture area-Codegen-AOT-mono labels Jun 1, 2021
@ghost
Copy link

ghost commented Jun 1, 2021

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Based off #53314 .

Steps to reproduce:

  • install dotnet preview 4 (dotnet.microsoft.com/download/dotnet/6.0), into a system location
  • install the workload packs with: dotnet workload install microsoft-net-sdk-blazorwebassembly-aot. You would need to run that as root
  • create a new blazor project with dotnet new blazorwasm
  • Add <RunAOTCompilation>true</RunAOTcompilation>, and <PackageReference Include="Microsoft.AspNetCore.SignalR.Protocols.MessagePack" Version="6.0.0-preview.4.21253.5" /> to the project
  • build with dotnet publish -c:release -p:RunAOTCompilation=true -bl

Fails while running mono-aot-cross for MessagePack.dll:

Running: C:\Program Files\dotnet\packs\microsoft.netcore.app.runtime.aot.win-x64.cross.browser-wasm\6.0.0-preview.4.21222.10\Sdk\..\tools\mono-aot-cross.exe --debug
--llvm "--aot=no-opt,static,direct-icalls,deterministic,nodebug,llvm-path=C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\bin\,llvmonly,interp,asmonly,llvm-outfile=C:\dev\bl0\obj\release\net6.0\wasm\MessagePack.dll.bc" MessagePack.dll (TaskId:134)
       Using working directory: C:\dev\bl0\obj\release\net6.0\linked (TaskId:134)
       Setting environment variables for execution: (TaskId:134)
           MONO_PATH = C:\dev\bl0\obj\release\net6.0\linked;C:\Users\Ankit Jain\.nuget\packages\microsoft.netcore.app.runtime.mono.browser-wasm\6.0.0-preview.4.21253.7\runtimes\browser-wasm\native\;C:\Users\Ankit Jain\.nuget\packages\microsoft.netcore.app.runtime.mono.browser-wasm\6.0.0-preview.4.21253.7\runtimes\browser-wasm\\lib\net6.0 (TaskId:134)
           MONO_ENV_OPTIONS =  (TaskId:134)

Exit code: -1073740940 (TaskId:134)

System.Exception: Error:

  at Utils.RunProcess(String path, String args, IDictionary`2 envVars, String workingDir, Boolean ignoreErrors, Boolean silent, MessageImportance outputMessageImportance, MessageImportance debugMessageImportance) in MonoAOTCompiler.dll:token 0x600002f+0x1fd
  at MonoAOTCompiler.PrecompileLibrary(ITaskItem assemblyItem, String monoPaths) in MonoAOTCompiler.dll:token 0x600002b+0x436 (TaskId:134)

exit code -1073740940(0xC0000374, heap corruption?)

cc @vargaz @lewing @radekdoulik @steveisok

Author: radical
Assignees: -
Labels:

arch-wasm, area-Codegen-AOT-mono

Milestone: -

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Jun 1, 2021
@lewing lewing added this to the 6.0.0 milestone Jun 1, 2021
@lewing lewing removed the untriaged New issue has not been triaged by the area owner label Jun 1, 2021
@vargaz
Copy link
Contributor

vargaz commented Jun 1, 2021

Need some kind of stacktrace in order to track this down.

@sxotney
Copy link

sxotney commented Jul 1, 2021

@vargaz I'm happy to provide one here only it's a bit tricky to get at so would need some help in the right direction.

The output is already using diagnostic verbosity. From what I understand AOT build comes after C# converts to IL and then to C, so I take it I need to run the correct emscripten against a packaged dll?

@lewing
Copy link
Member

lewing commented Jul 12, 2021

@sxotney are you able to reproduce this with later previews?

@sxotney
Copy link

sxotney commented Jul 13, 2021

I tried just this week with 6.0.0-preview.6.21355.2 and the issue still occurs. I'll try a preview 7 and revert back asap

@sxotney
Copy link

sxotney commented Jul 15, 2021

Still happening in 6.0.0-preview.7.21363.16

@lewing
Copy link
Member

lewing commented Jul 15, 2021

cc @vargaz

@radical radical changed the title [wasm][aot] Windows: mono-aot-cross crashes with 0xC0000374 [wasm][aot] Windows: mono-aot-cross crashes with 0xC0000374 compiling MessagePack.dll Jul 23, 2021
@sxotney
Copy link

sxotney commented Jul 26, 2021

FYI - Still present in 6.0.0-rc.1.21376.1

radekdoulik added a commit to radekdoulik/runtime that referenced this issue Jul 29, 2021
Fixes dotnet#56526 and probably also
dotnet#53546

Allocate enough memory when `SIZEOF_REGISTER == 4`, so that code like

    defs [ins->dreg + 1] = NULL;
    defs [ins->dreg + 2] = NULL;

doesn't write after allocated range.
radekdoulik added a commit that referenced this issue Jul 29, 2021
Fixes #56526 and probably also
#53546

Allocate enough memory when `SIZEOF_REGISTER == 4`, so that code like

    defs [ins->dreg + 1] = NULL;
    defs [ins->dreg + 2] = NULL;

doesn't write after allocated range.
vargaz added a commit to vargaz/runtime that referenced this issue Jul 29, 2021
It was replacing 64 bit opcodes with 32 bit ones while keeping the input/output 64 bit
vregs, leading to memory overwrites in later passes. Instead simply keep the 64
bit opcodes, llvm can handle them.

Fixes dotnet#53546.
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 29, 2021
vargaz added a commit that referenced this issue Jul 29, 2021
It was replacing 64 bit opcodes with 32 bit ones while keeping the input/output 64 bit
vregs, leading to memory overwrites in later passes. Instead simply keep the 64
bit opcodes, llvm can handle them.

Fixes #53546.
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 29, 2021
Lxiamail added a commit that referenced this issue Aug 5, 2021
* bring up WASM AOT microbenchmarks run

* bring up WASM AOT microbenchmarks run

* adding diagnostice log for testing

* commend out some jobs while testing to save lab run resource

* commend out some jobs while testing to save lab run resource

* Add WASM AOT in not scheduled run

* Only run selected jobs

* add "ls -l" to print out permission before calling performance-setup.sh

* add "ls -l" to print out permission before calling performance-setup.sh

* Add "--buildTimeout 3600" to benchmarkdotnet to extend default timeout time. Adjust -aotcompilermode format to "--aotcompilermode=wasm".

* Add "--buildTimeout 3600" to benchmarkdotnet to extend default timeout time. Adjust -aotcompilermode format to "--aotcompilermode=wasm".

* Correct runtimeSrcDir value

* Correct --runtimeSrcDir location to $HELIX_CORRELATION_PAYLOAD

* Correct --runtimeSrcDir location to $HELIX_CORRELATION_PAYLOAD

* Remove not used parameters in WASM AOT mode and  add temperary diagnostic info

* Remove not used parameters in WASM AOT mode and  add temperary diagnostic info

* make \$HELIX_CORRELATION_PAYLOAD/dotnet-wasm to be the root directory of runtime repo source

* Only copy runtime repo source directory to helix payload when it's wasm aot

* Add diagnosic log for testing only

* omite unneeded runtime repo file from helix payload

* omite unneeded runtime repo file from helix payload

* use rsync instead of cp exclusion command since cp doesn't work with sub directory as expected.

* Add temp diagnostic logging, should be removed before PR.

* remove artifacts/obj from helix payload

* remove artifacts/obj from helix payload

* copy back helix job for diagnosis and silent rsync

* For testing only, clone performance repor alicial/hikeMicro-benchmark private branch, which contains copying back helix job folder for diagnosis

* enable rsync verbose for diagnosis

* add --keepfiles for wasm intepreter as well for diagnosis

* Remove temporary diagnostic code

* recover all the jobs in the pipeline

* recover all the jobs in the pipeline

* copy wasm build drop to the location that aot build expects

* using private perf branch to enable binlog for benchmarkdotnet for diagnosis

* comment out none wasm runs temporarily for testing only

* comment out none wasm runs temporarily for testing only

* add "--keepfiles" to keep diagnostic files

* fixed $wasm_dotnet_path/artifacts/BrowserWasm/artifacts path

* instead of copy, move $wasm_dotnet_path/artifacts/BrowserWasm/artifacts/* to save space

* benchmarkdotnet using --cli \$HELIX_CORRELATION_PAYLOAD/dotnet-wasm/dotnet.sh instead

* benchmarkdotnet using --cli \$HELIX_CORRELATION_PAYLOAD/dotnet-wasm/dotnet.sh instead

* add executable permission

* Enable other non run to compare

* Enable other non run to compare

* remove "--cli \$HELIX_CORRELATION_PAYLOAD/dotnet-wasm/dotnet.sh"

* fixes per CR

* copy EMSDK_PATH to expected location in helix payload

* install emsdk

* fix emsdk path

* minor fix

* removed more not used files to reduce helix payload size

* remove more jobs for testing purpose

* Fix the download path

* Add BenchmarkDotNet sources to the payload

* Fix __download__ path

* set BenchmarkDotNetSources to use benchmarkdotnet built from source

* use benchmarkdotnet private branch:alicial/tempwasmaot for BenchmarkDotNetSources

* use benchmarkdonet private branch with BenchmarkDotNetSources option

* Debug prints

* Clone B.NET to different location

Fix rsync exclusion
Add more debug prints

* Move B.NET back, clone upstream B.NET

To avoid:

    [INFO] ERROR(S):
    [INFO]   Option 'BenchmarkDotNetSources' is unknown.

* Use my branch for performance repo

* Revert "set BenchmarkDotNetSources to use benchmarkdotnet built from source"

This reverts commit bccfadd.

To avoid:

    [INFO] ERROR(S):
    [INFO]   Option 'BenchmarkDotNetSources' is unknown.

* Disable one more job I missed in merge

* Switch back to nuget packages for BenchmarkDotNet

* tempararily use pr-alicias-branch to test new fixes

* [mono] Fix invalid memory write

Fixes #56526 and probably also
#53546

Allocate enough memory when `SIZEOF_REGISTER == 4`, so that code like

    defs [ins->dreg + 1] = NULL;
    defs [ins->dreg + 2] = NULL;

doesn't write after allocated range.

* [mono] Fix an uninitialized memory access in the ABCREM pass.

Detected by valgrind.

* clean up PR

* add missing param

* fix perf.yml and make wasm interpreter to be consistent with wasm AOT

* comment out more job for fast testing

* remove extra job

* exclude copying __download__ dir instead of delete it.

* add parameters.codeGenType

* fix aot javascriptengine

* correct livelibrariesbuildconfig from release to Release

* install emsdk before copy $source_directory

* add back all the temporarily commented out jobs in perf.yml

* clean up work around code

* minor clean up

* remove performance submodule

* switch back to use performance repo main branch

* add --quiet to git clone

Co-authored-by: Radek Doulik <radekdoulik@gmail.com>
Co-authored-by: Zoltan Varga <vargaz@gmail.com>
@ghost ghost locked as resolved and limited conversation to collaborators Aug 28, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly architecture area-Codegen-AOT-mono
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants