--blame-hang causing crash with AccessViolationException

[nick-beer]

I'm using --blame-hang in an invocation to dotnet vstest (via a runsettings file) on an AzDO pipeline. In the logs, I can see that when the blame collector identifies a hang, it tries to create a dump of the hung process, but it's failing with the following AccessViolationException:

Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Repeat 2 times:
--------------------------------
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.WindowsHangDumper+NativeMethods.MiniDumpWriteDump(IntPtr, UInt32, Microsoft.Win32.SafeHandles.SafeFileHandle, MINIDUMP_TYPE, MINIDUMP_EXCEPTION_INFORMATION ByRef, IntPtr, IntPtr)
--------------------------------
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.WindowsHangDumper.CollectDump(System.Diagnostics.Process, System.String, Microsoft.TestPlatform.Extensions.BlameDataCollector.DumpTypeOption)
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.WindowsHangDumper.Dump(Int32, System.String, Microsoft.TestPlatform.Extensions.BlameDataCollector.DumpTypeOption)
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.ProcessDumpUtility.HangDump(Int32, System.String, Microsoft.TestPlatform.Extensions.BlameDataCollector.DumpTypeOption, System.String, System.Action`1<System.String>)
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.ProcessDumpUtility.StartHangBasedProcessDump(Int32, System.String, Boolean, System.String, System.Action`1<System.String>)
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.BlameCollector.CollectDumpAndAbortTesthost()
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.InactivityTimer+<>c__DisplayClass1_0.<.ctor>b__0(System.Object)
   at System.Threading.TimerQueueTimer+<>c.<.cctor>b__23_0(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.TimerQueueTimer.CallCallback(Boolean)
   at System.Threading.TimerQueueTimer.Fire(Boolean)
   at System.Threading.TimerQueue.FireNextTimers()
   at System.Threading.TimerQueue.AppDomainTimerCallback(Int32)

Because of this crash, the test host isn't aborted, which requires the pipeline stage to terminate at its given timeout.

We're using version 5.0.302 of the dotnet SDK on our pipeline agents. I have limited access to the agents, so I'm hoping maybe someone's seen this before, or has a troubleshooting step I can try? If not, I can try to gain access to one of the agents to track the issue down.

[nohwnd]

Hello, the windows hang dumper uses only this p\Invoke call, and there is no option to switch to the native hang dumper of .NET Core, I did it this way to keep it simpler. And to avoid depending on external executables (procdump) on Windows.

I haven't seen it fail in this yet. Does your code target net5.0 target framework? Or do you only build with 5.0 SDK and target older framework?

We could possibly add option to force the native 5.0 hang dumper on windows. But that would require code changes, and a new build, so it would take time.

[nick-beer]

Our target framework is <TargetFramework>net5.0-windows</TargetFramework>. On our CI machines, we're using the dotnet-install scripts to install the SDK/runtime.

It also appears to be working-ish on some agents. On some agents, it's able to create a full dump that I can open and debug, but I'm not able to debug the mini dumps it creates. The visual studio debugger indicates that there was no compatible code running (although I know there was managed code running because I hand crafted the hang).

The various agents we run on all have different sets of software installed - what's really required for this collector to work correctly?

[nohwnd]

Once this is shipped you will be able to use the VSTEST_DUMP_FORCEPROCDUMP=1 env variable to force using procdump over the PInvoke call for hang dumps. You need procdump installed in your PATH. Easiest way to install it on azdo is via choco.