Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System.Text.Json.Tests crashing on Mono #58828

Closed
ViktorHofer opened this issue Sep 8, 2021 · 11 comments · Fixed by #73480
Closed

System.Text.Json.Tests crashing on Mono #58828

ViktorHofer opened this issue Sep 8, 2021 · 11 comments · Fixed by #73480
Assignees
Milestone

Comments

@ViktorHofer
Copy link
Member

Configuration: net6.0-Linux-Debug-x64-Mono_release-Ubuntu.1604.Amd64.Open

Dump available: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-58558-merge-1c1e3bdb886344679d/System.Text.Json.Tests/1/core.1000.13620?sv=2019-07-07&se=2021-09-22T16%3A19%3A20Z&sr=c&sp=rl&sig=sjEJQ%2BswuvmnAmzLahc2AgFLgot5gYVvnZSow7cL6Jg%3D

https://dev.azure.com/dnceng/public/_build/results?buildId=1337984&view=ms.vss-test-web.build-test-results-tab&runId=39211016&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab&resultId=187646

pushd .
/datadisks/disk1/work/B13909DB/p/dotnet exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/datadisks/disk1/work/B13909DB/w/A4760949/e /datadisks/disk1/work/B13909DB/w/A4760949/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 2770 of 2809 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 2)

=================================================================
	Native Crash Reporting
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

=================================================================
	Native stacktrace:
=================================================================
	0x7f9dacb3c5a2 - Unknown
	0x7f9dacae1eee - Unknown
	0x7f9daca45d96 - Unknown
	0x7f9dadb44390 - Unknown
	0x7f9daca09e05 - Unknown
	0x7f9daca21d7b - Unknown
	0x7f9daca2d971 - Unknown
	0x7f9dadb3a6ba - Unknown
	0x7f9dacdcb51d - Unknown

=================================================================
	External Debugger Dump:
=================================================================
[New LWP 13621]
[New LWP 13622]
[New LWP 13623]
[New LWP 13624]
[New LWP 13627]
[New LWP 13628]
[New LWP 13629]
[New LWP 13631]
[New LWP 13632]
[New LWP 13638]
[New LWP 13655]
[New LWP 13671]
[New LWP 13674]
[New LWP 13675]
[New LWP 13676]
[New LWP 13677]
[New LWP 13678]
[New LWP 13679]
[New LWP 13680]
[New LWP 13681]
[New LWP 13682]
[New LWP 13683]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
30	../sysdeps/unix/sysv/linux/sigsuspend.c: No such file or directory.
  Id   Target Id         Frame 
* 1    Thread 0x7f9dadf63740 (LWP 13620) "dotnet" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  2    Thread 0x7f9dabfff700 (LWP 13621) "SGen worker" 0x00007f9dadb43f7b in __waitpid (pid=13685, stat_loc=0x7f9dabffdb40, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
  3    Thread 0x7f9daa490700 (LWP 13622) "dotnet" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  4    Thread 0x7f9daa28f700 (LWP 13623) "Finalizer" 0x00007f9dadb42827 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x7f9daccac2a0 <finalizer_sem>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
  5    Thread 0x7f9da8356700 (LWP 13624) "dotnet" 0x00007f9dadb4351d in read () at ../sysdeps/unix/syscall-template.S:84
  6    Thread 0x7f9da82bc700 (LWP 13627) ".NET Long Runni" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  7    Thread 0x7f9da2548700 (LWP 13628) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  8    Thread 0x7f9da1c93700 (LWP 13629) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  9    Thread 0x7f9da1a41700 (LWP 13631) ".NET Long Runni" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  10   Thread 0x7f9da1840700 (LWP 13632) ".NET Long Runni" __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
  11   Thread 0x7f9d85aff700 (LWP 13638) ".NET Timers" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  12   Thread 0x7f9d789f9700 (LWP 13655) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  13   Thread 0x7f9da1c52700 (LWP 13671) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  14   Thread 0x7f9d7b77b700 (LWP 13674) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  15   Thread 0x7f9d7b3ff700 (LWP 13675) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  16   Thread 0x7f9d7a8ff700 (LWP 13676) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  17   Thread 0x7f9d7a6fe700 (LWP 13677) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  18   Thread 0x7f9d7a05b700 (LWP 13678) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  19   Thread 0x7f9d7977b700 (LWP 13679) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  20   Thread 0x7f9d78dff700 (LWP 13680) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  21   Thread 0x7f9d78bfe700 (LWP 13681) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  22   Thread 0x7f9d786f7700 (LWP 13682) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  23   Thread 0x7f9d784f6700 (LWP 13683) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30

Thread 23 (Thread 0x7f9d784f6700 (LWP 13683)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d784f5280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d784f58d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d784f58d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d5401b350 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d784f5940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 22 (Thread 0x7f9d786f7700 (LWP 13682)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d786f6280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d786f68d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d786f68d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 21 (Thread 0x7f9d78bfe700 (LWP 13681)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d78bfd280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d78bfd8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d78bfd8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 20 (Thread 0x7f9d78dff700 (LWP 13680)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d78dfe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d78dfe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d78dfe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d4c007f20 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d78dfe940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 19 (Thread 0x7f9d7977b700 (LWP 13679)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7977a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7977a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7977a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x000055c44f89e330 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d7977a940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 18 (Thread 0x7f9d7a05b700 (LWP 13678)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a05a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a05a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a05a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 17 (Thread 0x7f9d7a6fe700 (LWP 13677)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a6fd280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a6fd8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a6fd8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 16 (Thread 0x7f9d7a8ff700 (LWP 13676)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a8fe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a8fe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a8fe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 15 (Thread 0x7f9d7b3ff700 (LWP 13675)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7b3fe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7b3fe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7b3fe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d60000fe0 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d7b3fe940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 14 (Thread 0x7f9d7b77b700 (LWP 13674)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7b77a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7b77a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7b77a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 13 (Thread 0x7f9da1c52700 (LWP 13671)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1c51280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9da1c518d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9da1c518d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 12 (Thread 0x7f9d789f9700 (LWP 13655)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d789f8280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d789f88d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d789f88d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 11 (Thread 0x7f9d85aff700 (LWP 13638)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d85afe240) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x7f9d7c00ca40) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac171bf0 in ?? ()
#7  0x00007f9dac174980 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9dac1749b0 in ?? ()
#11 0x00007f9dac174980 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x00007f9d7c000fe0 in ?? ()
#14 0x00007f9d85afe940 in ?? ()
#15 0x00007f9d85afe850 in ?? ()
#16 0x00007f9d7c0008c0 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9dac1749b0 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x00007f9d7c001050 in ?? ()
#23 0x0000000000000000 in ?? ()

Thread 10 (Thread 0x7f9da1840700 (LWP 13632)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007f9dadb3cdbd in __GI___pthread_mutex_lock (mutex=0x7f9daccb8cd0 <pin_queue_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2  0x00007f9daca21c5d in mono_os_mutex_lock (mutex=0x7f9daccb8cd0 <pin_queue_mutex>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-os-mutex.h:105
#3  sgen_init_pinning_for_conc () at /__w/1/s/src/mono/mono/sgen/sgen-pinning.c:52
#4  0x00007f9dac9f9d09 in collect_nursery (reason=0x7f9dacbf739f "Nursery full", is_overflow=0) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:1858
#5  0x00007f9dac9f6453 in sgen_perform_collection_inner (requested_size=<optimized out>, generation_to_collect=<optimized out>, reason=<optimized out>, forced_serial=<optimized out>, stw=<optimized out>) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2656
#6  sgen_perform_collection (requested_size=4096, generation_to_collect=0, reason=0x7f9dacbf739f "Nursery full", forced_serial=0, stw=1) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2768
#7  0x00007f9dac9f6253 in sgen_ensure_free_space (size=4096, generation=<optimized out>) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2622
#8  0x00007f9dac9ea417 in sgen_alloc_obj_nolock (vtable=0x55c44f552ce8, size=144) at /__w/1/s/src/mono/mono/sgen/sgen-alloc.c:279
#9  0x00007f9dac9ea8f9 in sgen_alloc_obj (vtable=0x55c44f552ce8, size=144) at /__w/1/s/src/mono/mono/sgen/sgen-alloc.c:454
#10 0x00007f9dac9b9606 in mono_gc_alloc_obj (vtable=0x7f9daccb8cd0 <pin_queue_mutex>, size=128) at /__w/1/s/src/mono/mono/metadata/sgen-mono.c:911
#11 0x00000000411246f8 in ?? ()
#12 0x00007f9dac140c88 in ?? ()
#13 0x00007f9d80000f30 in ?? ()
#14 0x00007f9d800008c0 in ?? ()
#15 0x000055c44f552ce8 in ?? ()
#16 0x0000000000000090 in ?? ()
#17 0x00007f9da183aa48 in ?? ()
#18 0x00007f9dac140d18 in ?? ()
#19 0x00007f9da183a2e0 in ?? ()
#20 0x00007f9dac140b70 in ?? ()
#21 0x00000000410f9d44 in ?? ()
#22 0x00007f9da183a870 in ?? ()
#23 0x00007f9da183a4a0 in ?? ()
#24 0x00007f9d9806a468 in ?? ()
#25 0x00007f9daddf5c88 in ?? ()
#26 0x0000000000000003 in ?? ()
#27 0x0000000000000008 in ?? ()
#28 0x0000000000000003 in ?? ()
#29 0x00000000414c6054 in ?? ()
#30 0x00007f9dac140ba0 in ?? ()
#31 0x0000000000000008 in ?? ()
#32 0x0000000000000008 in ?? ()
#33 0x00000000414c5e70 in ?? ()
#34 0x0000000000000002 in ?? ()
#35 0x0000000000000004 in ?? ()
#36 0x00007f9dac140b70 in ?? ()
#37 0x00007f9da1540c30 in ?? ()
#38 0x00007f9daddf5c88 in ?? ()
#39 0x00000000414c5dc0 in ?? ()
#40 0x0000000000000003 in ?? ()
#41 0x0000000000000008 in ?? ()
#42 0x0000000000000002 in ?? ()
#43 0x00000000412fa2b4 in ?? ()
#44 0x00007f9dac3d24a8 in ?? ()
#45 0x00007f9dac3d2518 in ?? ()
#46 0x00007f9dac3d2518 in ?? ()
#47 0x00000000413cc670 in ?? ()
#48 0x00007f9d81bedda0 in ?? ()
#49 0x0000001500000004 in ?? ()
#50 0x0000000080000000 in ?? ()
#51 0x0000000000000000 in ?? ()

Thread 9 (Thread 0x7f9da1a41700 (LWP 13631)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1a38d00) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:133
#4  0x00007f9dadb3cdbd in __GI___pthread_mutex_lock (mutex=0x7f9daccc29e8 <sgen_gc_mutex>) at ../nptl/pthread_mutex_lock.c:80
#5  0x00007f9dac9f6c79 in mono_os_mutex_lock (mutex=<optimized out>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-os-mutex.h:105
#6  mono_coop_mutex_lock (mutex=<optimized out>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:57
#7  sgen_gc_lock () at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:3939
#8  0x00007f9dac9ba0dc in mono_gc_alloc_vector (vtable=0x55c44f13e268, size=64, max_length=16) at /__w/1/s/src/mono/mono/metadata/sgen-mono.c:1338
#9  0x0000000041115e0b in ?? ()
#10 0x00007f9dac1a60c8 in ?? ()
#11 0x00007f9d8c000f30 in ?? ()
#12 0x0000000000000040 in ?? ()
#13 0x000055c44f13e268 in ?? ()
#14 0x00007f9d8c0008c0 in ?? ()
#15 0x00007f9da1a3b888 in ?? ()
#16 0x00007f9da1a39450 in ?? ()
#17 0x00007f9da1a393a0 in ?? ()
#18 0x0000000000000001 in ?? ()
#19 0x00000000410fa4ec in ?? ()
#20 0x00007f9dac1b4f70 in ?? ()
#21 0x0000000000000010 in ?? ()
#22 0x00007f9dac1b4f70 in ?? ()
#23 0x00007f9dac1a6108 in ?? ()
#24 0x00007f9dac1b4f70 in ?? ()
#25 0x00007f9d84ad55a1 in ?? ()
#26 0x0000000000002767 in ?? ()
#27 0x000000000000001e in ?? ()
#28 0x00007f9da1a39650 in ?? ()
#29 0x0000000000000010 in ?? ()
#30 0x0000000000000010 in ?? ()
#31 0x000000000000000e in ?? ()
#32 0x0000000000000005 in ?? ()
#33 0x000000004111e9c8 in ?? ()
#34 0x00007f9dac1a6058 in ?? ()
#35 0x0000000000000000 in ?? ()

Thread 8 (Thread 0x7f9da1c93700 (LWP 13629)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1c921c0) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac476740 in SystemNative_LowLevelMonitor_TimedWait (monitor=0x7f9d94004db0, timeoutMilliseconds=500) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:192
#5  0x0000000041235c97 in ?? ()
#6  0x00000000000a5485 in ?? ()
#7  0x0000000000000000 in ?? ()

Thread 7 (Thread 0x7f9da2548700 (LWP 13628)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da2547040) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac476740 in SystemNative_LowLevelMonitor_TimedWait (monitor=0x7f9d90039510, timeoutMilliseconds=12000) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:192
#5  0x0000000041235c97 in ?? ()
#6  0x00000000000a4c59 in ?? ()
#7  0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7f9da82bc700 (LWP 13627)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da82bb100) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x7f9d9833eb90) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac145908 in ?? ()
#7  0x00007f9da1ca3920 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9da1ca3950 in ?? ()
#11 0x00007f9da1ca3920 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x00007f9d980f64c0 in ?? ()
#14 0x00007f9da82bb800 in ?? ()
#15 0x00007f9da82bb710 in ?? ()
#16 0x00007f9d98341620 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9da1ca3950 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x00007f9dacbe2d59 in ?? () from /datadisks/disk1/work/B13909DB/p/shared/Microsoft.NETCore.App/7.0.0/libcoreclr.so
#23 0x0000000000000000 in ?? ()

Thread 5 (Thread 0x7f9da8356700 (LWP 13624)):
#0  0x00007f9dadb4351d in read () at ../sysdeps/unix/syscall-template.S:84
#1  0x00007f9dac475a2e in SignalHandlerLoop (arg=0x55c44f79ace0) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_signal.c:321
#2  0x00007f9dadb3a6ba in start_thread (arg=0x7f9da8356700) at pthread_create.c:333
#3  0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 4 (Thread 0x7f9daa28f700 (LWP 13623)):
#0  0x00007f9dadb42827 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x7f9daccac2a0 <finalizer_sem>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem@entry=0x7f9daccac2a0 <finalizer_sem>, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f9dadb428d4 in __new_sem_wait_slow (sem=0x7f9daccac2a0 <finalizer_sem>, abstime=0x0) at sem_waitcommon.c:181
#3  0x00007f9dadb4297a in __new_sem_wait (sem=<optimized out>) at sem_wait.c:29
#4  0x00007f9dac9a9cd6 in mono_os_sem_wait (sem=<optimized out>, flags=MONO_SEM_FLAGS_ALERTABLE) at /__w/1/s/src/mono/mono/mini/../utils/mono-os-semaphore.h:204
#5  mono_coop_sem_wait (sem=<optimized out>, flags=MONO_SEM_FLAGS_ALERTABLE) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-semaphore.h:41
#6  finalizer_thread (unused=<optimized out>) at /__w/1/s/src/mono/mono/metadata/gc.c:876
#7  0x00007f9dac98245a in start_wrapper_internal (start_info=0x0, stack_ptr=<optimized out>) at /__w/1/s/src/mono/mono/metadata/threads.c:1200
#8  0x00007f9dac9822e9 in start_wrapper (data=0x55c44effb890) at /__w/1/s/src/mono/mono/metadata/threads.c:1262
#9  0x00007f9dadb3a6ba in start_thread (arg=0x7f9daa28f700) at pthread_create.c:333
#10 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 3 (Thread 0x7f9daa490700 (LWP 13622)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9daa48f780) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  0x00007f9dacdbf84d in poll () at ../sysdeps/unix/syscall-template.S:84
#4  0x00007f9dacbd813a in ipc_poll_fds (fds=<optimized out>, nfds=1, timeout=4294967295) at /__w/1/s/src/native/eventpipe/ds-ipc-pal-socket.c:470
#5  ds_ipc_poll (poll_handles_data=0x7f9da4002250, poll_handles_data_len=1, timeout_ms=4294967295, callback=0x7f9dacbd7630 <server_warning_callback>) at /__w/1/s/src/native/eventpipe/ds-ipc-pal-socket.c:1082
#6  0x00007f9dacbd5785 in ds_ipc_stream_factory_get_next_available_stream (callback=0x7f9dacbd7630 <server_warning_callback>) at /__w/1/s/src/native/eventpipe/ds-ipc.c:395
#7  0x00007f9dacbd3fe9 in server_thread (data=<optimized out>) at /__w/1/s/src/native/eventpipe/ds-server.c:127
#8  0x00007f9dacbd7611 in ep_rt_thread_mono_start_func (data=0x55c44efee5c0) at /__w/1/s/src/mono/mono/mini/../eventpipe/ep-rt-mono.h:1266
#9  0x00007f9dadb3a6ba in start_thread (arg=0x7f9daa490700) at pthread_create.c:333
#10 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 2 (Thread 0x7f9dabfff700 (LWP 13621)):
#0  0x00007f9dadb43f7b in __waitpid (pid=13685, stat_loc=0x7f9dabffdb40, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00007f9dacb3c6e7 in dump_native_stacktrace (signal=<optimized out>, mctx=<optimized out>) at /__w/1/s/src/mono/mono/mini/mini-posix.c:842
#2  mono_dump_native_crash_info (signal=<optimized out>, mctx=0x7f9dabffe6b0, info=<optimized out>) at /__w/1/s/src/mono/mono/mini/mini-posix.c:869
#3  0x00007f9dacae1eee in mono_handle_native_crash (signal=0x7f9dacbf5d7a "SIGSEGV", mctx=0x7f9dabffe6b0, info=0x7f9dabffe970) at /__w/1/s/src/mono/mono/mini/mini-exceptions.c:2940
#4  0x00007f9daca45d96 in mono_sigsegv_signal_handler_debug (_dummy=11, _info=0x7f9dabffe970, context=0x7f9dabffe840, debug_fault_addr=0x7f9d00000000) at /__w/1/s/src/mono/mono/mini/mini-runtime.c:3663
#5  <signal handler called>
#6  major_copy_or_mark_object_concurrent_with_evacuation (ptr=<optimized out>, obj=<optimized out>, queue=0x7f9dadf71010) at /__w/1/s/src/mono/mono/mini/../sgen/sgen-marksweep-drain-gray-stack.h:137
#7  major_scan_object_concurrent_with_evacuation (full_object=0x7f9dac1b4e00, desc=<optimized out>, queue=<optimized out>) at /__w/1/s/src/mono/mono/mini/../sgen/sgen-scan-object.h:64
#8  0x00007f9daca21d7b in sgen_scan_pin_queue_objects (ctx=...) at /__w/1/s/src/mono/mono/sgen/sgen-pinning.c:84
#9  0x00007f9daca2d971 in thread_func (data=0x0) at /__w/1/s/src/mono/mono/sgen/sgen-thread-pool.c:209
#10 0x00007f9dadb3a6ba in start_thread (arg=0x7f9dabfff700) at pthread_create.c:333
#11 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 1 (Thread 0x7f9dadf63740 (LWP 13620)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7ffd3b805500) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x55c44f9dc790) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac141658 in ?? ()
#7  0x00007f9da22cba10 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9da22cba40 in ?? ()
#11 0x00007f9da22cba10 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x000055c44efe1410 in ?? ()
#14 0x00007ffd3b805bd0 in ?? ()
#15 0x00007ffd3b805ae0 in ?? ()
#16 0x000055c44ef82bf0 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9da22cba40 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x000055c44ef83290 in ?? ()
#23 0x0000000000000000 in ?? ()

=================================================================
	Basic Fault Address Reporting
=================================================================
Memory around native instruction pointer (0x7f9daca09e05):0x7f9daca09df5  84 00 00 00 00 00 0f 1f 44 00 00 41 0f 18 5d 00  ........D..A..].
0x7f9daca09e05  49 8b 45 00 48 89 c7 48 83 e7 f8 48 8b 6f 08 40  I.E.H..H...H.o.@
0x7f9daca09e15  f6 c5 04 74 2c a8 01 74 0c 48 85 ff 74 07 48 8b  ...t,..t.H..t.H.
0x7f9daca09e25  3f 48 83 e7 f8 4c 89 ee e8 fe 22 01 00 48 83 c0  ?H...L...."..H..
./RunTests.sh: line 162: 13620 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/datadisks/disk1/work/B13909DB/w/A4760949/e
----- end Thu Sep 2 16:31:16 UTC 2021 ----- exit code 134 ----------------------------------------------------------
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Sep 8, 2021
@ghost
Copy link

ghost commented Sep 8, 2021

Tagging subscribers to this area: @eiriktsarpalis, @layomia
See info in area-owners.md if you want to be subscribed.

Issue Details

Configuration: net6.0-Linux-Debug-x64-Mono_release-Ubuntu.1604.Amd64.Open

Dump available: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-58558-merge-1c1e3bdb886344679d/System.Text.Json.Tests/1/core.1000.13620?sv=2019-07-07&se=2021-09-22T16%3A19%3A20Z&sr=c&sp=rl&sig=sjEJQ%2BswuvmnAmzLahc2AgFLgot5gYVvnZSow7cL6Jg%3D

https://dev.azure.com/dnceng/public/_build/results?buildId=1337984&view=ms.vss-test-web.build-test-results-tab&runId=39211016&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab&resultId=187646

pushd .
/datadisks/disk1/work/B13909DB/p/dotnet exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/datadisks/disk1/work/B13909DB/w/A4760949/e /datadisks/disk1/work/B13909DB/w/A4760949/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 2770 of 2809 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 2)

=================================================================
	Native Crash Reporting
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

=================================================================
	Native stacktrace:
=================================================================
	0x7f9dacb3c5a2 - Unknown
	0x7f9dacae1eee - Unknown
	0x7f9daca45d96 - Unknown
	0x7f9dadb44390 - Unknown
	0x7f9daca09e05 - Unknown
	0x7f9daca21d7b - Unknown
	0x7f9daca2d971 - Unknown
	0x7f9dadb3a6ba - Unknown
	0x7f9dacdcb51d - Unknown

=================================================================
	External Debugger Dump:
=================================================================
[New LWP 13621]
[New LWP 13622]
[New LWP 13623]
[New LWP 13624]
[New LWP 13627]
[New LWP 13628]
[New LWP 13629]
[New LWP 13631]
[New LWP 13632]
[New LWP 13638]
[New LWP 13655]
[New LWP 13671]
[New LWP 13674]
[New LWP 13675]
[New LWP 13676]
[New LWP 13677]
[New LWP 13678]
[New LWP 13679]
[New LWP 13680]
[New LWP 13681]
[New LWP 13682]
[New LWP 13683]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
30	../sysdeps/unix/sysv/linux/sigsuspend.c: No such file or directory.
  Id   Target Id         Frame 
* 1    Thread 0x7f9dadf63740 (LWP 13620) "dotnet" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  2    Thread 0x7f9dabfff700 (LWP 13621) "SGen worker" 0x00007f9dadb43f7b in __waitpid (pid=13685, stat_loc=0x7f9dabffdb40, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
  3    Thread 0x7f9daa490700 (LWP 13622) "dotnet" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  4    Thread 0x7f9daa28f700 (LWP 13623) "Finalizer" 0x00007f9dadb42827 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x7f9daccac2a0 <finalizer_sem>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
  5    Thread 0x7f9da8356700 (LWP 13624) "dotnet" 0x00007f9dadb4351d in read () at ../sysdeps/unix/syscall-template.S:84
  6    Thread 0x7f9da82bc700 (LWP 13627) ".NET Long Runni" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  7    Thread 0x7f9da2548700 (LWP 13628) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  8    Thread 0x7f9da1c93700 (LWP 13629) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  9    Thread 0x7f9da1a41700 (LWP 13631) ".NET Long Runni" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  10   Thread 0x7f9da1840700 (LWP 13632) ".NET Long Runni" __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
  11   Thread 0x7f9d85aff700 (LWP 13638) ".NET Timers" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  12   Thread 0x7f9d789f9700 (LWP 13655) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  13   Thread 0x7f9da1c52700 (LWP 13671) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  14   Thread 0x7f9d7b77b700 (LWP 13674) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  15   Thread 0x7f9d7b3ff700 (LWP 13675) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  16   Thread 0x7f9d7a8ff700 (LWP 13676) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  17   Thread 0x7f9d7a6fe700 (LWP 13677) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  18   Thread 0x7f9d7a05b700 (LWP 13678) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  19   Thread 0x7f9d7977b700 (LWP 13679) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  20   Thread 0x7f9d78dff700 (LWP 13680) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  21   Thread 0x7f9d78bfe700 (LWP 13681) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  22   Thread 0x7f9d786f7700 (LWP 13682) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  23   Thread 0x7f9d784f6700 (LWP 13683) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30

Thread 23 (Thread 0x7f9d784f6700 (LWP 13683)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d784f5280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d784f58d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d784f58d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d5401b350 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d784f5940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 22 (Thread 0x7f9d786f7700 (LWP 13682)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d786f6280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d786f68d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d786f68d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 21 (Thread 0x7f9d78bfe700 (LWP 13681)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d78bfd280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d78bfd8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d78bfd8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 20 (Thread 0x7f9d78dff700 (LWP 13680)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d78dfe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d78dfe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d78dfe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d4c007f20 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d78dfe940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 19 (Thread 0x7f9d7977b700 (LWP 13679)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7977a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7977a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7977a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x000055c44f89e330 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d7977a940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 18 (Thread 0x7f9d7a05b700 (LWP 13678)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a05a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a05a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a05a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 17 (Thread 0x7f9d7a6fe700 (LWP 13677)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a6fd280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a6fd8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a6fd8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 16 (Thread 0x7f9d7a8ff700 (LWP 13676)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a8fe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a8fe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a8fe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 15 (Thread 0x7f9d7b3ff700 (LWP 13675)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7b3fe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7b3fe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7b3fe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d60000fe0 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d7b3fe940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 14 (Thread 0x7f9d7b77b700 (LWP 13674)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7b77a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7b77a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7b77a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 13 (Thread 0x7f9da1c52700 (LWP 13671)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1c51280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9da1c518d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9da1c518d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 12 (Thread 0x7f9d789f9700 (LWP 13655)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d789f8280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d789f88d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d789f88d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 11 (Thread 0x7f9d85aff700 (LWP 13638)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d85afe240) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x7f9d7c00ca40) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac171bf0 in ?? ()
#7  0x00007f9dac174980 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9dac1749b0 in ?? ()
#11 0x00007f9dac174980 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x00007f9d7c000fe0 in ?? ()
#14 0x00007f9d85afe940 in ?? ()
#15 0x00007f9d85afe850 in ?? ()
#16 0x00007f9d7c0008c0 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9dac1749b0 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x00007f9d7c001050 in ?? ()
#23 0x0000000000000000 in ?? ()

Thread 10 (Thread 0x7f9da1840700 (LWP 13632)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007f9dadb3cdbd in __GI___pthread_mutex_lock (mutex=0x7f9daccb8cd0 <pin_queue_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2  0x00007f9daca21c5d in mono_os_mutex_lock (mutex=0x7f9daccb8cd0 <pin_queue_mutex>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-os-mutex.h:105
#3  sgen_init_pinning_for_conc () at /__w/1/s/src/mono/mono/sgen/sgen-pinning.c:52
#4  0x00007f9dac9f9d09 in collect_nursery (reason=0x7f9dacbf739f "Nursery full", is_overflow=0) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:1858
#5  0x00007f9dac9f6453 in sgen_perform_collection_inner (requested_size=<optimized out>, generation_to_collect=<optimized out>, reason=<optimized out>, forced_serial=<optimized out>, stw=<optimized out>) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2656
#6  sgen_perform_collection (requested_size=4096, generation_to_collect=0, reason=0x7f9dacbf739f "Nursery full", forced_serial=0, stw=1) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2768
#7  0x00007f9dac9f6253 in sgen_ensure_free_space (size=4096, generation=<optimized out>) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2622
#8  0x00007f9dac9ea417 in sgen_alloc_obj_nolock (vtable=0x55c44f552ce8, size=144) at /__w/1/s/src/mono/mono/sgen/sgen-alloc.c:279
#9  0x00007f9dac9ea8f9 in sgen_alloc_obj (vtable=0x55c44f552ce8, size=144) at /__w/1/s/src/mono/mono/sgen/sgen-alloc.c:454
#10 0x00007f9dac9b9606 in mono_gc_alloc_obj (vtable=0x7f9daccb8cd0 <pin_queue_mutex>, size=128) at /__w/1/s/src/mono/mono/metadata/sgen-mono.c:911
#11 0x00000000411246f8 in ?? ()
#12 0x00007f9dac140c88 in ?? ()
#13 0x00007f9d80000f30 in ?? ()
#14 0x00007f9d800008c0 in ?? ()
#15 0x000055c44f552ce8 in ?? ()
#16 0x0000000000000090 in ?? ()
#17 0x00007f9da183aa48 in ?? ()
#18 0x00007f9dac140d18 in ?? ()
#19 0x00007f9da183a2e0 in ?? ()
#20 0x00007f9dac140b70 in ?? ()
#21 0x00000000410f9d44 in ?? ()
#22 0x00007f9da183a870 in ?? ()
#23 0x00007f9da183a4a0 in ?? ()
#24 0x00007f9d9806a468 in ?? ()
#25 0x00007f9daddf5c88 in ?? ()
#26 0x0000000000000003 in ?? ()
#27 0x0000000000000008 in ?? ()
#28 0x0000000000000003 in ?? ()
#29 0x00000000414c6054 in ?? ()
#30 0x00007f9dac140ba0 in ?? ()
#31 0x0000000000000008 in ?? ()
#32 0x0000000000000008 in ?? ()
#33 0x00000000414c5e70 in ?? ()
#34 0x0000000000000002 in ?? ()
#35 0x0000000000000004 in ?? ()
#36 0x00007f9dac140b70 in ?? ()
#37 0x00007f9da1540c30 in ?? ()
#38 0x00007f9daddf5c88 in ?? ()
#39 0x00000000414c5dc0 in ?? ()
#40 0x0000000000000003 in ?? ()
#41 0x0000000000000008 in ?? ()
#42 0x0000000000000002 in ?? ()
#43 0x00000000412fa2b4 in ?? ()
#44 0x00007f9dac3d24a8 in ?? ()
#45 0x00007f9dac3d2518 in ?? ()
#46 0x00007f9dac3d2518 in ?? ()
#47 0x00000000413cc670 in ?? ()
#48 0x00007f9d81bedda0 in ?? ()
#49 0x0000001500000004 in ?? ()
#50 0x0000000080000000 in ?? ()
#51 0x0000000000000000 in ?? ()

Thread 9 (Thread 0x7f9da1a41700 (LWP 13631)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1a38d00) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:133
#4  0x00007f9dadb3cdbd in __GI___pthread_mutex_lock (mutex=0x7f9daccc29e8 <sgen_gc_mutex>) at ../nptl/pthread_mutex_lock.c:80
#5  0x00007f9dac9f6c79 in mono_os_mutex_lock (mutex=<optimized out>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-os-mutex.h:105
#6  mono_coop_mutex_lock (mutex=<optimized out>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:57
#7  sgen_gc_lock () at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:3939
#8  0x00007f9dac9ba0dc in mono_gc_alloc_vector (vtable=0x55c44f13e268, size=64, max_length=16) at /__w/1/s/src/mono/mono/metadata/sgen-mono.c:1338
#9  0x0000000041115e0b in ?? ()
#10 0x00007f9dac1a60c8 in ?? ()
#11 0x00007f9d8c000f30 in ?? ()
#12 0x0000000000000040 in ?? ()
#13 0x000055c44f13e268 in ?? ()
#14 0x00007f9d8c0008c0 in ?? ()
#15 0x00007f9da1a3b888 in ?? ()
#16 0x00007f9da1a39450 in ?? ()
#17 0x00007f9da1a393a0 in ?? ()
#18 0x0000000000000001 in ?? ()
#19 0x00000000410fa4ec in ?? ()
#20 0x00007f9dac1b4f70 in ?? ()
#21 0x0000000000000010 in ?? ()
#22 0x00007f9dac1b4f70 in ?? ()
#23 0x00007f9dac1a6108 in ?? ()
#24 0x00007f9dac1b4f70 in ?? ()
#25 0x00007f9d84ad55a1 in ?? ()
#26 0x0000000000002767 in ?? ()
#27 0x000000000000001e in ?? ()
#28 0x00007f9da1a39650 in ?? ()
#29 0x0000000000000010 in ?? ()
#30 0x0000000000000010 in ?? ()
#31 0x000000000000000e in ?? ()
#32 0x0000000000000005 in ?? ()
#33 0x000000004111e9c8 in ?? ()
#34 0x00007f9dac1a6058 in ?? ()
#35 0x0000000000000000 in ?? ()

Thread 8 (Thread 0x7f9da1c93700 (LWP 13629)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1c921c0) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac476740 in SystemNative_LowLevelMonitor_TimedWait (monitor=0x7f9d94004db0, timeoutMilliseconds=500) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:192
#5  0x0000000041235c97 in ?? ()
#6  0x00000000000a5485 in ?? ()
#7  0x0000000000000000 in ?? ()

Thread 7 (Thread 0x7f9da2548700 (LWP 13628)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da2547040) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac476740 in SystemNative_LowLevelMonitor_TimedWait (monitor=0x7f9d90039510, timeoutMilliseconds=12000) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:192
#5  0x0000000041235c97 in ?? ()
#6  0x00000000000a4c59 in ?? ()
#7  0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7f9da82bc700 (LWP 13627)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da82bb100) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x7f9d9833eb90) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac145908 in ?? ()
#7  0x00007f9da1ca3920 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9da1ca3950 in ?? ()
#11 0x00007f9da1ca3920 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x00007f9d980f64c0 in ?? ()
#14 0x00007f9da82bb800 in ?? ()
#15 0x00007f9da82bb710 in ?? ()
#16 0x00007f9d98341620 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9da1ca3950 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x00007f9dacbe2d59 in ?? () from /datadisks/disk1/work/B13909DB/p/shared/Microsoft.NETCore.App/7.0.0/libcoreclr.so
#23 0x0000000000000000 in ?? ()

Thread 5 (Thread 0x7f9da8356700 (LWP 13624)):
#0  0x00007f9dadb4351d in read () at ../sysdeps/unix/syscall-template.S:84
#1  0x00007f9dac475a2e in SignalHandlerLoop (arg=0x55c44f79ace0) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_signal.c:321
#2  0x00007f9dadb3a6ba in start_thread (arg=0x7f9da8356700) at pthread_create.c:333
#3  0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 4 (Thread 0x7f9daa28f700 (LWP 13623)):
#0  0x00007f9dadb42827 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x7f9daccac2a0 <finalizer_sem>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem@entry=0x7f9daccac2a0 <finalizer_sem>, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f9dadb428d4 in __new_sem_wait_slow (sem=0x7f9daccac2a0 <finalizer_sem>, abstime=0x0) at sem_waitcommon.c:181
#3  0x00007f9dadb4297a in __new_sem_wait (sem=<optimized out>) at sem_wait.c:29
#4  0x00007f9dac9a9cd6 in mono_os_sem_wait (sem=<optimized out>, flags=MONO_SEM_FLAGS_ALERTABLE) at /__w/1/s/src/mono/mono/mini/../utils/mono-os-semaphore.h:204
#5  mono_coop_sem_wait (sem=<optimized out>, flags=MONO_SEM_FLAGS_ALERTABLE) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-semaphore.h:41
#6  finalizer_thread (unused=<optimized out>) at /__w/1/s/src/mono/mono/metadata/gc.c:876
#7  0x00007f9dac98245a in start_wrapper_internal (start_info=0x0, stack_ptr=<optimized out>) at /__w/1/s/src/mono/mono/metadata/threads.c:1200
#8  0x00007f9dac9822e9 in start_wrapper (data=0x55c44effb890) at /__w/1/s/src/mono/mono/metadata/threads.c:1262
#9  0x00007f9dadb3a6ba in start_thread (arg=0x7f9daa28f700) at pthread_create.c:333
#10 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 3 (Thread 0x7f9daa490700 (LWP 13622)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9daa48f780) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  0x00007f9dacdbf84d in poll () at ../sysdeps/unix/syscall-template.S:84
#4  0x00007f9dacbd813a in ipc_poll_fds (fds=<optimized out>, nfds=1, timeout=4294967295) at /__w/1/s/src/native/eventpipe/ds-ipc-pal-socket.c:470
#5  ds_ipc_poll (poll_handles_data=0x7f9da4002250, poll_handles_data_len=1, timeout_ms=4294967295, callback=0x7f9dacbd7630 <server_warning_callback>) at /__w/1/s/src/native/eventpipe/ds-ipc-pal-socket.c:1082
#6  0x00007f9dacbd5785 in ds_ipc_stream_factory_get_next_available_stream (callback=0x7f9dacbd7630 <server_warning_callback>) at /__w/1/s/src/native/eventpipe/ds-ipc.c:395
#7  0x00007f9dacbd3fe9 in server_thread (data=<optimized out>) at /__w/1/s/src/native/eventpipe/ds-server.c:127
#8  0x00007f9dacbd7611 in ep_rt_thread_mono_start_func (data=0x55c44efee5c0) at /__w/1/s/src/mono/mono/mini/../eventpipe/ep-rt-mono.h:1266
#9  0x00007f9dadb3a6ba in start_thread (arg=0x7f9daa490700) at pthread_create.c:333
#10 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 2 (Thread 0x7f9dabfff700 (LWP 13621)):
#0  0x00007f9dadb43f7b in __waitpid (pid=13685, stat_loc=0x7f9dabffdb40, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00007f9dacb3c6e7 in dump_native_stacktrace (signal=<optimized out>, mctx=<optimized out>) at /__w/1/s/src/mono/mono/mini/mini-posix.c:842
#2  mono_dump_native_crash_info (signal=<optimized out>, mctx=0x7f9dabffe6b0, info=<optimized out>) at /__w/1/s/src/mono/mono/mini/mini-posix.c:869
#3  0x00007f9dacae1eee in mono_handle_native_crash (signal=0x7f9dacbf5d7a "SIGSEGV", mctx=0x7f9dabffe6b0, info=0x7f9dabffe970) at /__w/1/s/src/mono/mono/mini/mini-exceptions.c:2940
#4  0x00007f9daca45d96 in mono_sigsegv_signal_handler_debug (_dummy=11, _info=0x7f9dabffe970, context=0x7f9dabffe840, debug_fault_addr=0x7f9d00000000) at /__w/1/s/src/mono/mono/mini/mini-runtime.c:3663
#5  <signal handler called>
#6  major_copy_or_mark_object_concurrent_with_evacuation (ptr=<optimized out>, obj=<optimized out>, queue=0x7f9dadf71010) at /__w/1/s/src/mono/mono/mini/../sgen/sgen-marksweep-drain-gray-stack.h:137
#7  major_scan_object_concurrent_with_evacuation (full_object=0x7f9dac1b4e00, desc=<optimized out>, queue=<optimized out>) at /__w/1/s/src/mono/mono/mini/../sgen/sgen-scan-object.h:64
#8  0x00007f9daca21d7b in sgen_scan_pin_queue_objects (ctx=...) at /__w/1/s/src/mono/mono/sgen/sgen-pinning.c:84
#9  0x00007f9daca2d971 in thread_func (data=0x0) at /__w/1/s/src/mono/mono/sgen/sgen-thread-pool.c:209
#10 0x00007f9dadb3a6ba in start_thread (arg=0x7f9dabfff700) at pthread_create.c:333
#11 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 1 (Thread 0x7f9dadf63740 (LWP 13620)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7ffd3b805500) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x55c44f9dc790) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac141658 in ?? ()
#7  0x00007f9da22cba10 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9da22cba40 in ?? ()
#11 0x00007f9da22cba10 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x000055c44efe1410 in ?? ()
#14 0x00007ffd3b805bd0 in ?? ()
#15 0x00007ffd3b805ae0 in ?? ()
#16 0x000055c44ef82bf0 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9da22cba40 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x000055c44ef83290 in ?? ()
#23 0x0000000000000000 in ?? ()

=================================================================
	Basic Fault Address Reporting
=================================================================
Memory around native instruction pointer (0x7f9daca09e05):0x7f9daca09df5  84 00 00 00 00 00 0f 1f 44 00 00 41 0f 18 5d 00  ........D..A..].
0x7f9daca09e05  49 8b 45 00 48 89 c7 48 83 e7 f8 48 8b 6f 08 40  I.E.H..H...H.o.@
0x7f9daca09e15  f6 c5 04 74 2c a8 01 74 0c 48 85 ff 74 07 48 8b  ...t,..t.H..t.H.
0x7f9daca09e25  3f 48 83 e7 f8 4c 89 ee e8 fe 22 01 00 48 83 c0  ?H...L...."..H..
./RunTests.sh: line 162: 13620 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/datadisks/disk1/work/B13909DB/w/A4760949/e
----- end Thu Sep 2 16:31:16 UTC 2021 ----- exit code 134 ----------------------------------------------------------
Author: ViktorHofer
Assignees: -
Labels:

area-System.Text.Json, untriaged

Milestone: -

@danmoseley
Copy link
Member

@BrzVlad any clue where this should go? I can't see evidence it's JSON (which is all managed code anyway) . But I can't tell which of the threads above is the crash.

@BrzVlad BrzVlad self-assigned this Sep 10, 2021
@ghost
Copy link

ghost commented Sep 10, 2021

Tagging subscribers to this area: @BrzVlad
See info in area-owners.md if you want to be subscribed.

Issue Details

Configuration: net6.0-Linux-Debug-x64-Mono_release-Ubuntu.1604.Amd64.Open

Dump available: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-58558-merge-1c1e3bdb886344679d/System.Text.Json.Tests/1/core.1000.13620?sv=2019-07-07&se=2021-09-22T16%3A19%3A20Z&sr=c&sp=rl&sig=sjEJQ%2BswuvmnAmzLahc2AgFLgot5gYVvnZSow7cL6Jg%3D

https://dev.azure.com/dnceng/public/_build/results?buildId=1337984&view=ms.vss-test-web.build-test-results-tab&runId=39211016&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab&resultId=187646

pushd .
/datadisks/disk1/work/B13909DB/p/dotnet exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/datadisks/disk1/work/B13909DB/w/A4760949/e /datadisks/disk1/work/B13909DB/w/A4760949/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 2770 of 2809 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 2)

=================================================================
	Native Crash Reporting
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

=================================================================
	Native stacktrace:
=================================================================
	0x7f9dacb3c5a2 - Unknown
	0x7f9dacae1eee - Unknown
	0x7f9daca45d96 - Unknown
	0x7f9dadb44390 - Unknown
	0x7f9daca09e05 - Unknown
	0x7f9daca21d7b - Unknown
	0x7f9daca2d971 - Unknown
	0x7f9dadb3a6ba - Unknown
	0x7f9dacdcb51d - Unknown

=================================================================
	External Debugger Dump:
=================================================================
[New LWP 13621]
[New LWP 13622]
[New LWP 13623]
[New LWP 13624]
[New LWP 13627]
[New LWP 13628]
[New LWP 13629]
[New LWP 13631]
[New LWP 13632]
[New LWP 13638]
[New LWP 13655]
[New LWP 13671]
[New LWP 13674]
[New LWP 13675]
[New LWP 13676]
[New LWP 13677]
[New LWP 13678]
[New LWP 13679]
[New LWP 13680]
[New LWP 13681]
[New LWP 13682]
[New LWP 13683]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
30	../sysdeps/unix/sysv/linux/sigsuspend.c: No such file or directory.
  Id   Target Id         Frame 
* 1    Thread 0x7f9dadf63740 (LWP 13620) "dotnet" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  2    Thread 0x7f9dabfff700 (LWP 13621) "SGen worker" 0x00007f9dadb43f7b in __waitpid (pid=13685, stat_loc=0x7f9dabffdb40, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
  3    Thread 0x7f9daa490700 (LWP 13622) "dotnet" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  4    Thread 0x7f9daa28f700 (LWP 13623) "Finalizer" 0x00007f9dadb42827 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x7f9daccac2a0 <finalizer_sem>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
  5    Thread 0x7f9da8356700 (LWP 13624) "dotnet" 0x00007f9dadb4351d in read () at ../sysdeps/unix/syscall-template.S:84
  6    Thread 0x7f9da82bc700 (LWP 13627) ".NET Long Runni" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  7    Thread 0x7f9da2548700 (LWP 13628) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  8    Thread 0x7f9da1c93700 (LWP 13629) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  9    Thread 0x7f9da1a41700 (LWP 13631) ".NET Long Runni" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  10   Thread 0x7f9da1840700 (LWP 13632) ".NET Long Runni" __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
  11   Thread 0x7f9d85aff700 (LWP 13638) ".NET Timers" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  12   Thread 0x7f9d789f9700 (LWP 13655) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  13   Thread 0x7f9da1c52700 (LWP 13671) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  14   Thread 0x7f9d7b77b700 (LWP 13674) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  15   Thread 0x7f9d7b3ff700 (LWP 13675) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  16   Thread 0x7f9d7a8ff700 (LWP 13676) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  17   Thread 0x7f9d7a6fe700 (LWP 13677) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  18   Thread 0x7f9d7a05b700 (LWP 13678) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  19   Thread 0x7f9d7977b700 (LWP 13679) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  20   Thread 0x7f9d78dff700 (LWP 13680) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  21   Thread 0x7f9d78bfe700 (LWP 13681) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  22   Thread 0x7f9d786f7700 (LWP 13682) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
  23   Thread 0x7f9d784f6700 (LWP 13683) ".NET ThreadPool" 0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30

Thread 23 (Thread 0x7f9d784f6700 (LWP 13683)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d784f5280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d784f58d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d784f58d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d5401b350 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d784f5940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 22 (Thread 0x7f9d786f7700 (LWP 13682)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d786f6280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d786f68d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d786f68d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 21 (Thread 0x7f9d78bfe700 (LWP 13681)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d78bfd280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d78bfd8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d78bfd8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 20 (Thread 0x7f9d78dff700 (LWP 13680)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d78dfe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d78dfe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d78dfe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d4c007f20 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d78dfe940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 19 (Thread 0x7f9d7977b700 (LWP 13679)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7977a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7977a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7977a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x000055c44f89e330 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d7977a940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 18 (Thread 0x7f9d7a05b700 (LWP 13678)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a05a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a05a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a05a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 17 (Thread 0x7f9d7a6fe700 (LWP 13677)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a6fd280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a6fd8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a6fd8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 16 (Thread 0x7f9d7a8ff700 (LWP 13676)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7a8fe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7a8fe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7a8fe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 15 (Thread 0x7f9d7b3ff700 (LWP 13675)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7b3fe280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7b3fe8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7b3fe8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000002 in ?? ()
#9  0x0000000000000046 in ?? ()
#10 0x00007f9dac14ada8 in ?? ()
#11 0x00007f9dac14ada8 in ?? ()
#12 0x0000000000004e20 in ?? ()
#13 0x00007f9d60000fe0 in ?? ()
#14 0x0000000000000046 in ?? ()
#15 0x00007f9d7b3fe940 in ?? ()
#16 0x0000000000000000 in ?? ()

Thread 14 (Thread 0x7f9d7b77b700 (LWP 13674)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d7b77a280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d7b77a8d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d7b77a8d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 13 (Thread 0x7f9da1c52700 (LWP 13671)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1c51280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9da1c518d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9da1c518d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 12 (Thread 0x7f9d789f9700 (LWP 13655)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d789f8280) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac9d6c44 in mono_os_cond_timedwait (cond=0x7f9d789f88d0, mutex=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/mono-os-mutex.c:75
#5  0x00007f9dac9ddf44 in mono_coop_cond_timedwait (cond=0x7f9d789f88d0, mutex=<optimized out>, timeout_ms=20000) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:103
#6  mono_lifo_semaphore_timed_wait (semaphore=0x7f9d9836c950, timeout_ms=20000) at /__w/1/s/src/mono/mono/utils/lifo-semaphore.c:48
#7  0x000000004123f4e7 in ?? ()
#8  0x0000000000000000 in ?? ()

Thread 11 (Thread 0x7f9d85aff700 (LWP 13638)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9d85afe240) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x7f9d7c00ca40) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac171bf0 in ?? ()
#7  0x00007f9dac174980 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9dac1749b0 in ?? ()
#11 0x00007f9dac174980 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x00007f9d7c000fe0 in ?? ()
#14 0x00007f9d85afe940 in ?? ()
#15 0x00007f9d85afe850 in ?? ()
#16 0x00007f9d7c0008c0 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9dac1749b0 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x00007f9d7c001050 in ?? ()
#23 0x0000000000000000 in ?? ()

Thread 10 (Thread 0x7f9da1840700 (LWP 13632)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007f9dadb3cdbd in __GI___pthread_mutex_lock (mutex=0x7f9daccb8cd0 <pin_queue_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2  0x00007f9daca21c5d in mono_os_mutex_lock (mutex=0x7f9daccb8cd0 <pin_queue_mutex>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-os-mutex.h:105
#3  sgen_init_pinning_for_conc () at /__w/1/s/src/mono/mono/sgen/sgen-pinning.c:52
#4  0x00007f9dac9f9d09 in collect_nursery (reason=0x7f9dacbf739f "Nursery full", is_overflow=0) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:1858
#5  0x00007f9dac9f6453 in sgen_perform_collection_inner (requested_size=<optimized out>, generation_to_collect=<optimized out>, reason=<optimized out>, forced_serial=<optimized out>, stw=<optimized out>) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2656
#6  sgen_perform_collection (requested_size=4096, generation_to_collect=0, reason=0x7f9dacbf739f "Nursery full", forced_serial=0, stw=1) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2768
#7  0x00007f9dac9f6253 in sgen_ensure_free_space (size=4096, generation=<optimized out>) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:2622
#8  0x00007f9dac9ea417 in sgen_alloc_obj_nolock (vtable=0x55c44f552ce8, size=144) at /__w/1/s/src/mono/mono/sgen/sgen-alloc.c:279
#9  0x00007f9dac9ea8f9 in sgen_alloc_obj (vtable=0x55c44f552ce8, size=144) at /__w/1/s/src/mono/mono/sgen/sgen-alloc.c:454
#10 0x00007f9dac9b9606 in mono_gc_alloc_obj (vtable=0x7f9daccb8cd0 <pin_queue_mutex>, size=128) at /__w/1/s/src/mono/mono/metadata/sgen-mono.c:911
#11 0x00000000411246f8 in ?? ()
#12 0x00007f9dac140c88 in ?? ()
#13 0x00007f9d80000f30 in ?? ()
#14 0x00007f9d800008c0 in ?? ()
#15 0x000055c44f552ce8 in ?? ()
#16 0x0000000000000090 in ?? ()
#17 0x00007f9da183aa48 in ?? ()
#18 0x00007f9dac140d18 in ?? ()
#19 0x00007f9da183a2e0 in ?? ()
#20 0x00007f9dac140b70 in ?? ()
#21 0x00000000410f9d44 in ?? ()
#22 0x00007f9da183a870 in ?? ()
#23 0x00007f9da183a4a0 in ?? ()
#24 0x00007f9d9806a468 in ?? ()
#25 0x00007f9daddf5c88 in ?? ()
#26 0x0000000000000003 in ?? ()
#27 0x0000000000000008 in ?? ()
#28 0x0000000000000003 in ?? ()
#29 0x00000000414c6054 in ?? ()
#30 0x00007f9dac140ba0 in ?? ()
#31 0x0000000000000008 in ?? ()
#32 0x0000000000000008 in ?? ()
#33 0x00000000414c5e70 in ?? ()
#34 0x0000000000000002 in ?? ()
#35 0x0000000000000004 in ?? ()
#36 0x00007f9dac140b70 in ?? ()
#37 0x00007f9da1540c30 in ?? ()
#38 0x00007f9daddf5c88 in ?? ()
#39 0x00000000414c5dc0 in ?? ()
#40 0x0000000000000003 in ?? ()
#41 0x0000000000000008 in ?? ()
#42 0x0000000000000002 in ?? ()
#43 0x00000000412fa2b4 in ?? ()
#44 0x00007f9dac3d24a8 in ?? ()
#45 0x00007f9dac3d2518 in ?? ()
#46 0x00007f9dac3d2518 in ?? ()
#47 0x00000000413cc670 in ?? ()
#48 0x00007f9d81bedda0 in ?? ()
#49 0x0000001500000004 in ?? ()
#50 0x0000000080000000 in ?? ()
#51 0x0000000000000000 in ?? ()

Thread 9 (Thread 0x7f9da1a41700 (LWP 13631)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1a38d00) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:133
#4  0x00007f9dadb3cdbd in __GI___pthread_mutex_lock (mutex=0x7f9daccc29e8 <sgen_gc_mutex>) at ../nptl/pthread_mutex_lock.c:80
#5  0x00007f9dac9f6c79 in mono_os_mutex_lock (mutex=<optimized out>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-os-mutex.h:105
#6  mono_coop_mutex_lock (mutex=<optimized out>) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-mutex.h:57
#7  sgen_gc_lock () at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:3939
#8  0x00007f9dac9ba0dc in mono_gc_alloc_vector (vtable=0x55c44f13e268, size=64, max_length=16) at /__w/1/s/src/mono/mono/metadata/sgen-mono.c:1338
#9  0x0000000041115e0b in ?? ()
#10 0x00007f9dac1a60c8 in ?? ()
#11 0x00007f9d8c000f30 in ?? ()
#12 0x0000000000000040 in ?? ()
#13 0x000055c44f13e268 in ?? ()
#14 0x00007f9d8c0008c0 in ?? ()
#15 0x00007f9da1a3b888 in ?? ()
#16 0x00007f9da1a39450 in ?? ()
#17 0x00007f9da1a393a0 in ?? ()
#18 0x0000000000000001 in ?? ()
#19 0x00000000410fa4ec in ?? ()
#20 0x00007f9dac1b4f70 in ?? ()
#21 0x0000000000000010 in ?? ()
#22 0x00007f9dac1b4f70 in ?? ()
#23 0x00007f9dac1a6108 in ?? ()
#24 0x00007f9dac1b4f70 in ?? ()
#25 0x00007f9d84ad55a1 in ?? ()
#26 0x0000000000002767 in ?? ()
#27 0x000000000000001e in ?? ()
#28 0x00007f9da1a39650 in ?? ()
#29 0x0000000000000010 in ?? ()
#30 0x0000000000000010 in ?? ()
#31 0x000000000000000e in ?? ()
#32 0x0000000000000005 in ?? ()
#33 0x000000004111e9c8 in ?? ()
#34 0x00007f9dac1a6058 in ?? ()
#35 0x0000000000000000 in ?? ()

Thread 8 (Thread 0x7f9da1c93700 (LWP 13629)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da1c921c0) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac476740 in SystemNative_LowLevelMonitor_TimedWait (monitor=0x7f9d94004db0, timeoutMilliseconds=500) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:192
#5  0x0000000041235c97 in ?? ()
#6  0x00000000000a5485 in ?? ()
#7  0x0000000000000000 in ?? ()

Thread 7 (Thread 0x7f9da2548700 (LWP 13628)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da2547040) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#4  0x00007f9dac476740 in SystemNative_LowLevelMonitor_TimedWait (monitor=0x7f9d90039510, timeoutMilliseconds=12000) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:192
#5  0x0000000041235c97 in ?? ()
#6  0x00000000000a4c59 in ?? ()
#7  0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7f9da82bc700 (LWP 13627)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9da82bb100) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x7f9d9833eb90) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac145908 in ?? ()
#7  0x00007f9da1ca3920 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9da1ca3950 in ?? ()
#11 0x00007f9da1ca3920 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x00007f9d980f64c0 in ?? ()
#14 0x00007f9da82bb800 in ?? ()
#15 0x00007f9da82bb710 in ?? ()
#16 0x00007f9d98341620 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9da1ca3950 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x00007f9dacbe2d59 in ?? () from /datadisks/disk1/work/B13909DB/p/shared/Microsoft.NETCore.App/7.0.0/libcoreclr.so
#23 0x0000000000000000 in ?? ()

Thread 5 (Thread 0x7f9da8356700 (LWP 13624)):
#0  0x00007f9dadb4351d in read () at ../sysdeps/unix/syscall-template.S:84
#1  0x00007f9dac475a2e in SignalHandlerLoop (arg=0x55c44f79ace0) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_signal.c:321
#2  0x00007f9dadb3a6ba in start_thread (arg=0x7f9da8356700) at pthread_create.c:333
#3  0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 4 (Thread 0x7f9daa28f700 (LWP 13623)):
#0  0x00007f9dadb42827 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x7f9daccac2a0 <finalizer_sem>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem@entry=0x7f9daccac2a0 <finalizer_sem>, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f9dadb428d4 in __new_sem_wait_slow (sem=0x7f9daccac2a0 <finalizer_sem>, abstime=0x0) at sem_waitcommon.c:181
#3  0x00007f9dadb4297a in __new_sem_wait (sem=<optimized out>) at sem_wait.c:29
#4  0x00007f9dac9a9cd6 in mono_os_sem_wait (sem=<optimized out>, flags=MONO_SEM_FLAGS_ALERTABLE) at /__w/1/s/src/mono/mono/mini/../utils/mono-os-semaphore.h:204
#5  mono_coop_sem_wait (sem=<optimized out>, flags=MONO_SEM_FLAGS_ALERTABLE) at /__w/1/s/src/mono/mono/mini/../../mono/utils/mono-coop-semaphore.h:41
#6  finalizer_thread (unused=<optimized out>) at /__w/1/s/src/mono/mono/metadata/gc.c:876
#7  0x00007f9dac98245a in start_wrapper_internal (start_info=0x0, stack_ptr=<optimized out>) at /__w/1/s/src/mono/mono/metadata/threads.c:1200
#8  0x00007f9dac9822e9 in start_wrapper (data=0x55c44effb890) at /__w/1/s/src/mono/mono/metadata/threads.c:1262
#9  0x00007f9dadb3a6ba in start_thread (arg=0x7f9daa28f700) at pthread_create.c:333
#10 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 3 (Thread 0x7f9daa490700 (LWP 13622)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7f9daa48f780) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  0x00007f9dacdbf84d in poll () at ../sysdeps/unix/syscall-template.S:84
#4  0x00007f9dacbd813a in ipc_poll_fds (fds=<optimized out>, nfds=1, timeout=4294967295) at /__w/1/s/src/native/eventpipe/ds-ipc-pal-socket.c:470
#5  ds_ipc_poll (poll_handles_data=0x7f9da4002250, poll_handles_data_len=1, timeout_ms=4294967295, callback=0x7f9dacbd7630 <server_warning_callback>) at /__w/1/s/src/native/eventpipe/ds-ipc-pal-socket.c:1082
#6  0x00007f9dacbd5785 in ds_ipc_stream_factory_get_next_available_stream (callback=0x7f9dacbd7630 <server_warning_callback>) at /__w/1/s/src/native/eventpipe/ds-ipc.c:395
#7  0x00007f9dacbd3fe9 in server_thread (data=<optimized out>) at /__w/1/s/src/native/eventpipe/ds-server.c:127
#8  0x00007f9dacbd7611 in ep_rt_thread_mono_start_func (data=0x55c44efee5c0) at /__w/1/s/src/mono/mono/mini/../eventpipe/ep-rt-mono.h:1266
#9  0x00007f9dadb3a6ba in start_thread (arg=0x7f9daa490700) at pthread_create.c:333
#10 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 2 (Thread 0x7f9dabfff700 (LWP 13621)):
#0  0x00007f9dadb43f7b in __waitpid (pid=13685, stat_loc=0x7f9dabffdb40, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00007f9dacb3c6e7 in dump_native_stacktrace (signal=<optimized out>, mctx=<optimized out>) at /__w/1/s/src/mono/mono/mini/mini-posix.c:842
#2  mono_dump_native_crash_info (signal=<optimized out>, mctx=0x7f9dabffe6b0, info=<optimized out>) at /__w/1/s/src/mono/mono/mini/mini-posix.c:869
#3  0x00007f9dacae1eee in mono_handle_native_crash (signal=0x7f9dacbf5d7a "SIGSEGV", mctx=0x7f9dabffe6b0, info=0x7f9dabffe970) at /__w/1/s/src/mono/mono/mini/mini-exceptions.c:2940
#4  0x00007f9daca45d96 in mono_sigsegv_signal_handler_debug (_dummy=11, _info=0x7f9dabffe970, context=0x7f9dabffe840, debug_fault_addr=0x7f9d00000000) at /__w/1/s/src/mono/mono/mini/mini-runtime.c:3663
#5  <signal handler called>
#6  major_copy_or_mark_object_concurrent_with_evacuation (ptr=<optimized out>, obj=<optimized out>, queue=0x7f9dadf71010) at /__w/1/s/src/mono/mono/mini/../sgen/sgen-marksweep-drain-gray-stack.h:137
#7  major_scan_object_concurrent_with_evacuation (full_object=0x7f9dac1b4e00, desc=<optimized out>, queue=<optimized out>) at /__w/1/s/src/mono/mono/mini/../sgen/sgen-scan-object.h:64
#8  0x00007f9daca21d7b in sgen_scan_pin_queue_objects (ctx=...) at /__w/1/s/src/mono/mono/sgen/sgen-pinning.c:84
#9  0x00007f9daca2d971 in thread_func (data=0x0) at /__w/1/s/src/mono/mono/sgen/sgen-thread-pool.c:209
#10 0x00007f9dadb3a6ba in start_thread (arg=0x7f9dabfff700) at pthread_create.c:333
#11 0x00007f9dacdcb51d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 1 (Thread 0x7f9dadf63740 (LWP 13620)):
#0  0x00007f9daccf9836 in __GI___sigsuspend (set=0x7f9daccb1488 <suspend_signal_mask>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:30
#1  0x00007f9dac9e5a02 in suspend_signal_handler (_dummy=<optimized out>, info=<optimized out>, context=0x7ffd3b805500) at /__w/1/s/src/mono/mono/utils/mono-threads-posix-signals.c:200
#2  <signal handler called>
#3  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#4  0x00007f9dac4765fc in SystemNative_LowLevelMonitor_Wait (monitor=0x55c44f9dc790) at /__w/1/s/src/libraries/Native/Unix/System.Native/pal_threading.c:152
#5  0x00000000411ddc7c in ?? ()
#6  0x00007f9dac141658 in ?? ()
#7  0x00007f9da22cba10 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x0000000000000001 in ?? ()
#10 0x00007f9da22cba40 in ?? ()
#11 0x00007f9da22cba10 in ?? ()
#12 0xffffffffffffffff in ?? ()
#13 0x000055c44efe1410 in ?? ()
#14 0x00007ffd3b805bd0 in ?? ()
#15 0x00007ffd3b805ae0 in ?? ()
#16 0x000055c44ef82bf0 in ?? ()
#17 0x00000000411ddbbc in ?? ()
#18 0xffffffffffffffff in ?? ()
#19 0x00000000411ddb7c in ?? ()
#20 0x00007f9da22cba40 in ?? ()
#21 0x00000000411dd258 in ?? ()
#22 0x000055c44ef83290 in ?? ()
#23 0x0000000000000000 in ?? ()

=================================================================
	Basic Fault Address Reporting
=================================================================
Memory around native instruction pointer (0x7f9daca09e05):0x7f9daca09df5  84 00 00 00 00 00 0f 1f 44 00 00 41 0f 18 5d 00  ........D..A..].
0x7f9daca09e05  49 8b 45 00 48 89 c7 48 83 e7 f8 48 8b 6f 08 40  I.E.H..H...H.o.@
0x7f9daca09e15  f6 c5 04 74 2c a8 01 74 0c 48 85 ff 74 07 48 8b  ...t,..t.H..t.H.
0x7f9daca09e25  3f 48 83 e7 f8 4c 89 ee e8 fe 22 01 00 48 83 c0  ?H...L...."..H..
./RunTests.sh: line 162: 13620 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Text.Json.Tests.runtimeconfig.json --depsfile System.Text.Json.Tests.deps.json xunit.console.dll System.Text.Json.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/datadisks/disk1/work/B13909DB/w/A4760949/e
----- end Thu Sep 2 16:31:16 UTC 2021 ----- exit code 134 ----------------------------------------------------------
Author: ViktorHofer
Assignees: BrzVlad
Labels:

area-System.Text.Json, untriaged, area-GC-mono

Milestone: -

@BrzVlad BrzVlad removed area-System.Text.Json untriaged New issue has not been triaged by the area owner labels Sep 10, 2021
@SamMonoRT SamMonoRT added this to the 7.0.0 milestone Sep 10, 2021
@lambdageek
Copy link
Member

@BrzVlad hmm... I wonder if I introduced a GC hole here: https://github.com/dotnet/runtime/pull/58215/files#diff-ca87147826c7221d1c277d7a22d042444f21defad5b7a901d771c158243d6e14R3428-R3435

For a runtime-invoke with a large valuetype (>256 bytes) return value, it's allocating the return buffer using malloc instead of writing to the stack. Which means if there's a GC between when the invoke returns and the return valuetype is boxed, the GC won't see any references in it.

@BrzVlad
Copy link
Member

BrzVlad commented Nov 3, 2021

@danmoseley The issue is not related to that. There is some GC issue but the stack trace is not enough to figure out what is going on. I tried to reproduce the issue the last time but I failed. I'll try again to reproduce by running this test in a loop for a very long amount of time.

@danmoseley
Copy link
Member

Thanks @BrzVlad .. the dump is not helpful here?

@radical
Copy link
Member

radical commented Mar 28, 2022

Hitting this again - https://dev.azure.com/dnceng/public/_build/results?buildId=1684019&view=logs&jobId=b4344b0d-0f92-5d69-ccaf-e0b24fbf14a2&j=b4344b0d-0f92-5d69-ccaf-e0b24fbf14a2&t=65a58faa-a542-5344-b5c8-897139f242cd with similar looking trace:

Thread 2 (Thread 0x7f2a84fff700 (LWP 26)):
#0  0x00007f2a866230ca in __waitpid (pid=44, stat_loc=0x7f2a84ffda40, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#1  0x00007f2a85f9bf97 in dump_native_stacktrace (signal=<optimized out>, mctx=<optimized out>) at /__w/1/s/src/mono/mono/mini/mini-posix.c:843
#2  mono_dump_native_crash_info (signal=<optimized out>, mctx=0x7f2a84ffe5b0, info=<optimized out>) at /__w/1/s/src/mono/mono/mini/mini-posix.c:870
#3  0x00007f2a85f414ce in mono_handle_native_crash (signal=0x7f2a85cbfbf6 "SIGSEGV", mctx=0x7f2a84ffe5b0, info=0x7f2a84ffe870) at /__w/1/s/src/mono/mono/mini/mini-exceptions.c:3016
#4  0x00007f2a85ea2156 in mono_sigsegv_signal_handler_debug (_dummy=11, _info=0x7f2a84ffe870, context=0x7f2a84ffe740, debug_fault_addr=0x7f2a00000000) at /__w/1/s/src/mono/mono/mini/mini-runtime.c:3724
#5  <signal handler called>
#6  0x00007f2a85e65dd1 in major_scan_object_concurrent_with_evacuation (full_object=0x7f2a4b99a330, desc=<optimized out>, queue=<optimized out>) at /__w/1/s/src/mono/mono/mini/../sgen/sgen-scan-object.h:64
#7  0x00007f2a85e7983c in scan_card_table_for_block (block=0x7f2a4b998000, scan_type=<optimized out>, ctx=...) at /__w/1/s/src/mono/mono/sgen/sgen-marksweep.c:2667
#8  0x00007f2a85e5ede7 in major_scan_card_table (scan_type=CARDTABLE_SCAN_MOD_UNION_PRECLEAN, ctx=..., job_index=<optimized out>, job_split_count=<optimized out>, block_count=<optimized out>) at /__w/1/s/src/mono/mono/sgen/sgen-marksweep.c:2745
#9  0x00007f2a85e5894a in job_major_mod_union_preclean (worker_data_untyped=0x7f2a86636008, job=0x7f2a7a63f008) at /__w/1/s/src/mono/mono/sgen/sgen-gc.c:1590
#10 0x00007f2a85e899d1 in thread_func (data=0x0) at /__w/1/s/src/mono/mono/sgen/sgen-thread-pool.c:209
#11 0x00007f2a86618fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#12 0x00007f2a862234cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

@BrzVlad
Copy link
Member

BrzVlad commented Aug 3, 2022

@radical Is there a way to see how often this crash happened in the last months and for which suites ? Crashes containing major_scan_object_concurrent_with_evacuation

@radical
Copy link
Member

radical commented Aug 3, 2022

https://runfo.azurewebsites.net can be used for this. But I'm not being able to get it to search the helix console logs.
@akoeplinger could you please help with this?

@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Aug 5, 2022
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Aug 8, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Sep 7, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants