System.Threading.Tasks.Parallel.Tests are timing out on s390x

[akoeplinger]

Recent runs of the runtime-community pipeline are timing out because of a long running test that is likely hanging:

  Discovering: System.Threading.Tasks.Parallel.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Threading.Tasks.Parallel.Tests (found 118 of 197 test cases)
  Starting:    System.Threading.Tasks.Parallel.Tests (parallel test collections = on [2 threads], stop on fail = off)
   System.Threading.Tasks.Parallel.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.ParallelForEachAsyncTests.AllItemsEnumeratedOnce_For', Elapsed: 00:02:11
   System.Threading.Tasks.Parallel.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.ParallelForEachAsyncTests.AllItemsEnumeratedOnce_For', Elapsed: 00:04:11
   System.Threading.Tasks.Parallel.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.ParallelForEachAsyncTests.AllItemsEnumeratedOnce_For', Elapsed: 00:06:11
   System.Threading.Tasks.Parallel.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.ParallelForEachAsyncTests.AllItemsEnumeratedOnce_For', Elapsed: 00:08:11
   System.Threading.Tasks.Parallel.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.ParallelForEachAsyncTests.AllItemsEnumeratedOnce_For', Elapsed: 00:10:11
   System.Threading.Tasks.Parallel.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.ParallelForEachAsyncTests.AllItemsEnumeratedOnce_For', Elapsed: 00:12:11
   System.Threading.Tasks.Parallel.Tests: [Long Running Test] 'System.Threading.Tasks.Tests.ParallelForEachAsyncTests.AllItemsEnumeratedOnce_For', Elapsed: 00:14:11
['System.Threading.Tasks.Parallel.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]

/cc @uweigand

[akoeplinger]

Interesting, the same run also failed System.Threading.Tests but for some reason the Helix workitem didn't fail...

  Discovering: System.Threading.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Threading.Tests (found 263 of 297 test cases)
  Starting:    System.Threading.Tests (parallel test collections = on [2 threads], stop on fail = off)
    System.Threading.Tests.InterlockedTests.InterlockedExchange_Int16 [FAIL]
      Assert.Equal() Failure: Values differ
      Expected: 42
      Actual:   0
      Stack Trace:
        /_/src/libraries/System.Threading/tests/InterlockedTests.cs(171,0): at System.Threading.Tests.InterlockedTests.InterlockedExchange_Int16()
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.Mono.cs(22,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
    System.Threading.Tests.InterlockedTests.InterlockedCompareExchange_UInt16 [FAIL]
      Assert.Equal() Failure: Values differ
      Expected: 42
      Actual:   0
      Stack Trace:
        /_/src/libraries/System.Threading/tests/InterlockedTests.cs(369,0): at System.Threading.Tests.InterlockedTests.InterlockedCompareExchange_UInt16()
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.Mono.cs(22,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
    System.Threading.Tests.InterlockedTests.InterlockedExchange_UInt16 [FAIL]
      Assert.Equal() Failure: Values differ
      Expected: 42
      Actual:   0
      Stack Trace:
        /_/src/libraries/System.Threading/tests/InterlockedTests.cs(182,0): at System.Threading.Tests.InterlockedTests.InterlockedExchange_UInt16()
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.Mono.cs(22,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
    System.Threading.Tests.InterlockedTests.InterlockedCompareExchange_Int16 [FAIL]
      Assert.Equal() Failure: Values differ
      Expected: 42
      Actual:   0
      Stack Trace:
        /_/src/libraries/System.Threading/tests/InterlockedTests.cs(353,0): at System.Threading.Tests.InterlockedTests.InterlockedCompareExchange_Int16()
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.Mono.cs(22,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
    System.Threading.Tests.ThreadLocalTests.RunThreadLocalTest8_Values_NegativeCases [SKIP]
      Condition(s) not met: "IsPreciseGcSupported"
    System.Threading.Tests.ThreadLocalTests.RunThreadLocalTest5_Dispose [SKIP]
      Condition(s) not met: "IsThreadingAndPreciseGcSupported"
    System.Threading.Tests.ThreadLocalTests.RunThreadLocalTest7_WeakReference [SKIP]
      Condition(s) not met: "IsPreciseGcSupported"
  Finished:    System.Threading.Tests
=== TEST EXECUTION SUMMARY ===
   System.Threading.Tests  Total: 531, Errors: 0, Failed: 4, Skipped: 3, Time: 10.432s

[uweigand]

We've seen the timeouts on the CI machine, and @giritrivedi has been trying to reproduce them. Unfortunately, when building and running the tests locally, those timeouts do not occur ... Giridhar is currently trying to see if using cross-built test assemblies makes any difference.

Thanks for pointing out the SystemThreading.Tests failures, we'll look into those.

[akoeplinger]

I disabled the AllItemsEnumeratedOnce_For test in #101389

[uweigand]

Thanks for pointing out the SystemThreading.Tests failures, we'll look into those.

Looks like #99011 introduced a regression on big-endian hosts:

int bitOffset =
    (int)((BitConverter.IsLittleEndian ? offset : sizeof(uint) - offset - sizeof(byte)) * 8); // to bit offset

This computation is used for both 8-bit and 16-bit fallbacks - but it is incorrect for 16-bit (should be sizeof(short) instead of sizeof(byte)).

[stephentoub]

cc: @MichalPetryka