Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Linux] mixed Interpreter and JIT compiler #13759

Closed
xiangzhai opened this issue Nov 8, 2019 · 13 comments · Fixed by #37825
Closed

[Linux] mixed Interpreter and JIT compiler #13759

xiangzhai opened this issue Nov 8, 2019 · 13 comments · Fixed by #37825
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Milestone

Comments

@xiangzhai
Copy link
Contributor

Hi,

OpenJDK is able to just enable interpreter:

java -Xint

or just enable JIT compilers:

java -Xcomp

or mixed:

java -Xmixed

And what about CoreCLR?

  1. enable Interpreter build Enable interpreter build coreclr#11252
./build.sh debug cmakeargs "-DFEATURE_INTERPRETER=1" skipcrossgen ignorewarnings
  1. HelloWorld testcase for mixing Interpreter and JIT compiler
  • X86
export CORE_LIBRARIES=~/dotnet-sdk-3.0.100-linux-x64/shared/Microsoft.NETCore.App/3.0.0

# interpreter
export COMPlus_DumpInterpreterStubs=1
export COMPlus_Interpret=*
export COMPlus_InterpreterJITThreshold=999999
export COMPlus_InterpreterPrintPostMortem=1
export COMPlus_InterpreterDoLoopMethods=1
export COMPlus_TraceInterpreterEntries=1
export COMPlus_TieredCompilation=1
export COMPlus_JITMinOpts=1
export COMPlus_ZapDisable=1

# jit
export COMPlus_JitFunctionTrace=1

./bin/Product/Linux.x64.Debug/corerun ~/HelloWorld.dll
  • ARM64 with similar environment variables
./bin/Product/Linux.arm64.Debug/corerun ~/HelloWorld.dll
  • MIPS64 ditto
./bin/Product/Linux.mips64.Debug/corerun ~/HelloWorld.dll

But there are only Start Jitting and Jitted Entry, no Generating interpretation stub at all:

...
{ Start Jitting ILStubClass:IL_STUB_PInvoke(long) (MethodHash=3045e69f)
} Jitted Entry 22e at 000000ff`722b1650 method ILStubClass:IL_STUB_PInvoke(long) size 000001d0
Histogram of method executions:
   # of execs   |   # meths (%)    |   cum % | % cum execs
   -------------------------------------------------------
            0   |       0 (  nan%) |    nan% |    nan%
            1   |       0 (  nan%) |    nan% |    nan%
            2   |       0 (  nan%) |    nan% |    nan%
            3   |       0 (  nan%) |    nan% |    nan%
            4   |       0 (  nan%) |    nan% |    nan%
            5   |       0 (  nan%) |    nan% |    nan%
            6   |       0 (  nan%) |    nan% |    nan%
            7   |       0 (  nan%) |    nan% |    nan%
            8   |       0 (  nan%) |    nan% |    nan%
            9   |       0 (  nan%) |    nan% |    nan%
           10   |       0 (  nan%) |    nan% |    nan%

For methods sorted in ascending # of executions order, cumulative % of executions:

Total number of calls from interpreted code: 0.
    Also, 0 are intrinsics; 0 of these are not currently handled intrinsically.
    Of these, 0 to potential property getters (0 of these dead simple), 0 to setters.
    Of the dead simple getter calls, 0 have been short-circuited.

Token resolutions by category:
Category     |  opportunities  |   calls   |      %
---------------------------------------------------
   Undefined |               0 |         0 |   0.00%
 Constrained |               0 |         0 |   0.00%
      NewObj |               0 |         0 |   0.00%
      NewArr |               0 |         0 |   0.00%
     LdToken |               0 |         0 |   0.00%
       LdFtn |               0 |         0 |   0.00%
   LdVirtFtn |               0 |         0 |   0.00%
    SFldAddr |               0 |         0 |   0.00%
      LdElem |               0 |         0 |   0.00%
        Call |               0 |         0 |   0.00%
       LdObj |               0 |         0 |   0.00%
       StObj |               0 |         0 |   0.00%
       CpObj |               0 |         0 |   0.00%
     InitObj |               0 |         0 |   0.00%
      IsInst |               0 |         0 |   0.00%
   CastClass |               0 |         0 |   0.00%
    MkRefAny |               0 |         0 |   0.00%
   RefAnyVal |               0 |         0 |   0.00%
      Sizeof |               0 |         0 |   0.00%
      StElem |               0 |         0 |   0.00%
         Box |               0 |         0 |   0.00%
       Unbox |               0 |         0 |   0.00%
    UnboxAny |               0 |         0 |   0.00%
       LdFld |               0 |         0 |   0.00%
      LdFldA |               0 |         0 |   0.00%
       StFld |               0 |         0 |   0.00%
   FindClass |               0 |         0 |   0.00%
   Exception |               0 |         0 |   0.00%

Although it is able to hack tieredcompilation https://github.com/dotnet/coreclr/issues/24824#issuecomment-497048982 to force interpreter.
There is SEGFAULT issue:

  • X86
Generating interpretation stub (# 6 = 0x6, hash = 0x9d983a13) for ILStubClass:IL_STUB_InstantiatingStub.
Generating interpretation stub (# 7 = 0x7, hash = 0x253d4d8d) for System.Collections.Generic.EqualityComparer`1[__Canon][System.__Canon]:.cctor.
Entering method dotnet/coreclr#5 (= 0x5): System.Collections.Generic.EqualityComparer`1[__Canon][System.__Canon]:.cctor().
 arguments:
         0:  nativeint: 140737318125363 (= 0x7ffff5da7f33)

Program received signal SIGSEGV, Segmentation fault.
MethodTable::GetTypeDefRid_NoLogging (this=0x758b48f0758948f8) at /home/zhaixiang/coreclr-mips64-dev/src/vm/methodtable.cpp:8161
8161        WORD token = m_wToken;

(gdb) bt
#0  MethodTable::GetTypeDefRid_NoLogging (this=0x758b48f0758948f8)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/methodtable.cpp:8161
dotnet/coreclr#1  0x00007ffff5e6473f in MethodTable::GetTypeDefRid (this=0x758b48f0758948f8)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/methodtable.cpp:8153
dotnet/coreclr#2  0x00007ffff5e64620 in MethodTable::HasSameTypeDefAs (this=0x758b48f0758948f8, pMT=0x7fff7c355c48)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/methodtable.cpp:644
dotnet/coreclr#3  0x00007ffff5e771f0 in MethodTable::GetMethodTableMatchingParentClass (this=
    0x7ffff5da7f33 <FrameWithCookie<GCFrame>::FrameWithCookie(OBJECTREF*, unsigned int, int)+67>, pWhichParent=0x7fff7c355c48)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/methodtable.cpp:8361
dotnet/coreclr#4  0x00007ffff6019bf5 in InterpreterMethodInfo::GetPreciseGenericsContext (this=0x6b9620, thisArg=0x0, 
    genericsCtxtArg=0x7ffff5da7f33 <FrameWithCookie<GCFrame>::FrameWithCookie(OBJECTREF*, unsigned int, int)+67>)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:8857
dotnet/coreclr#5  0x00007ffff6021d9c in Interpreter::GetPreciseGenericsContext (this=0x7ffffffec198)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.h:1781
dotnet/coreclr#6  0x00007ffff60216da in Interpreter::ResolveToken (this=0x7ffffffec198, resTok=0x7ffffffeb808, token=452984835, 
    tokenType=CORINFO_TOKENKIND_Ldtoken, rtk=Interpreter::RTK_LdToken)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.hpp:396
dotnet/coreclr#7  0x00007ffff601102f in Interpreter::LdToken (this=0x7ffffffec198)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:6337
dotnet/coreclr#8  0x00007ffff6006b35 in Interpreter::ExecuteMethod (this=0x7ffffffec198, retVal=0x7ffffffec180, pDoJmpCall=0x7ffffffed6ef, 
    pJmpCallToken=0x7ffffffec17c) at /home/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:2942
dotnet/coreclr#9  0x00007ffff600347c in Interpreter::ExecuteMethodWrapper (interpMethInfo=0x6b9620, directCall=false, 
    ilArgs=0x7ffffffed990 "3\177\332\365\377\177", stubContext=0x7ffffffeddf0, pDoJmpCall=0x7ffffffed6ef, 
    pResolvedToken=0x7ffffffed698) at /home/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1589
dotnet/coreclr#10 0x00007ffff6021b02 in Interpreter::InterpretMethodBody (interpMethInfo=0x6b9620, directCall=false, 
    ilArgs=0x7ffffffed990 "3\177\332\365\377\177", stubContext=0x7ffffffeddf0)
    at /home/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1624
dotnet/coreclr#11 0x00007ffff6002d07 in InterpretMethod (interpMethInfo=0x6b9620, ilArgs=0x7ffffffed990 "3\177\332\365\377\177", 
    stubContext=0x7ffffffeddf0) at /home/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1745
dotnet/coreclr#12 0x00007fff7c0cd2e4 in ?? ()
dotnet/coreclr#13 0x00007fffffffe0a0 in ?? ()
dotnet/coreclr#14 0x0000000000000000 in ?? ()
  • ARM64
Generating interpretation stub (# 1 = 0x1, hash = 0xd8570543) for System.AppContext:Setup.

Thread 1 "corerun" received signal SIGSEGV, Segmentation fault.
0x0000007fb70b5658 in Interpreter::Interpreter (this=0x7fffffe710, methInfo_=0x0, directCall_=127, ilArgs_=0x0, stubContext_=0x7f3d43ab60, frameMemory=0x7f3d43ab60 "0~D=\177") at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.h:812
812        {
(gdb) bt
#0  0x0000007fb70b5658 in Interpreter::Interpreter (this=0x7fffffe710, methInfo_=0x0, directCall_=127, ilArgs_=0x0, stubContext_=0x7f3d43ab60, frameMemory=0x7f3d43ab60 "0~D=\177") at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.h:812
dotnet/coreclr#1  0x0000007fb7098eb4 in Interpreter::ExecuteMethodWrapper (interpMethInfo=0x7fb6f8d4c4 <MetaSig::NextArg()+156>, directCall=false, ilArgs=0x7fffffe2f8 "\020vC", stubContext=0x0, pDoJmpCall=0x7fffffe01c, pResolvedToken=0x7fffffdfc8) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1577
dotnet/coreclr#2  0x0000007fb70b5f7c in Interpreter::InterpretMethodBody (interpMethInfo=0x7fb6f8d4c4 <MetaSig::NextArg()+156>, directCall=false, ilArgs=0x7fffffe2f8 "\020vC", stubContext=0x0) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1624
dotnet/coreclr#3  0x0000007fb709887c in InterpretMethod (interpMethInfo=0x7fb6f8d4c4 <MetaSig::NextArg()+156>, ilArgs=0x7fffffe2f8 "\020vC", stubContext=0x0) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1745
dotnet/coreclr#4  0x0000007f3d19d060 in ?? ()
dotnet/coreclr#5  0x0000000000437610 in ?? ()
  • MIPS64
Generating interpretation stub (# 1 = 0x1, hash = 0xd8570543) for System.AppContext:Setup.

Program received signal SIGSEGV, Segmentation fault.
0x000000fff696b224 in Interpreter::Interpreter (this=<error reading variable: Cannot access memory at address 0xf33332ba70>, methInfo_=<error reading variable: Cannot access memory at address 0xf33332ba68>, directCall_=<error reading variable: Cannot access memory at address 0xf33332ba64>, ilArgs_=<error reading variable: Cannot access memory at address 0xf33332ba58>, stubContext_=<error reading variable: Cannot access memory at address 0xf33332ba50>, frameMemory=<error reading variable: Cannot access memory at address 0xf33332ba48>) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.h:812
812        {
Missing separate debuginfos, use: debuginfo-install libgcc-4.9.3-11.fc21.loongson.11.mips64el libstdc++-4.9.3-11.fc21.loongson.11.mips64el lttng-ust-2.4.1-3.fc21.loongson.mips64el userspace-rcu-0.8.1-6.fc21.loongson.mips64el xz-libs-5.1.2-14alpha.fc21.loongson.mips64el
(gdb) bt
#0  0x000000fff696b224 in Interpreter::Interpreter (this=<error reading variable: Cannot access memory at address 0xf33332ba70>, methInfo_=<error reading variable: Cannot access memory at address 0xf33332ba68>, directCall_=<error reading variable: Cannot access memory at address 0xf33332ba64>, ilArgs_=<error reading variable: Cannot access memory at address 0xf33332ba58>, stubContext_=<error reading variable: Cannot access memory at address 0xf33332ba50>, frameMemory=<error reading variable: Cannot access memory at address 0xf33332ba48>) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.h:812
dotnet/coreclr#1  0x000000fff693d75c in Interpreter::ExecuteMethodWrapper (interpMethInfo=<error reading variable: Cannot access memory at address 0xf33332bb38>, directCall=<error reading variable: Cannot access memory at address 0xf33332bb34>, ilArgs=<error reading variable: Cannot access memory at address 0xf33332bb28>, stubContext=<error reading variable: Cannot access memory at address 0xf33332bb20>, pDoJmpCall=<error reading variable: Cannot access memory at address 0xf33332bb18>, pResolvedToken=<error reading variable: Cannot access memory at address 0xf33332bb10>) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1627
dotnet/coreclr#2  0x000000fff696c1b4 in Interpreter::InterpretMethodBody (interpMethInfo=0x1200be190, directCall=false, ilArgs=0xffffffa0b8 "py\003 \001", stubContext=0x0) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1674
dotnet/coreclr#3  0x000000fff693cbec in InterpretMethod (interpMethInfo=0x1200be190, ilArgs=0xffffffa0b8 "py\003 \001", stubContext=0x0) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/interpreter.cpp:1795
dotnet/coreclr#4  0x000000fff6c0b608 in CallDescrWorkerInternal () at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/mips64/calldescrworkermips64.S:74
Backtrace stopped: frame did not save the PC

How to mix Interpreter and JIT compiler? Please give me some hint.

Thanks,
Leslie Zhai

category:correctness
theme:interpreter
skill-level:intermediate
cost:small

@xiangzhai
Copy link
Contributor Author

\cc @BruceForstall

@sandreenko
Copy link
Contributor

cc @dotnet/jit-contrib

@BruceForstall
Copy link
Member

@noahfalk @kouvel Did the implementation of Tiered Compilation possibly break the interpreter path? (If so, do you have a suggested solution to get the interpreter working again?) Or does this deserve more investigation?

@noahfalk
Copy link
Member

noahfalk commented Nov 9, 2019

The last time I tested interpreter + tiered compilation was probably 2-3 years ago when I refactored it a bit. I don't know if Kount tested it more recently. In the current code you could set a breakpoint here to determine how tiered compilation is setting the jit flags and here to see if those flags made it to the JIT without further alteration and we enter GenerateInterpreterStub.

@AndyAyersMS
Copy link
Member

@xiangzhai have you tried what I suggested a while back?

https://github.com/dotnet/coreclr/issues/24824#issuecomment-497048982

@xiangzhai
Copy link
Contributor Author

Hi @AndyAyersMS

I tried https://github.com/dotnet/coreclr/issues/24824#issuecomment-497166854

diff --git a/src/vm/tieredcompilation.cpp b/src/vm/tieredcompilation.cpp
index 8cdfc48..d13fb86 100644
--- a/src/vm/tieredcompilation.cpp
+++ b/src/vm/tieredcompilation.cpp
@@ -860,9 +860,6 @@ CORJIT_FLAGS TieredCompilationManager::GetJitFlags(NativeCodeVersion nativeCodeV
     CORJIT_FLAGS flags;
     if (!nativeCodeVersion.GetMethodDesc()->IsEligibleForTieredCompilation())
     {
-#ifdef FEATURE_INTERPRETER
-        flags.Set(CORJIT_FLAGS::CORJIT_FLAG_MAKEFINALCODE);
-#endif
         return flags;
     }
 
@@ -877,9 +874,6 @@ CORJIT_FLAGS TieredCompilationManager::GetJitFlags(NativeCodeVersion nativeCodeV
             // fall through
 
         case NativeCodeVersion::OptimizationTierOptimized:
-#ifdef FEATURE_INTERPRETER
-            flags.Set(CORJIT_FLAGS::CORJIT_FLAG_MAKEFINALCODE);
-#endif
             break;
 
         default:

But it lead to SEGFAULT when mixed Interpreter and JIT as described above https://github.com/dotnet/coreclr/issues/27761#issue-519851291 .

Thanks,
Leslie Zhai

@xiangzhai
Copy link
Contributor Author

Hi @noahfalk

I tracked interpreterFallback https://github.com/dotnet/coreclr/issues/24654#issue-445775637 and g_CORDebuggerControlFlags https://github.com/dotnet/coreclr/issues/24654 for 2.1 branch.

We started porting CoreCLR to MIPS64 based on 2.1 branch, then migrated to historical (about 5 months ago) master dotnet/coreclr@e999f37 So I need to track them again.

Thanks,
Leslie Zhai

@xiangzhai
Copy link
Contributor Author

xiangzhai commented Nov 11, 2019

Hi @noahfalk

  1. Build option without further alteration https://github.com/dotnet/coreclr/issues/27761#issuecomment-552259913
./build.sh cmakeargs "-DFEATURE_INTERPRETER=1" skipcrossgen ignorewarnings
  1. HelloWorld testcase for mixing Interpreter and JIT compiler
# corefx && aspnetcore ALLIN1
export CORE_LIBRARIES=/home/loongson/zhaixiang/corefx-3.1-Linux.mips64.Debug

# interpreter
export COMPlus_DumpInterpreterStubs=1
export COMPlus_Interpret=*
export COMPlus_InterpreterJITThreshold=999999
export COMPlus_InterpreterFallback=1
export COMPlus_InterpreterPrintPostMortem=1
export COMPlus_InterpreterDoLoopMethods=1
export COMPlus_TraceInterpreterEntries=1
export COMPlus_TieredCompilation=1
export COMPlus_ZapDisable=1
export COMPlus_JITMinOpts=1

# jit
export COMPlus_JitFunctionTrace=1

And X86 and ARM64 use the similar environment variables.

Breakpoint 2, TieredCompilationManager::GetJitFlags (nativeCodeVersion=...) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/tieredcompilation.cpp:860
860	    CORJIT_FLAGS flags;
(gdb) n
861	    if (!nativeCodeVersion.GetMethodDesc()->IsEligibleForTieredCompilation())
(gdb) n
864	        flags.Set(CORJIT_FLAGS::CORJIT_FLAG_MAKEFINALCODE);
(gdb) n
866	        return flags;
(gdb) i r v0
v0: 0x40000

(gdb) c
Continuing.

Breakpoint 1, invokeCompileMethodHelper (jitMgr=0x12007cf50, comp=0xffffff89a0, info=0xffffff8810, jitFlags=..., nativeEntry=0xffffff87c0, nativeSizeOfCode=0xffffff87bc) at /home/loongson/zhaixiang/coreclr-mips64-dev/src/vm/jitinterface.cpp:12350
12350	    CorJitResult ret = CORJIT_SKIPPED;   // Note that CORJIT_SKIPPED is an error exit status code
(gdb) n
12352	    comp->setJitFlags(jitFlags);
(gdb) n
12360	    if (FAILED(ret) && jitMgr->m_alternateJit
(gdb) n
12393	    bool isInterpreterStub   = false;
(gdb) n
12394	    bool interpreterFallback = (s_InterpreterFallback.val(CLRConfig::INTERNAL_InterpreterFallback) != 0);
(gdb) n
12396	    if (interpreterFallback == false)
(gdb) p/x interpreterFallback
$5 = 0x1
(gdb) n
12409	    if (FAILED(ret) && jitMgr->m_jit)
(gdb) n
12411	        ret = CompileMethodWithEtwWrapper(jitMgr, 
(gdb) n
12412	                                          comp,
(gdb) n
12413	                                          info,
(gdb) n
12415	                                          nativeEntry,
(gdb) n
12416	                                          nativeSizeOfCode);
(gdb) n
12411	        ret = CompileMethodWithEtwWrapper(jitMgr, 
(gdb) n
{ Start Jitting System.AppContext:Setup(long,long,int) (MethodHash=d8570543)
} Jitted Entry 001 at 000000ff`7c826988 method System.AppContext:Setup(long,long,int) size 00000220
12417	    }
(gdb) n
12419	    if (interpreterFallback == true)
(gdb) n
12423	        if (FAILED(ret) && !jitFlags.IsSet(CORJIT_FLAGS::CORJIT_FLAG_IMPORT_ONLY) && !jitFlags.IsSet(CORJIT_FLAGS::CORJIT_FLAG_MAKEFINALCODE))
(gdb) n
12430	    }

The flags value is 0x40000, and jitFlags.IsSet(CORJIT_FLAGS::CORJIT_FLAG_MAKEFINALCODE) is false, so Interpreter::GenerateInterpreterStub will NOT be called.

Thanks,
Leslie Zhai

@noahfalk
Copy link
Member

This code flow:

861 if (!nativeCodeVersion.GetMethodDesc()->IsEligibleForTieredCompilation())
(gdb) n
864 flags.Set(CORJIT_FLAGS::CORJIT_FLAG_MAKEFINALCODE);

suggests that you debugged a method that was not eligible for tiered compilation, and from there it would be by design not to use the interpreter because without tiering we can never switch from interpreter back to JIT.

Two potential cases here that would be useful to distinguish:

  1. One specific method that you stepped through didn't support tiering, but others would have supported it (set a breakpoint in the case where the if evaluates false and see if execution ever goes that way)
  2. All the methods are reporting that they don't support tiering. Given that COMPlus_TieredCompilation=1, this is not the behavior I would expect. The code here has the rules to determine whether each method will support tiering so we'd want to understand why it always evaluated false.

@xiangzhai
Copy link
Contributor Author

Hi @noahfalk

Thanks for your kind response!

HelloWorld.x64.JitFunctionTrace.log ARM64 and MIPS64 also goes to case 2. I will investigate why it always evaluated false.

Thanks,
Leslie Zhai

@xiangzhai
Copy link
Contributor Author

xiangzhai commented Nov 13, 2019

Hi @noahfalk

Base on dotnet/coreclr@03b7b31

Build and run HelloWorld option:

./build.sh cmakeargs "-DFEATURE_INTERPRETER=1" skipcrossgen ignorewarnings

# corefx
export CORE_LIBRARIES=~/dotnet-sdk-3.0.100-linux-x64/shared/Microsoft.NETCore.App/3.0.0

# interpreter
export COMPlus_DumpInterpreterStubs=1
export COMPlus_Interpret=*
export COMPlus_InterpreterJITThreshold=999999
export COMPlus_InterpreterFallback=1
export COMPlus_InterpreterPrintPostMortem=1
export COMPlus_InterpreterDoLoopMethods=1
export COMPlus_TraceInterpreterEntries=1
export COMPlus_TieredCompilation=1
export COMPlus_ZapDisable=1

# jit
export COMPlus_JitFunctionTrace=1

gdb --args bin/Product/Linux.x64.Debug/corerun ~/HelloWorld.dll

Please apply the patch dotnet/coreclr#27850 suggested by @jkotas at first.

Debug patch:

diff --git a/src/vm/method.cpp b/src/vm/method.cpp
index b36c78c..8d42f10 100644
--- a/src/vm/method.cpp
+++ b/src/vm/method.cpp
@@ -4850,6 +4850,7 @@ bool MethodDesc::DetermineAndSetIsEligibleForTieredCompilation()
     {
         m_bFlags2 |= enum_flag2_IsEligibleForTieredCompilation;
         _ASSERTE(IsVersionable());
+        _ASSERTE(!"WhoReachHere");
         return true;
     }
 #endif

It seems that only HelloWorld.Program:Main(System.String[]) method will support tiering:

{ Start Jitting Method  211 Internal.Runtime.CompilerServices.Unsafe:IsNullRef(byref):bool (MethodHash=b7a571a2) MinOpts
} Jitted Method  211 at 00007fff`7c0b84c0 method Internal.Runtime.CompilerServices.Unsafe:IsNullRef(byref):bool size 0000003d
{ Start Jitting Method  212 HelloWorld.Program:Main(System.String[]) (MethodHash=ab41dfb7) MinOpts

Assert failure(PID 14534 [0x000038c6], Thread: 14534 [0x38c6]): !"WhoReachHere"
    File: /home/zhaixiang/coreclr/src/vm/method.cpp Line: 4853
    Image: /home/zhaixiang/coreclr/bin/Product/Linux.x64.Debug/corerun

Changed the debug patch:

diff --git a/src/vm/method.cpp b/src/vm/method.cpp
index b36c78c..4b7665e 100644
--- a/src/vm/method.cpp
+++ b/src/vm/method.cpp
@@ -4850,6 +4850,7 @@ bool MethodDesc::DetermineAndSetIsEligibleForTieredCompilation()
     {
         m_bFlags2 |= enum_flag2_IsEligibleForTieredCompilation;
         _ASSERTE(IsVersionable());
+        printf("WhoReachHere\n");
         return true;
     }
 #endif

HelloWorld.WhoReachHere.JitFunctionTrace.log
A small number of methods from System.Private.CoreLib and corefx will support tiering, but lots of them still goto JIT. Compared with HotSpot Tiered compilation http://cr.openjdk.java.net/~vlivanov/talks/2015_JIT_Overview.pdf Bytecode → Interpreter + JITted executable is what we desire.

Or is it able to just enable Interpreter only just like java -Xint?

Thanks,
Leslie Zhai

@msftgits msftgits transferred this issue from dotnet/coreclr Jan 31, 2020
@msftgits msftgits added this to the Future milestone Jan 31, 2020
@xiangzhai
Copy link
Contributor Author

It is not able to work equivalent to java -Xmixed, so please reopen the issue.

@BruceForstall BruceForstall reopened this Jun 29, 2020
@BruceForstall BruceForstall added the JitUntriaged CLR JIT issues needing additional triage label Oct 28, 2020
@BruceForstall BruceForstall removed the JitUntriaged CLR JIT issues needing additional triage label Oct 30, 2022
@BruceForstall
Copy link
Member

We currently have no work ongoing to support the interpreter, so I am going to close this.

@ghost ghost locked as resolved and limited conversation to collaborators Nov 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
Projects
None yet
6 participants