Skip to content

Commit

Permalink
[mono][interp] Add tiering within interpreter (#68823)
Browse files Browse the repository at this point in the history
* [mono][interp] Remove error argument from mono_interp_get_imethod

It is unused

* [mono][interp] Implement tiering at method execution start

For a single MonoMethod*, we can have two InterpMethod* instances, one with optimized flag false and the other with true. When tiering is enabled, when first getting an InterpMethod* for a MonoMethod* we set the optimized flag to false. When generatig code for this method, if optimized is false we must emit a special MINT_TIER_ENTER_METHOD at the start and later in the codegen process we skip applying optimizations to method code.

MINT_TIER_ENTER_METHOD opcode is invoked with every method start and it will bump a counter. Once we hit the limit, the method will be tiered up. This process consists of creating a new InterpMethod* instance which have optimized set and storing it in the interp_code_hash, changing the mapping from the old MonoMethod. The optimized and unoptimized method use the same argument space, so tiering the method up requires just to set the ip to the start of the tiered up method code.

An additional problem that happens with tiering is that we have to replace all instances of the untiered method from generated code. InterpMethod* instances are stored stored inside data_items of other methods and also inside vtables. When generating code for any method, we have to store in a hash table mappings from untiered InterpMethod* instance to addresses where this instance was stored. When we tier up the unoptimized method, we will traverse the list of addresses where this references is stored and update it to the optimized version.

* [mono][interp] Add option to enable/disable optimizations

Some optimizations might not be enabled by default, so add option to enable them.

* [mono][interp] Implement on stack replacement tiering up

In unoptimized code, we add a patchpoint instruction when entering basic blocks that are targets of backward branches, if the stack state is empty. This means that when tiering up a frame we just need to jump to the equivalent basic block in the tiered up method and execution can continue. Since the arguments and IL locals reside in the same space in both versions of the method (their offsets are computed in interp_method_compute_offsets)

* [mono][interp] Remove calc_section mutex

We always take jit_mm lock when finishing compilation of method, use it also for publishing InterpMethod* fields. This also prevents weird races where the method can be tiered up before the we take the jit_mm lock, resulting in publishing the seq_points for the untiered method

* [mono][interp] Fix incorrect linking of bblocks

Once we emit a tailcall, execution in the current bblock is finished.

* [mono][interp] Fix implicit conversion from i4 to native int

We were doing unsigned conversion before

* [mono][interp] Fix execution of clauses from il state when tiering is enabled

When invoking these clauses we obtained the InterpMethod from the MonoMethod* and make use of the jit info stored during frame unwinding. However, the method might have been tiered up since storing the jit info, so the native offsets stored there will no longer be relative to the optimized imethod. Fetch again the MonoJitInfo* from the imethod that we will be executing.

* [mono][interp] Enable tiering by default
  • Loading branch information
BrzVlad authored May 25, 2022
1 parent 1601356 commit 962a455
Show file tree
Hide file tree
Showing 13 changed files with 548 additions and 138 deletions.
4 changes: 3 additions & 1 deletion src/mono/mono/mini/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,9 @@ set(interp_sources
interp/interp-intrins.c
interp/mintops.h
interp/mintops.c
interp/transform.c)
interp/transform.c
interp/tiering.h
interp/tiering.c)
set(interp_stub_sources
interp-stubs.c)

Expand Down
6 changes: 3 additions & 3 deletions src/mono/mono/mini/ee.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
#ifndef __MONO_EE_H__
#define __MONO_EE_H__

#define MONO_EE_API_VERSION 0x16
#define MONO_EE_API_VERSION 0x17

typedef struct _MonoInterpStackIter MonoInterpStackIter;

Expand All @@ -38,7 +38,7 @@ typedef gpointer MonoInterpFrameHandle;
MONO_EE_CALLBACK (void, get_resume_state, (const MonoJitTlsData *jit_tls, gboolean *has_resume_state, MonoInterpFrameHandle *interp_frame, gpointer *handler_ip)) \
MONO_EE_CALLBACK (gboolean, run_finally, (StackFrameInfo *frame, int clause_index, gpointer handler_ip, gpointer handler_ip_end)) \
MONO_EE_CALLBACK (gboolean, run_filter, (StackFrameInfo *frame, MonoException *ex, int clause_index, gpointer handler_ip, gpointer handler_ip_end)) \
MONO_EE_CALLBACK (gboolean, run_clause_with_il_state, (gpointer il_state, int clause_index, gpointer handler_ip, gpointer handler_ip_end, MonoObject *ex, gboolean *filtered, MonoExceptionEnum clause_type)) \
MONO_EE_CALLBACK (gboolean, run_clause_with_il_state, (gpointer il_state, int clause_index, MonoObject *ex, gboolean *filtered)) \
MONO_EE_CALLBACK (void, frame_iter_init, (MonoInterpStackIter *iter, gpointer interp_exit_data)) \
MONO_EE_CALLBACK (gboolean, frame_iter_next, (MonoInterpStackIter *iter, StackFrameInfo *frame)) \
MONO_EE_CALLBACK (MonoJitInfo*, find_jit_info, (MonoMethod *method)) \
Expand All @@ -63,7 +63,7 @@ typedef gpointer MonoInterpFrameHandle;
MONO_EE_CALLBACK (void, jit_info_foreach, (InterpJitInfoFunc func, gpointer user_data)) \
MONO_EE_CALLBACK (gboolean, sufficient_stack, (gsize size)) \
MONO_EE_CALLBACK (void, entry_llvmonly, (gpointer res, gpointer *args, gpointer imethod)) \
MONO_EE_CALLBACK (gpointer, get_interp_method, (MonoMethod *method, MonoError *error)) \
MONO_EE_CALLBACK (gpointer, get_interp_method, (MonoMethod *method)) \
MONO_EE_CALLBACK (MonoJitInfo*, compile_interp_method, (MonoMethod *method, MonoError *error)) \

typedef struct _MonoEECallbacks {
Expand Down
5 changes: 2 additions & 3 deletions src/mono/mono/mini/interp-stubs.c
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,7 @@ stub_run_filter (StackFrameInfo *frame, MonoException *ex, int clause_index, gpo
}

static gboolean
stub_run_clause_with_il_state (gpointer il_state, int clause_index, gpointer handler_ip, gpointer handler_ip_end, MonoObject *ex,
gboolean *filtered, MonoExceptionEnum clause_type)
stub_run_clause_with_il_state (gpointer il_state, int clause_index, MonoObject *ex, gboolean *filtered)
{
g_assert_not_reached ();
}
Expand Down Expand Up @@ -240,7 +239,7 @@ stub_entry_llvmonly (gpointer res, gpointer *args, gpointer imethod)
}

static gpointer
stub_get_interp_method (MonoMethod *method, MonoError *error)
stub_get_interp_method (MonoMethod *method)
{
g_assert_not_reached ();
return NULL;
Expand Down
29 changes: 25 additions & 4 deletions src/mono/mono/mini/interp/interp-internals.h
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,13 @@ typedef enum {

#define PROFILE_INTERP 0

#define INTERP_IMETHOD_TAG_UNBOX(im) ((gpointer)((mono_u)(im) | 1))
#define INTERP_IMETHOD_IS_TAGGED_UNBOX(im) ((mono_u)(im) & 1)
#define INTERP_IMETHOD_UNTAG_UNBOX(im) ((InterpMethod*)((mono_u)(im) & ~1))
#define INTERP_IMETHOD_TAG_1(im) ((gpointer)((mono_u)(im) | 1))
#define INTERP_IMETHOD_IS_TAGGED_1(im) ((mono_u)(im) & 1)
#define INTERP_IMETHOD_UNTAG_1(im) ((InterpMethod*)((mono_u)(im) & ~1))

#define INTERP_IMETHOD_TAG_UNBOX(im) INTERP_IMETHOD_TAG_1(im)
#define INTERP_IMETHOD_IS_TAGGED_UNBOX(im) INTERP_IMETHOD_IS_TAGGED_1(im)
#define INTERP_IMETHOD_UNTAG_UNBOX(im) INTERP_IMETHOD_UNTAG_1(im)

/*
* Structure representing a method transformed for the interpreter
Expand Down Expand Up @@ -144,8 +148,25 @@ struct InterpMethod {
#ifdef ENABLE_EXPERIMENT_TIERED
MiniTieredCounter tiered_counter;
#endif
gint32 entry_count;
InterpMethod *optimized_imethod;
// This data is used to resolve native offsets from unoptimized method to native offsets
// in the optimized method. We rely on keys identifying a certain logical execution point
// to be equal between unoptimized and optimized method. In unoptimized method we map from
// native_offset to a key and in optimized_method we map from key to a native offset.
//
// The logical execution points that are being tracked are some basic block starts (in this
// case we don't need any tracking in the unoptimized method, just the mapping from bbindex
// to its native offset) and call handler returns. Call handler returns store the return ip
// on the stack so once we tier up the method we need to update these to IPs in the optimized
// method. The key for a call handler is its index, in appearance order in the IL, multiplied
// by -1. (So we don't collide with basic block indexes)
//
// Since we have both positive and negative keys in this array, we use G_MAXINTRE as terminator.
int *patchpoint_data;
unsigned int init_locals : 1;
unsigned int vararg : 1;
unsigned int optimized : 1;
unsigned int needs_thread_attach : 1;
#if PROFILE_INTERP
long calls;
Expand Down Expand Up @@ -265,7 +286,7 @@ void
mono_interp_transform_init (void);

InterpMethod *
mono_interp_get_imethod (MonoMethod *method, MonoError *error);
mono_interp_get_imethod (MonoMethod *method);

void
mono_interp_print_code (InterpMethod *imethod);
Expand Down
Loading

0 comments on commit 962a455

Please sign in to comment.