Skip to content

Commit

Permalink
Revert "Add support for cross module inlining and cross module generi…
Browse files Browse the repository at this point in the history 
…c compilation to Crossgen2 (#68919)" (#71076)

This reverts commit 18ec279.
  • Loading branch information
@davidwrighton
davidwrighton authored Jun 21, 2022
1 parent 775a7ec commit 89dc073
Show file tree
Hide file tree
Showing 95 changed files with 1,526 additions and 4,930 deletions.
85 changes: 14 additions & 71 deletions docs/design/coreclr/botr/readytorun-format.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ Revisions:
* 4.1 - [Tomas Rylek](https://github.com/trylek) - 2020
* 5.3 - [Tomas Rylek](https://github.com/trylek) - 2021
* 5.4 - [David Wrighton](https://github.com/davidwrighton) - 2021
* 6.3 - [David Wrighton](https://github.com/davidwrighton) - 2022

# Introduction

Expand Down Expand Up @@ -125,15 +124,12 @@ struct READYTORUN_CORE_HEADER

### READYTORUN_CORE_HEADER::Flags

| Flag | Value | Description
|:-------------------------------------------|-----------:|:-----------
| READYTORUN_FLAG_PLATFORM_NEUTRAL_SOURCE | 0x00000001 | Set if the original IL image was platform neutral. The platform neutrality is part of assembly name. This flag can be used to reconstruct the full original assembly name.
| READYTORUN_FLAG_COMPOSITE | 0x00000002 | The image represents a composite R2R file resulting from a combined compilation of a larger number of input MSIL assemblies.
| READYTORUN_FLAG_PARTIAL | 0x00000004 |
| READYTORUN_FLAG_NONSHARED_PINVOKE_STUBS | 0x00000008 | PInvoke stubs compiled into image are non-shareable (no secret parameter)
| READYTORUN_FLAG_EMBEDDED_MSIL | 0x00000010 | Input MSIL is embedded in the R2R image.
| READYTORUN_FLAG_COMPONENT | 0x00000020 | This is a component assembly of a composite R2R image
| READYTORUN_FLAG_MULTIMODULE_VERSION_BUBBLE | 0x00000040 | This R2R module has multiple modules within its version bubble (For versions before version 6.2, all modules are assumed to possibly have this characteristic)
| Flag | Value | Description
|:----------------------------------------|-----------:|:-----------
| READYTORUN_FLAG_PLATFORM_NEUTRAL_SOURCE | 0x00000001 | Set if the original IL image was platform neutral. The platform neutrality is part of assembly name. This flag can be used to reconstruct the full original assembly name.
| READYTORUN_FLAG_COMPOSITE | 0x00000002 | The image represents a composite R2R file resulting from a combined compilation of a larger number of input MSIL assemblies.
| READYTORUN_FLAG_EMBEDDED_MSIL | 0x00000004 | Input MSIL is embedded in the R2R image.
| READYTORUN_FLAG_COMPONENT | 0x00000008 | This is a component assembly of a composite R2R image

## READYTORUN_SECTION

Expand Down Expand Up @@ -177,7 +173,6 @@ The following section types are defined and described later in this document:
| OwnerCompositeExecutable | 116 | Image (added in V4.1)
| PgoInstrumentationData | 117 | Image (added in V5.2)
| ManifestAssemblyMvids | 118 | Image (added in V5.3)
| CrossModuleInlineInfo | 119 | Image (added in V6.3)

## ReadyToRunSectionType.CompilerIdentifier

Expand Down Expand Up @@ -208,19 +203,13 @@ struct READYTORUN_IMPORT_SECTION

| ReadyToRunImportSectionFlags | Value | Description
|:---------------------------------------|-------:|:-----------
| ReadyToRunImportSectionFlags::None | 0x0000 | None
| ReadyToRunImportSectionFlags::Eager | 0x0001 | Set if the slots in the section have to be initialized at image load time. It is used to avoid lazy initialization when it cannot be done or when it would have undesirable reliability or performance effects (unexpected failure or GC trigger points, overhead of lazy initialization).
| ReadyToRunImportSectionFlags::PCode | 0x0004 | Section contains pointers to code

| READYTORUN_IMPORT_SECTION_FLAGS_EAGER | 0x0001 | Set if the slots in the section have to be initialized at image load time. It is used to avoid lazy initialization when it cannot be done or when it would have undesirable reliability or performance effects (unexpected failure or GC trigger points, overhead of lazy initialization).

### READYTORUN_IMPORT_SECTIONS::Type

| ReadyToRunImportSectionType | Value | Description
|:--------------------------------------------|-------:|:-----------
| ReadyToRunImportSectionType::Unknown | 0 | The type of slots in this section is unspecified.
| ReadyToRunImportSectionType::StubDispatch | 2 | The type of slots in this section rely on stubs for dispatch.
| ReadyToRunImportSectionType::StringHandle | 3 | The type of slots in this section hold strings
| ReadyToRunImportSectionType::ILBodyFixups | 7 | The type of slots in this section represent cross module IL bodies
| ReadyToRunImportSectionType | Value | Description
|:---------------------------------------|-------:|:-----------
| READYTORUN_IMPORT_SECTION_TYPE_UNKNOWN | 0 | The type of slots in this section is unspecified.

*Future*: The section type can be used to group slots of the same type together. For example, all virtual
stub dispatch slots may be grouped together to simplify resetting of virtual stub dispatch cells into their
Expand Down Expand Up @@ -275,8 +264,6 @@ fixup kind, the rest of the signature varies based on the fixup kind.
| READYTORUN_FIXUP_Verify_TypeLayout | 0x32 | Generate a runtime check to ensure that the field offset matches between compile and runtime. Unlike CheckFieldOffset, this will generate a runtime exception on failure instead of silently dropping the method
| READYTORUN_FIXUP_Check_VirtualFunctionOverride | 0x33 | Generate a runtime check to ensure that virtual function resolution has equivalent behavior at runtime as at compile time. If not equivalent, code will not be used. See [Virtual override signatures](virtual-override-signatures) for details of the signature used.
| READYTORUN_FIXUP_Verify_VirtualFunctionOverride | 0x33 | Generate a runtime check to ensure that virtual function resolution has equivalent behavior at runtime as at compile time. If not equivalent, generate runtime failure. See [Virtual override signatures](virtual-override-signatures) for details of the signature used.
| READYTORUN_FIXUP_Check_IL_Body | 0x35 | Check to see if an IL method is defined the same at runtime as at compile time. A failed match will cause code not to be used. See[IL Body signatures](il-body-signatures) for details.
| READYTORUN_FIXUP_Verify_IL_Body | 0x36 | Verify an IL body is defined the same at compile time and runtime. A failed match will cause a hard runtime failure. See[IL Body signatures](il-body-signatures) for details.
| READYTORUN_FIXUP_ModuleOverride | 0x80 | When or-ed to the fixup ID, the fixup byte in the signature is followed by an encoded uint with assemblyref index, either within the MSIL metadata of the master context module for the signature or within the manifest metadata R2R header table (used in cases inlining brings in references to assemblies not seen in the input MSIL).

#### Method Signatures
Expand Down Expand Up @@ -317,9 +304,6 @@ ECMA 335 does not have a natural encoding for describing an overriden method. Th
| READYTORUN_VIRTUAL_OVERRIDE_None | 0x00 | No flags are set
| READYTORUN_VIRTUAL_OVERRIDE_VirtualFunctionOverriden | 0x01 | If set, then the virtual function has an implementation, which is encoded in the optional method implementation signature.

#### IL Body signatures

ECMA 335 does not define a format that can represent the exact implementation of a method by itself. This signature holds all of the IL of the method, the EH table, the locals table, and each token (other than type references) in those tables is replaced with an index into a local stream of signatures. Those signatures are simply verbatim copies of the needed metadata to describe MemberRefs, TypeSpecs, MethodSpecs, StandaloneSignatures and strings. All of that is bundled into a large byte array. In addition, a series of TypeSignatures follows which allow the type references to be resolved, as well as a methodreference to the uninstantiated method. Assuming all of this matches with the data that is present at runtime, the fixup is considered to be satisfied. See ReadyToRunStandaloneMetadata.cs for the exact details of the format.

### READYTORUN_IMPORT_SECTIONS::AuxiliaryData

Expand Down Expand Up @@ -511,7 +495,7 @@ properly look up methods stored in this section in the composite R2R case.

**TODO**: document profile data encoding

## ReadyToRunSectionType.ManifestMetadata (v2.3+ with changes for v6.3+)
## ReadyToRunSectionType.ManifestMetadata (v2.3+)

Manifest metadata is an [ECMA-335] metadata blob containing extra reference assemblies within
the version bubble introduced by inlining on top of assembly references stored in the input MSIL.
Expand All @@ -520,16 +504,12 @@ translate module override indices in signatures to the actual reference modules
the `READYTORUN_FIXUP_ModuleOverride` bit flag on the signature fixup byte or the
`ELEMENT_TYPE_MODULE_ZAPSIG` COR element type).

**Note:** It doesn't make sense to use references to assemblies external to the version bubble
in the manifest metadata via the `READYTORUN_FIXUP_ModuleOverride` or `ELEMENT_TYPE_MODULE_ZAPSIG` concept
as there's no guarantee that their metadata token values remain constant; thus we cannot encode signatures relative to them.
However, as of R2R version 6.2, the native manifest metadata may contain tokens to be further resolved to actual
implementation assemblies.
**Note:** It doesn't make sense to store references to assemblies external to the version bubble
in the manifest metadata as there's no guarantee that their metadata token values remain
constant; thus we cannot encode signatures relative to them.

The module override index translation algorithm is as follows (**ILAR** = *the number of `AssemblyRef` rows in the input MSIL*):

For R2R version 6.2 and below

| Module override index (*i*) | Reference assembly
|:----------------------------|:------------------
| *i* = 0 | Global context - assembly containing the signature
Expand All @@ -538,16 +518,6 @@ For R2R version 6.2 and below

**Note:** This means that the entry corresponding to *i* = **ILAR** + 1 is actually undefined as it corresponds to the `NULL` entry (ROWID #0) in the manifest metadata AssemblyRef table. The first meaningful index into the manifest metadata, *i* = **ILAR** + 2, corresponding to ROWID #1, is historically filled in by Crossgen with the input assembly info but this shouldn't be depended upon, in fact the input assembly is useless in the manifest metadata as the module override to it can be encoded by using the special index 0.

For R2R version 6.3 and above
| Module override index (*i*) | Reference assembly
|:----------------------------|:------------------
| *i* = 0 | Global context - assembly containing the signature
| 1 <= *i* <= **ILAR** | *i* is the index into the MSIL `AssemblyRef` table
| *i* = **ILAR** + 1 | *i* is the index which refers to the Manifest metadata itself
| *i* > **ILAR** + 1 | *i* - **ILAR** - 2 is the zero-based index into the `AssemblyRef` table in the manifest metadata

In addition, a ModuleRef within the module which refers to `System.Private.CoreLib` may be used to serve as the *ResolutionContext* of a *TypeRef* within the manifest metadata. This will always refer to the module which contains the `System.Object` type.

## ReadyToRunSectionType.AttributePresence (v3.1+)

**TODO**: document attribute presence encoding
Expand Down Expand Up @@ -608,33 +578,6 @@ Number of assemblies stored in the manifest metadata is equal to the number of M
MVID records are used at runtime to verify that the assemblies loaded match those referenced by the
manifest metadata representing the versioning bubble.

## ReadyToRunSectionType.CrossModuleInlineInfo (v6.3+)
The inlining information section captures what methods got inlined into other methods. It consists of a single _Native Format Hashtable_ (described below).

The entries in the hashtable are lists of inliners for each inlinee. One entry in the hashtable corresponds to one inlinee. The hashtable is hashed with the version resilient hashcode of the uninstantiated methoddef inlinee.

The entry of the hashtable is a counted sequence of compressed unsigned integers which begins with an InlineeIndex which combines a 30 bit index with 2 bits of flags which how the sequence of inliners shall be parsed and what table is to be indexed into to find the inlinee.

* InlineeIndex
* Index with 2 flags field in lowest 2 bits to define the inlinee
- If (flags & 1) == 0 then index is a MethodDef RID, and if the module is a composite image, a module index of the method follows
- If (flags & 1) == 1, then index is an index into the ILBody import section
- If (flags & 2) == 0 then inliner list is:
- Inliner RID deltas - See definition below
- if (flags & 2) == 2 then what follows is:
- count of delta encoded indices into the ILBody import section
- the sequence of delta encoded indices into the first import section with a type of READYTORUN_IMPORT_SECTION_TYPE_ILBODYFIXUPS
- Inliner RID deltas - See definition below

* Inliner RID deltas (for multi-module version bubble images specified by the module having the READYTORUN_FLAG_MULTIMODULE_VERSION_BUBBLE flag set)
- a sequence of inliner RID deltas with flag in the lowest bit
- if flag is set, the inliner RID is followed by a module ID
- otherwise the module is the same as the module of the inlinee method
* Inliner RID deltas (for single module version bubble images)
- a sequence of inliner RID deltas

This section may be included in addition to a InliningInfo2 section.

# Native Format

Native format is set of encoding patterns that allow persisting type system data in a binary format that is
Expand Down
2 changes: 1 addition & 1 deletion src/coreclr/debug/daccess/request.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1706,7 +1706,7 @@ ClrDataAccess::GetModuleData(CLRDATA_ADDRESS addr, struct DacpModuleData *Module
ModuleData->TypeRefToMethodTableMap = PTR_CDADDR(pModule->m_TypeRefToMethodTableMap.pTable);
ModuleData->MethodDefToDescMap = PTR_CDADDR(pModule->m_MethodDefToDescMap.pTable);
ModuleData->FieldDefToDescMap = PTR_CDADDR(pModule->m_FieldDefToDescMap.pTable);
ModuleData->MemberRefToDescMap = PTR_CDADDR(pModule->m_MemberRefMap.pTable);
ModuleData->MemberRefToDescMap = NULL;
ModuleData->FileReferencesMap = PTR_CDADDR(pModule->m_FileReferencesMap.pTable);
ModuleData->ManifestModuleReferencesMap = PTR_CDADDR(pModule->m_ManifestModuleReferencesMap.pTable);

Expand Down
15 changes: 2 additions & 13 deletions src/coreclr/inc/corcompile.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,16 +46,6 @@ inline ReadyToRunImportSectionFlags operator &( const ReadyToRunImportSectionFla
return static_cast<ReadyToRunImportSectionFlags>(static_cast<uint16_t>(left) & static_cast<uint16_t>(right));
}

inline ReadyToRunCrossModuleInlineFlags operator |( const ReadyToRunCrossModuleInlineFlags left, const ReadyToRunCrossModuleInlineFlags right)
{
return static_cast<ReadyToRunCrossModuleInlineFlags>(static_cast<uint32_t>(left) | static_cast<uint32_t>(right));
}

inline ReadyToRunCrossModuleInlineFlags operator &( const ReadyToRunCrossModuleInlineFlags left, const ReadyToRunCrossModuleInlineFlags right)
{
return static_cast<ReadyToRunCrossModuleInlineFlags>(static_cast<uint32_t>(left) & static_cast<uint32_t>(right));
}

#ifdef TARGET_X86

typedef DPTR(RUNTIME_FUNCTION) PTR_RUNTIME_FUNCTION;
Expand Down Expand Up @@ -159,18 +149,17 @@ enum CORCOMPILE_FIXUP_BLOB_KIND
ENCODE_CHECK_VIRTUAL_FUNCTION_OVERRIDE, /* Generate a runtime check to ensure that virtual function resolution has equivalent behavior at runtime as at compile time. If not equivalent, code will not be used */
ENCODE_VERIFY_VIRTUAL_FUNCTION_OVERRIDE, /* Generate a runtime check to ensure that virtual function resolution has equivalent behavior at runtime as at compile time. If not equivalent, generate runtime failure. */

ENCODE_CHECK_IL_BODY, /* Check to see if an IL method is defined the same at runtime as at compile time. A failed match will cause code not to be used. */
ENCODE_VERIFY_IL_BODY, /* Verify an IL body is defined the same at compile time and runtime. A failed match will cause a hard runtime failure. */

ENCODE_MODULE_HANDLE = 0x50, /* Module token */
ENCODE_STATIC_FIELD_ADDRESS, /* For accessing a static field */
ENCODE_MODULE_ID_FOR_STATICS, /* For accessing static fields */
ENCODE_MODULE_ID_FOR_GENERIC_STATICS, /* For accessing static fields */
ENCODE_CLASS_ID_FOR_STATICS, /* For accessing static fields */
ENCODE_SYNC_LOCK, /* For synchronizing access to a type */
ENCODE_PROFILING_HANDLE, /* For the method's profiling counter */
ENCODE_VARARGS_METHODDEF, /* For calling a varargs method */
ENCODE_VARARGS_METHODREF,
ENCODE_VARARGS_SIG,
ENCODE_ACTIVE_DEPENDENCY, /* Conditional active dependency */
};

enum EncodeMethodSigFlags
Expand Down
20 changes: 2 additions & 18 deletions src/coreclr/inc/readytorun.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

// Keep these in sync with src/coreclr/tools/Common/Internal/Runtime/ModuleHeaders.cs
#define READYTORUN_MAJOR_VERSION 0x0006
#define READYTORUN_MINOR_VERSION 0x0003
#define READYTORUN_MINOR_VERSION 0x0002

#define MINIMUM_READYTORUN_MAJOR_VERSION 0x006

Expand Down Expand Up @@ -60,7 +60,6 @@ enum ReadyToRunFlag
READYTORUN_FLAG_NONSHARED_PINVOKE_STUBS = 0x00000008, // PInvoke stubs compiled into image are non-shareable (no secret parameter)
READYTORUN_FLAG_EMBEDDED_MSIL = 0x00000010, // MSIL is embedded in the composite R2R executable
READYTORUN_FLAG_COMPONENT = 0x00000020, // This is the header describing a component assembly of composite R2R
READYTORUN_FLAG_MULTIMODULE_VERSION_BUBBLE = 0x00000040, // This R2R module has multiple modules within its version bubble (For versions before version 6.2, all modules are assumed to possibly have this characteristic)
};

enum class ReadyToRunSectionType : uint32_t
Expand All @@ -84,7 +83,6 @@ enum class ReadyToRunSectionType : uint32_t
OwnerCompositeExecutable = 116, // Added in V4.1
PgoInstrumentationData = 117, // Added in V5.2
ManifestAssemblyMvids = 118, // Added in V5.3
CrossModuleInlineInfo = 119, // Added in V6.2

// If you add a new section consider whether it is a breaking or non-breaking change.
// Usually it is non-breaking, but if it is preferable to have older runtimes fail
Expand All @@ -101,10 +99,9 @@ struct READYTORUN_SECTION

enum class ReadyToRunImportSectionType : uint8_t
{
Unknown = 0,
Unknown = 0,
StubDispatch = 2,
StringHandle = 3,
ILBodyFixups = 7,
};

enum class ReadyToRunImportSectionFlags : uint16_t
Expand Down Expand Up @@ -168,16 +165,6 @@ enum ReadyToRunVirtualFunctionOverrideFlags
READYTORUN_VIRTUAL_OVERRIDE_VirtualFunctionOverriden = 0x01,
};

enum class ReadyToRunCrossModuleInlineFlags : uint32_t
{
CrossModuleInlinee = 0x1,
HasCrossModuleInliners = 0x2,
CrossModuleInlinerIndexShift = 2,

InlinerRidHasModule = 0x1,
InlinerRidShift = 1,
};

//
// Constants for fixup signature encoding
//
Expand Down Expand Up @@ -240,9 +227,6 @@ enum ReadyToRunFixupKind

READYTORUN_FIXUP_Check_VirtualFunctionOverride = 0x33, /* Generate a runtime check to ensure that virtual function resolution has equivalent behavior at runtime as at compile time. If not equivalent, code will not be used */
READYTORUN_FIXUP_Verify_VirtualFunctionOverride = 0x34, /* Generate a runtime check to ensure that virtual function resolution has equivalent behavior at runtime as at compile time. If not equivalent, generate runtime failure. */

READYTORUN_FIXUP_Check_IL_Body = 0x35, /* Check to see if an IL method is defined the same at runtime as at compile time. A failed match will cause code not to be used. */
READYTORUN_FIXUP_Verify_IL_Body = 0x36, /* Verify an IL body is defined the same at compile time and runtime. A failed match will cause a hard runtime failure. */
};

//
Expand Down
Loading

0 comments on commit 89dc073

Please sign in to comment.