Skip to content

Commit

Permalink
Merge remote-tracking branch 'dotnet/main' into mono-vectoras-3
Browse files Browse the repository at this point in the history
  • Loading branch information
tannergooding committed Jul 8, 2024
2 parents d1f1b9f + 55747a5 commit 9388349
Show file tree
Hide file tree
Showing 388 changed files with 16,822 additions and 6,498 deletions.
224 changes: 224 additions & 0 deletions docs/design/datacontracts/RuntimeTypeSystem.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
# Contract RuntimeTypeSystem

This contract is for exploring the properties of the runtime types of values on the managed heap or on the stack in a .NET process.

## APIs of contract

A `MethodTable` is the runtime representation of the type information about a value. Given a `TargetPointer` address, the `RuntimeTypeSystem` contract provides a `MethodTableHandle` for querying the `MethodTable`.

``` csharp
struct MethodTableHandle
{
// no public properties or constructors
internal TargetPointer Address { get; }
}
```

``` csharp
#region MethodTable inspection APIs
public virtual MethodTableHandle GetMethodTableHandle(TargetPointer targetPointer);

public virtual TargetPointer GetModule(MethodTableHandle methodTable);
// A canonical method table is either the MethodTable itself, or in the case of a generic instantiation, it is the
// MethodTable of the prototypical instance.
public virtual TargetPointer GetCanonicalMethodTable(MethodTableHandle methodTable);
public virtual TargetPointer GetParentMethodTable(MethodTableHandle methodTable);

public virtual uint GetBaseSize(MethodTableHandle methodTable);
// The component size is only available for strings and arrays. It is the size of the element type of the array, or the size of an ECMA 335 character (2 bytes)
public virtual uint GetComponentSize(MethodTableHandle methodTable);

// True if the MethodTable is the sentinel value associated with unallocated space in the managed heap
public virtual bool IsFreeObjectMethodTable(MethodTableHandle methodTable);
public virtual bool IsString(MethodTableHandle methodTable);
// True if the MethodTable represents a type that contains managed references
public virtual bool ContainsGCPointers(MethodTableHandle methodTable);
public virtual bool IsDynamicStatics(MethodTableHandle methodTable);
public virtual ushort GetNumMethods(MethodTableHandle methodTable);
public virtual ushort GetNumInterfaces(MethodTableHandle methodTable);

// Returns an ECMA-335 TypeDef table token for this type, or for its generic type definition if it is a generic instantiation
public virtual uint GetTypeDefToken(MethodTableHandle methodTable);
// Returns the ECMA 335 TypeDef table Flags value (a bitmask of TypeAttributes) for this type,
// or for its generic type definition if it is a generic instantiation
public virtual uint GetTypeDefTypeAttributes(MethodTableHandle methodTable);
#endregion MethodTable inspection APIs
```

## Version 1

The `MethodTable` inspection APIs are implemented in terms of the following flags on the runtime `MethodTable` structure:

``` csharp
internal partial struct RuntimeTypeSystem_1
{
// The lower 16-bits of the MTFlags field are used for these flags,
// if WFLAGS_HIGH.HasComponentSize is unset
[Flags]
internal enum WFLAGS_LOW : uint
{
GenericsMask = 0x00000030,
GenericsMask_NonGeneric = 0x00000000, // no instantiation
StringArrayValues = GenericsMask_NonGeneric,
}

// Upper bits of MTFlags
[Flags]
internal enum WFLAGS_HIGH : uint
{
Category_Mask = 0x000F0000,
Category_Array = 0x00080000,
Category_Array_Mask = 0x000C0000,
Category_Interface = 0x000C0000,
ContainsGCPointers = 0x01000000,
HasComponentSize = 0x80000000, // This is set if lower 16 bits is used for the component size,
// otherwise the lower bits are used for WFLAGS_LOW
}

[Flags]
internal enum WFLAGS2_ENUM : uint
{
DynamicStatics = 0x0002,
}

// Encapsulates the MethodTable flags v1 uses
internal struct MethodTableFlags
{
public uint MTFlags { get; }
public uint MTFlags2 { get; }
public uint BaseSize { get; }

public WFLAGS_LOW GetFlag(WFLAGS_LOW mask) { ... /* mask & lower 16 bits of MTFlags */ }
public WFLAGS_HIGH GetFlag(WFLAGS_HIGH mask) { ... /* mask & upper 16 bits of MTFlags */ }

public WFLAGS2_ENUM GetFlag(WFLAGS2_ENUM mask) { ... /* mask & MTFlags2*/ }

private bool TestFlagWithMask(WFLAGS_LOW mask, WFLAGS_LOW flag)
{
if (IsStringOrArray)
{
return (WFLAGS_LOW.StringArrayValues & mask) == flag;
}
else
{
return (FlagsLow & mask) == flag;
}
}

public ushort ComponentSizeBits => (ushort)(MTFlags & 0x0000ffff); // only meaningful if HasComponentSize is set
public bool HasComponentSize => GetFlag(WFLAGS_HIGH.HasComponentSize) != 0;
public bool IsInterface => GetFlag(WFLAGS_HIGH.Category_Mask) == WFLAGS_HIGH.Category_Interface;
public bool IsString => HasComponentSize && !IsArray && ComponentSizeBits == 2;
public bool IsArray => GetFlag(WFLAGS_HIGH.Category_Array_Mask) == WFLAGS_HIGH.Category_Array;
public bool IsStringOrArray => HasComponentSize;
public ushort ComponentSize => HasComponentSize ? ComponentSizeBits : (ushort)0;
public bool HasInstantiation => !TestFlagWithMask(WFLAGS_LOW.GenericsMask, WFLAGS_LOW.GenericsMask_NonGeneric);
public bool ContainsGCPointers => GetFlag(WFLAGS_HIGH.ContainsGCPointers) != 0;
public bool IsDynamicStatics => GetFlag(WFLAGS2_ENUM.DynamicStatics) != 0;
}

[Flags]
internal enum EEClassOrCanonMTBits
{
EEClass = 0,
CanonMT = 1,
Mask = 1,
}
}
```

Internally the contract has a `MethodTable_1` struct that depends on the `MethodTable` data descriptor

```csharp
internal struct MethodTable_1
{
internal RuntimeTypeSystem_1.MethodTableFlags Flags { get; }
internal ushort NumInterfaces { get; }
internal ushort NumVirtuals { get; }
internal TargetPointer ParentMethodTable { get; }
internal TargetPointer Module { get; }
internal TargetPointer EEClassOrCanonMT { get; }
internal MethodTable_1(Data.MethodTable data)
{
Flags = new RuntimeTypeSystem_1.MethodTableFlags
{
MTFlags = data.MTFlags,
MTFlags2 = data.MTFlags2,
BaseSize = data.BaseSize,
};
NumInterfaces = data.NumInterfaces;
NumVirtuals = data.NumVirtuals;
EEClassOrCanonMT = data.EEClassOrCanonMT;
Module = data.Module;
ParentMethodTable = data.ParentMethodTable;
}
}
```

The contract depends on the global pointer value `FreeObjectMethodTablePointer`.
The contract additionally depends on the `EEClass` data descriptor.

```csharp
private readonly Dictionary<TargetPointer, MethodTable_1> _methodTables;

internal TargetPointer FreeObjectMethodTablePointer {get; }

public MethodTableHandle GetMethodTableHandle(TargetPointer methodTablePointer)
{
... // validate that methodTablePointer points to something that looks like a MethodTable.
... // read Data.MethodTable from methodTablePointer.
... // create a MethodTable_1 and add it to _methodTables.
return MethodTableHandle { Address = methodTablePointer }
}

internal static EEClassOrCanonMTBits GetEEClassOrCanonMTBits(TargetPointer eeClassOrCanonMTPtr)
{
return (EEClassOrCanonMTBits)(eeClassOrCanonMTPtr & (ulong)EEClassOrCanonMTBits.Mask);
}

public uint GetBaseSize(MethodTableHandle methodTableHandle) => _methodTables[methodTableHandle.Address].Flags.BaseSize;

public uint GetComponentSize(MethodTableHandle methodTableHandle) => GetComponentSize(_methodTables[methodTableHandle.Address]);

private TargetPointer GetClassPointer(MethodTableHandle methodTableHandle)
{
... // if the MethodTable stores a pointer to the EEClass, return it
// otherwise the MethodTable stores a pointer to the canonical MethodTable
// in that case, return the canonical MethodTable's EEClass.
// Canonical MethodTables always store an EEClass pointer.
}

private Data.EEClass GetClassData(MethodTableHandle methodTableHandle)
{
TargetPointer eeClassPtr = GetClassPointer(methodTableHandle);
... // read Data.EEClass data from eeClassPtr
}


public TargetPointer GetCanonicalMethodTable(MethodTableHandle methodTableHandle) => GetClassData(methodTableHandle).MethodTable;

public TargetPointer GetModule(MethodTableHandle methodTableHandle) => _methodTables[methodTableHandle.Address].Module;
public TargetPointer GetParentMethodTable(MethodTableHandle methodTableHandle) => _methodTables[methodTableHandle.Address].ParentMethodTable;

public bool IsFreeObjectMethodTable(MethodTableHandle methodTableHandle) => FreeObjectMethodTablePointer == methodTableHandle.Address;

public bool IsString(MethodTableHandle methodTableHandle) => _methodTables[methodTableHandle.Address].Flags.IsString;
public bool ContainsGCPointers(MethodTableHandle methodTableHandle) => _methodTables[methodTableHandle.Address].Flags.ContainsGCPointers;

public uint GetTypeDefToken(MethodTableHandle methodTableHandle)
{
MethodTable_1 methodTable = _methodTables[methodTableHandle.Address];
return (uint)(methodTable.Flags.GetTypeDefRid() | ((int)TableIndex.TypeDef << 24));
}

public ushort GetNumMethods(MethodTableHandle methodTableHandle) => GetClassData(methodTableHandle).NumMethods;

public ushort GetNumInterfaces(MethodTableHandle methodTableHandle) => _methodTables[methodTableHandle.Address].NumInterfaces;

public uint GetTypeDefTypeAttributes(MethodTableHandle methodTableHandle) => GetClassData(methodTableHandle).CorTypeAttr;

public bool IsDynamicStatics(MethodTableHandle methodTableHandle) => _methodTables[methodTableHandle.Address].Flags.IsDynamicStatics;
```
1 change: 1 addition & 0 deletions docs/project/list-of-diagnostics.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ The PR that reveals the implementation of the `<IncludeInternalObsoleteAttribute
| __`SYSLIB0053`__ | AesGcm should indicate the required tag size for encryption and decryption. Use a constructor that accepts the tag size. |
| __`SYSLIB0054`__ | Thread.VolatileRead and Thread.VolatileWrite are obsolete. Use Volatile.Read or Volatile.Write respectively instead. |
| __`SYSLIB0055`__ | The underlying hardware instruction does not perform a signed saturate narrowing operation, and it always returns an unsigned result. Use the unsigned overload instead. |
| __`SYSLIB0057`__ | Loading certificate data through the constructor or Import is obsolete. Use X509CertificateLoader instead to load certificates. |

## Analyzer Warnings

Expand Down
1 change: 1 addition & 0 deletions docs/workflow/ci/pr-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Anyone with write access can merge a pull request manually when the following co
* The PR has been approved by at least one reviewer and any other objections are addressed.
* You can request another review from the original reviewer.
* The PR successfully builds and passes all tests in the Continuous Integration (CI) system. In case of failures, refer to the [analyzing build failures](failure-analysis.md) doc.
* The CI results are no more than 1 week old.

Typically, PRs are merged as one commit (squash merges). It creates a simpler history than a Merge Commit. "Special circumstances" are rare, and typically mean that there are a series of cleanly separated changes that will be too hard to understand if squashed together, or for some reason we want to preserve the ability to disect them.

Expand Down
26 changes: 9 additions & 17 deletions docs/workflow/requirements/linux-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,23 +22,15 @@ Minimum RAM required to build is 1GB. The build is known to fail on 512 MB VMs (

### Toolchain Setup

Install the following packages for the toolchain:

* CMake 3.20 or newer
* llvm
* lld
* clang
* build-essential
* python-is-python3
* curl
* git
* lldb
* libicu-dev
* liblttng-ust-dev
* libssl-dev
* libkrb5-dev
* zlib1g-dev
* ninja-build (optional, enables building native code with ninja instead of make)
Install the packages listed in [debian-reqs.txt](/eng/debian-reqs.txt).

You can install all the above dependencies by running

```bash
sudo ./eng/install-native-dependencies.sh
```

### Community-supported environments

**NOTE**: If you have an Ubuntu version older than 22.04 LTS, or Debian version older than 12, don't install `cmake` using `apt` directly. Follow the note written down below.

Expand Down
10 changes: 10 additions & 0 deletions eng/DotNetBuild.props
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,16 @@
<InnerBuildArgs Condition="'$(SourceBuiltAssetManifestsDir)' != ''">$(InnerBuildArgs) /p:SourceBuiltAssetManifestsDir=$(SourceBuiltAssetManifestsDir)</InnerBuildArgs>
<InnerBuildArgs Condition="'$(SourceBuiltSymbolsDir)' != ''">$(InnerBuildArgs) /p:SourceBuiltSymbolsDir=$(SourceBuiltSymbolsDir)</InnerBuildArgs>
<InnerBuildArgs Condition="'$(GitHubRepositoryName)' != ''">$(InnerBuildArgs) /p:GitHubRepositoryName=$(GitHubRepositoryName)</InnerBuildArgs>

<!-- Handle system libraries -->
<UseSystemLibs Condition="'$(UseSystemLibs)' != ''">+$(UseSystemLibs)+</UseSystemLibs>
<InnerBuildArgs Condition="$(UseSystemLibs.Contains('+brotli+'))">$(InnerBuildArgs) --cmakeargs -DCLR_CMAKE_USE_SYSTEM_BROTLI=true</InnerBuildArgs>
<InnerBuildArgs Condition="$(UseSystemLibs.Contains('+libunwind+'))">$(InnerBuildArgs) --cmakeargs -DCLR_CMAKE_USE_SYSTEM_LIBUNWIND=true</InnerBuildArgs>
<!-- TODO: llvm-libunwind -->
<!-- TODO: LinuxTracepoints -->
<InnerBuildArgs Condition="$(UseSystemLibs.Contains('+rapidjson+'))">$(InnerBuildArgs) --cmakeargs -DCLR_CMAKE_USE_SYSTEM_RAPIDJSON=true</InnerBuildArgs>
<InnerBuildArgs Condition="$(UseSystemLibs.Contains('+zlib+'))">$(InnerBuildArgs) --cmakeargs -DCLR_CMAKE_USE_SYSTEM_ZLIB=true</InnerBuildArgs>

</PropertyGroup>
</Target>

Expand Down
2 changes: 1 addition & 1 deletion eng/Subsets.props
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@
<PropertyGroup>
<!-- CLR NativeAot only builds in a subset of the matrix -->
<_NativeAotSupportedOS Condition="'$(TargetOS)' == 'windows' or '$(TargetOS)' == 'linux' or '$(TargetOS)' == 'osx' or '$(TargetOS)' == 'maccatalyst' or '$(TargetOS)' == 'iossimulator' or '$(TargetOS)' == 'ios' or '$(TargetOS)' == 'tvossimulator' or '$(TargetOS)' == 'tvos' or '$(TargetOS)' == 'freebsd'">true</_NativeAotSupportedOS>
<_NativeAotSupportedArch Condition="'$(TargetArchitecture)' == 'x64' or '$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'arm' or ('$(TargetOS)' == 'windows' and '$(TargetArchitecture)' == 'x86')">true</_NativeAotSupportedArch>
<_NativeAotSupportedArch Condition="'$(TargetArchitecture)' == 'x64' or '$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'arm' or '$(TargetArchitecture)' == 'loongarch64' or ('$(TargetOS)' == 'windows' and '$(TargetArchitecture)' == 'x86')">true</_NativeAotSupportedArch>
<NativeAotSupported Condition="'$(_NativeAotSupportedOS)' == 'true' and '$(_NativeAotSupportedArch)' == 'true'">true</NativeAotSupported>
<UseNativeAotForComponents Condition="'$(NativeAotSupported)' == 'true' and '$(TargetOS)' == '$(HostOS)' and ('$(TargetOS)' != 'windows' or '$(TargetArchitecture)' != 'x86') and '$(TargetsLinuxBionic)' != 'true' and ('$(TargetsLinuxMusl)' != 'true' or '$(TargetArchitecture)' != 'arm')">true</UseNativeAotForComponents>

Expand Down
16 changes: 8 additions & 8 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
Expand Up @@ -360,17 +360,17 @@
<Uri>https://github.com/dotnet/runtime-assets</Uri>
<Sha>ec2da34cd7e31a605d6f30f02021a8d76947c99d</Sha>
</Dependency>
<Dependency Name="Microsoft.Net.Compilers.Toolset" Version="4.11.0-3.24329.1">
<Dependency Name="Microsoft.Net.Compilers.Toolset" Version="4.12.0-1.24355.3">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>92051d4c24bc13ff58232104a647910bf22cd105</Sha>
<Sha>19b5e961ecb97b008106f1b646c077e0bffde4a7</Sha>
</Dependency>
<Dependency Name="Microsoft.CodeAnalysis" Version="4.11.0-3.24329.1">
<Dependency Name="Microsoft.CodeAnalysis" Version="4.12.0-1.24355.3">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>92051d4c24bc13ff58232104a647910bf22cd105</Sha>
<Sha>19b5e961ecb97b008106f1b646c077e0bffde4a7</Sha>
</Dependency>
<Dependency Name="Microsoft.CodeAnalysis.CSharp" Version="4.11.0-3.24329.1">
<Dependency Name="Microsoft.CodeAnalysis.CSharp" Version="4.12.0-1.24355.3">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>92051d4c24bc13ff58232104a647910bf22cd105</Sha>
<Sha>19b5e961ecb97b008106f1b646c077e0bffde4a7</Sha>
</Dependency>
<Dependency Name="Microsoft.CodeAnalysis.Analyzers" Version="3.11.0-beta1.24324.1">
<Uri>https://github.com/dotnet/roslyn-analyzers</Uri>
Expand All @@ -381,9 +381,9 @@
<Sha>43709af7570da7140fb3e9a5237f55ffb24677e7</Sha>
</Dependency>
<!-- Intermediate is necessary for source build. -->
<Dependency Name="Microsoft.SourceBuild.Intermediate.roslyn" Version="4.11.0-3.24329.1">
<Dependency Name="Microsoft.SourceBuild.Intermediate.roslyn" Version="4.12.0-1.24355.3">
<Uri>https://github.com/dotnet/roslyn</Uri>
<Sha>92051d4c24bc13ff58232104a647910bf22cd105</Sha>
<Sha>19b5e961ecb97b008106f1b646c077e0bffde4a7</Sha>
<SourceBuild RepoName="roslyn" ManagedOnly="true" />
</Dependency>
<Dependency Name="Microsoft.DotNet.ApiCompat.Task" Version="9.0.100-preview.7.24323.5">
Expand Down
6 changes: 3 additions & 3 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@
Any tools that contribute to the design-time experience should use the MicrosoftCodeAnalysisVersion_LatestVS property above to ensure
they do not break the local dev experience.
-->
<MicrosoftCodeAnalysisCSharpVersion>4.11.0-3.24329.1</MicrosoftCodeAnalysisCSharpVersion>
<MicrosoftCodeAnalysisVersion>4.11.0-3.24329.1</MicrosoftCodeAnalysisVersion>
<MicrosoftNetCompilersToolsetVersion>4.11.0-3.24329.1</MicrosoftNetCompilersToolsetVersion>
<MicrosoftCodeAnalysisCSharpVersion>4.12.0-1.24355.3</MicrosoftCodeAnalysisCSharpVersion>
<MicrosoftCodeAnalysisVersion>4.12.0-1.24355.3</MicrosoftCodeAnalysisVersion>
<MicrosoftNetCompilersToolsetVersion>4.12.0-1.24355.3</MicrosoftNetCompilersToolsetVersion>
</PropertyGroup>
<!--
For source generator support we need to target multiple versions of Roslyn in order to be able to run on older versions of Roslyn.
Expand Down
19 changes: 19 additions & 0 deletions eng/debian-reqs.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
build-essential
clang
cmake
curl
gettext
git
libicu-dev
libkrb5-dev
liblldb-dev
liblttng-ust-dev
libssl-dev
libunwind8-dev
lld
lldb
llvm
locales
ninja-build
python-is-python3
zlib1g-dev
Loading

0 comments on commit 9388349

Please sign in to comment.