[mono] Assembly load failures in mono_class_is_subclass_of_internal

[uweigand]

Description

Using a Mono runtime based "dotnet" host to build the runtime libs.tests target currently fails for with an error building the System.Reflection.Metadata.ApplyUpdate test cases.

The actual command that fails is the following (after the fix for #54814 is applied):

uweigand@m8345019:~/runtime$ dotnet ~/.nuget/packages/microsoft.dotnet.hotreload.utils.generator.buildtool/1.0.1-alpha.0.21314.1/tools/net6.0/Microsoft.DotNet.HotReload.Utils.Generator.BuildTool.dll -msbuild:src/libraries/System.Runtime.Loader/tests/ApplyUpdate/System.Reflection.Metadata.ApplyUpdate.Test.MethodBody1/System.Reflection.Metadata.ApplyUpdate.Test.MethodBody1.csproj -script:src/libraries/System.Runtime.Loader/tests/ApplyUpdate/System.Reflection.Metadata.ApplyUpdate.Test.MethodBody1/deltascript.json
Unhandled Exception:
System.AggregateException: One or more errors occurred. (Could not load type of field 'Microsoft.CodeAnalysis.SQLite.Interop.SafeSqliteBlobHandle:_wrapper' (0) due to: Could not load file or assembly 'SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535' or one of its dependencies.)
 ---> System.TypeLoadException: Could not load type of field 'Microsoft.CodeAnalysis.SQLite.Interop.SafeSqliteBlobHandle:_wrapper' (0) due to: Could not load file or assembly 'SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535' or one of its dependencies.

Configuration

I'm building natively on s390x-ibm-linux, using a .NET 6 Preview5 host toolchain rebuilt for s390x from the official sources (plus some extra patches required to support s390x - these are mostly backports from mainline plus pending PRs). Note that this toolchain is based on the Mono runtime as we don't support CoreCLR on s390x.

Regression?

It is a regression in the sense that the libs.tests target used to build previously. But that is caused simply by the fact that the ApplyUpdates test exposing the problem was recently added. The underlying problem in the Mono loader was there previously.

Other information

It appears true that SQLitePCLRaw.core cannot be found in the current application context. However, I understand that the way the "wrapper" type is used in Microsoft.CodeAnalysis.SQLite.Interop.SafeSqliteBlobHandle is intended to allow for that case; the type is supposed to be allowed to be undefined until the wrapper is actually used.

The immediate reason for the failure is the following Mono call chain:

Thread 1 "dotnet" hit Breakpoint 1, mono_error_set_simple_file_not_found (error=0x3ffffffa000,
    file_name=0x2aa008eb7f0 "SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535") at /home/uweigand/runtime/src/mono/mono/metadata/exception.c:1459
1459            mono_error_set_file_not_found (error, file_name, "Could not load file or assembly '%s' or one of its dependencies.", file_name);
(gdb) bt
#0  mono_error_set_simple_file_not_found (error=0x3ffffffa000, file_name=0x2aa008eb7f0 "SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535")
    at /home/uweigand/runtime/src/mono/mono/metadata/exception.c:1459
#1  0x000003fffcfc2d20 in mono_class_from_typeref_checked (image=0x2aa00274330, type_token=16777622, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/class.c:213
#2  0x000003fffcfcb30a in mono_class_get_checked (image=0x2aa00274330, type_token=16777622, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/class.c:2877
#3  0x000003fffd078bb0 in do_mono_metadata_parse_type (type=0x3ffffff9b60, m=0x2aa00274330, container=0x0, transient=0,
    ptr=0x3fff9146690 "\004\006\021\207P\004\006\022\206M\004\006\022\206Q\004\006\022\217\210\t\006\025\022\201\314\001\022\217x\004\006\021\205\304\004\006\022\205\310\v\006\025\022\201!\002\022\031\022\200\315\t\006\025\022\212\f\001\021\205\304\a\006\025\022\212\f\001\002\004\006\021\205\330\004\006\021\205\360\a\006\025\022\202M\001\016\t\006\025\022\212\024\001\021\205\330\f\006\025\022\212\020\001\025\021\200\371\001\002\t\006\025\022\204\320\001\022\232\004\a\006\025\022\212\020\001\002\t\006\025\022\212\024\001\022\220\374\n\006\025\022\201!\002\021\202=\016\n\006\025\022\201!\002\016\021\202=\004\006\022\206a\a\006\037\202\341\022\206P\f\006\025\022\201\201\002\016\025\021!\001\016\004\006\022\206P\t\006\025\022\203\031\003\016"..., rptr=0x3ffffff9b80, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/metadata.c:3906
#4  0x000003fffd06adce in mono_metadata_parse_type_internal (m=0x2aa00274330, container=0x0, opt_attrs=33, transient=0,
    ptr=0x3fff914668d "\022\206Y\004\006\021\207P\004\006\022\206M\004\006\022\206Q\004\006\022\217\210\t\006\025\022\201\314\001\022\217x\004\006\021\205\304\004\006\022\205\310\v\006\025\022\201!\002\022\031\022\200\315\t\006\025\022\212\f\001\021\205\304\a\006\025\022\212\f\001\002\004\006\021\205\330\004\006\021\205\360\a\006\025\022\202M\001\016\t\006\025\022\212\024\001\021\205\330\f\006\025\022\212\020\001\025\021\200\371\001\002\t\006\025\022\204\320\001\022\232\004\a\006\025\022\212\020\001\002\t\006\025\022\212\024\001\022\220\374\n\006\025\022\201!\002\021\202=\016\n\006\025\022\201!\002\016\021\202=\004\006\022\206a\a\006\037\202\341\022\206P\f\006\025\022\201\201\002\016\025\021!\001\016\004\006\022\206P\t\006\025\022\203"..., rptr=0x3ffffff9e00, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/metadata.c:2096
#5  0x000003fffd06aa10 in mono_metadata_parse_type_checked (m=0x2aa00274330, container=0x0, opt_attrs=33, transient=0,
    ptr=0x3fff914668d "\022\206Y\004\006\021\207P\004\006\022\206M\004\006\022\206Q\004\006\022\217\210\t\006\025\022\201\314\001\022\217x\004\006\021\205\304\004\006\022\205\310\v\006\025\022\201!\002\022\031\022\200\315\t\006\025\022\212\f\001\021\205\304\a\006\025\022\212\f\001\002\004\006\021\205\330\004\006\021\205\360\a\006\025\022\202M\001\016\t\006\025\022\212\024\001\021\205\330\f\006\025\022\212\020\001\025\021\200\371\001\002\t\006\025\022\204\320\001\022\232\004\a\006\025\022\212\020\001\002\t\006\025\022\212\024\001\022\220\374\n\006\025\022\201!\002\021\202=\016\n\006\025\022\201!\002\016\021\202=\004\006\022\206a\a\006\037\202\341\022\206P\f\006\025\022\201\201\002\016\025\021!\001\016\004\006\022\206P\t\006\025\022\203"..., rptr=0x3ffffff9e00, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/metadata.c:2150
#6  0x000003fffcfd1510 in mono_field_resolve_type (field=0x2aa008e99a0, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/class.c:6416
#7  0x000003fffcfd6d28 in mono_class_setup_fields (klass=0x2aa006e20a8) at /home/uweigand/runtime/src/mono/mono/metadata/class-init.c:322
#8  0x000003fffcfe158e in init_sizes_with_info (klass=0x2aa006e20a8, cached_info=0x0) at /home/uweigand/runtime/src/mono/mono/metadata/class-init.c:1754
#9  0x000003fffcfd74d4 in mono_class_init_internal (klass=0x2aa006e20a8) at /home/uweigand/runtime/src/mono/mono/metadata/class-init.c:2804
#10 0x000003fffcfccbba in mono_class_is_subclass_of_internal (klass=0x2aa006e20a8, klassc=0x2aa000d5b98, check_interfaces=0)
    at /home/uweigand/runtime/src/mono/mono/metadata/class.c:3526
#11 0x000003fffd0053b0 in ves_icall_RuntimeTypeHandle_is_subclass_of (childType=0x2aa006e2158, baseType=0x2aa000d5c48) at /home/uweigand/runtime/src/mono/mono/metadata/icall.c:1763

where the is_subclass_of intrinsic is called from managed code here:

System.RuntimeTypeHandle:IsSubclassOf (System.RuntimeType,System.RuntimeType) [{0x2aa001a3810} + 0x92] [/home/uweigand/runtime/src/mono/System.Private.CoreLib/src/System/RuntimeTypeHandle.cs :: 254u]
System.RuntimeType:IsSubclassOf (System.Type) [{0x2aa000ed2d0} + 0x122] [/home/uweigand/runtime/src/mono/System.Private.CoreLib/src/System/RuntimeType.Mono.cs :: 2243u] (0x3fffdfb9af0 0x3fffdfb9c60) 
System.Type:get_IsValueType () [{0x2aa001a3378} + 0x50] [/home/uweigand/runtime/src/libraries/System.Private.CoreLib/src/System/Type.cs :: 112u] (0x3fffdfb9958 0x3fffdfb99b8) [0x2aa00077170 - Microso
System.Type:get_IsClass () [{0x2aa006bcd28} + 0x72] [/home/uweigand/runtime/src/libraries/System.Private.CoreLib/src/System/Type.cs :: 83u] (0x3fffca940d0 0x3fffca94180) [0x2aa00077170 - Microsoft.Do
System.Composition.TypedParts.Discovery.TypeInspector:InspectTypeForPart (System.Reflection.TypeInfo,System.Composition.TypedParts.Discovery.DiscoveredPart&) [{0x2aa00693118} + 0xb2]  (0x3fffca93a38 
System.Composition.TypedParts.TypedPartExportDescriptorProvider:.ctor (System.Collections.Generic.IEnumerable`1<System.Type>,System.Composition.Convention.AttributedModelProvider) [{0x2aa00687a38} +
System.Composition.Hosting.ContainerConfiguration:CreateContainer () [{0x2aa00483f30} + 0x1c6]  (0x3fffcaad480 0x3fffcaad828) [0x2aa00077170 - Microsoft.DotNet.HotReload.Utils.Generator.BuildTool.dll
Microsoft.CodeAnalysis.Host.Mef.MefHostServices:Create (System.Collections.Generic.IEnumerable`1<System.Reflection.Assembly>) [{0x2aa005a1ba8} + 0x102]  (0x3fffcaabb90 0x3fffcaabd38) [0x2aa00077170 -

The caller simply wants to invoke IsClass on the type (which internally checks for IsSubtypeOf the object type). In Mono,
this triggers a call to mono_class_init_internal which wants to instantiate all the member types by loading the defining assemblies if necessary. At this point the failure occurs.

This is different with the CoreCLR runtime, where IsSubtypeOf does not trigger any type resolution, and therefore this error does not occur.

Looking at the history of the Mono code, it seems the call to mono_class_init_internal was introduced by @vargaz here: 58255fa in order to fix https://bugzilla.xamarin.com/41/41466/bug.html .

However, that bug only refers to the interface scenario, which doesn't apply in the case where I see the problem. It seems a simple fix could therefore be to restrict the mono_class_init_internal call to the case where check_interfaces is true. This does in fact resolve the problem I'm seeing. I'll submit a PR proposing that fix shortly.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area:
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Using a Mono runtime based "dotnet" host to build the runtime libs.tests target currently fails for with an error building the System.Reflection.Metadata.ApplyUpdate test cases.

The actual command that fails is the following (after the fix for #54814 is applied):

uweigand@m8345019:~/runtime$ dotnet ~/.nuget/packages/microsoft.dotnet.hotreload.utils.generator.buildtool/1.0.1-alpha.0.21314.1/tools/net6.0/Microsoft.DotNet.HotReload.Utils.Generator.BuildTool.dll -msbuild:src/libraries/System.Runtime.Loader/tests/ApplyUpdate/System.Reflection.Metadata.ApplyUpdate.Test.MethodBody1/System.Reflection.Metadata.ApplyUpdate.Test.MethodBody1.csproj -script:src/libraries/System.Runtime.Loader/tests/ApplyUpdate/System.Reflection.Metadata.ApplyUpdate.Test.MethodBody1/deltascript.json
Unhandled Exception:
System.AggregateException: One or more errors occurred. (Could not load type of field 'Microsoft.CodeAnalysis.SQLite.Interop.SafeSqliteBlobHandle:_wrapper' (0) due to: Could not load file or assembly 'SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535' or one of its dependencies.)
 ---> System.TypeLoadException: Could not load type of field 'Microsoft.CodeAnalysis.SQLite.Interop.SafeSqliteBlobHandle:_wrapper' (0) due to: Could not load file or assembly 'SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535' or one of its dependencies.

Configuration

I'm building natively on s390x-ibm-linux, using a .NET 6 Preview5 host toolchain rebuilt for s390x from the official sources (plus some extra patches required to support s390x - these are mostly backports from mainline plus pending PRs). Note that this toolchain is based on the Mono runtime as we don't support CoreCLR on s390x.

Regression?

It is a regression in the sense that the libs.tests target used to build previously. But that is caused simply by the fact that the ApplyUpdates test exposing the problem was recently added. The underlying problem in the Mono loader was there previously.

Other information

It appears true that SQLitePCLRaw.core cannot be found in the current application context. However, I understand that the way the "wrapper" type is used in Microsoft.CodeAnalysis.SQLite.Interop.SafeSqliteBlobHandle is intended to allow for that case; the type is supposed to be allowed to be undefined until the wrapper is actually used.

The immediate reason for the failure is the following Mono call chain:

Thread 1 "dotnet" hit Breakpoint 1, mono_error_set_simple_file_not_found (error=0x3ffffffa000,
    file_name=0x2aa008eb7f0 "SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535") at /home/uweigand/runtime/src/mono/mono/metadata/exception.c:1459
1459            mono_error_set_file_not_found (error, file_name, "Could not load file or assembly '%s' or one of its dependencies.", file_name);
(gdb) bt
#0  mono_error_set_simple_file_not_found (error=0x3ffffffa000, file_name=0x2aa008eb7f0 "SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535")
    at /home/uweigand/runtime/src/mono/mono/metadata/exception.c:1459
#1  0x000003fffcfc2d20 in mono_class_from_typeref_checked (image=0x2aa00274330, type_token=16777622, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/class.c:213
#2  0x000003fffcfcb30a in mono_class_get_checked (image=0x2aa00274330, type_token=16777622, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/class.c:2877
#3  0x000003fffd078bb0 in do_mono_metadata_parse_type (type=0x3ffffff9b60, m=0x2aa00274330, container=0x0, transient=0,
    ptr=0x3fff9146690 "\004\006\021\207P\004\006\022\206M\004\006\022\206Q\004\006\022\217\210\t\006\025\022\201\314\001\022\217x\004\006\021\205\304\004\006\022\205\310\v\006\025\022\201!\002\022\031\022\200\315\t\006\025\022\212\f\001\021\205\304\a\006\025\022\212\f\001\002\004\006\021\205\330\004\006\021\205\360\a\006\025\022\202M\001\016\t\006\025\022\212\024\001\021\205\330\f\006\025\022\212\020\001\025\021\200\371\001\002\t\006\025\022\204\320\001\022\232\004\a\006\025\022\212\020\001\002\t\006\025\022\212\024\001\022\220\374\n\006\025\022\201!\002\021\202=\016\n\006\025\022\201!\002\016\021\202=\004\006\022\206a\a\006\037\202\341\022\206P\f\006\025\022\201\201\002\016\025\021!\001\016\004\006\022\206P\t\006\025\022\203\031\003\016"..., rptr=0x3ffffff9b80, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/metadata.c:3906
#4  0x000003fffd06adce in mono_metadata_parse_type_internal (m=0x2aa00274330, container=0x0, opt_attrs=33, transient=0,
    ptr=0x3fff914668d "\022\206Y\004\006\021\207P\004\006\022\206M\004\006\022\206Q\004\006\022\217\210\t\006\025\022\201\314\001\022\217x\004\006\021\205\304\004\006\022\205\310\v\006\025\022\201!\002\022\031\022\200\315\t\006\025\022\212\f\001\021\205\304\a\006\025\022\212\f\001\002\004\006\021\205\330\004\006\021\205\360\a\006\025\022\202M\001\016\t\006\025\022\212\024\001\021\205\330\f\006\025\022\212\020\001\025\021\200\371\001\002\t\006\025\022\204\320\001\022\232\004\a\006\025\022\212\020\001\002\t\006\025\022\212\024\001\022\220\374\n\006\025\022\201!\002\021\202=\016\n\006\025\022\201!\002\016\021\202=\004\006\022\206a\a\006\037\202\341\022\206P\f\006\025\022\201\201\002\016\025\021!\001\016\004\006\022\206P\t\006\025\022\203"..., rptr=0x3ffffff9e00, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/metadata.c:2096
#5  0x000003fffd06aa10 in mono_metadata_parse_type_checked (m=0x2aa00274330, container=0x0, opt_attrs=33, transient=0,
    ptr=0x3fff914668d "\022\206Y\004\006\021\207P\004\006\022\206M\004\006\022\206Q\004\006\022\217\210\t\006\025\022\201\314\001\022\217x\004\006\021\205\304\004\006\022\205\310\v\006\025\022\201!\002\022\031\022\200\315\t\006\025\022\212\f\001\021\205\304\a\006\025\022\212\f\001\002\004\006\021\205\330\004\006\021\205\360\a\006\025\022\202M\001\016\t\006\025\022\212\024\001\021\205\330\f\006\025\022\212\020\001\025\021\200\371\001\002\t\006\025\022\204\320\001\022\232\004\a\006\025\022\212\020\001\002\t\006\025\022\212\024\001\022\220\374\n\006\025\022\201!\002\021\202=\016\n\006\025\022\201!\002\016\021\202=\004\006\022\206a\a\006\037\202\341\022\206P\f\006\025\022\201\201\002\016\025\021!\001\016\004\006\022\206P\t\006\025\022\203"..., rptr=0x3ffffff9e00, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/metadata.c:2150
#6  0x000003fffcfd1510 in mono_field_resolve_type (field=0x2aa008e99a0, error=0x3ffffffa000) at /home/uweigand/runtime/src/mono/mono/metadata/class.c:6416
#7  0x000003fffcfd6d28 in mono_class_setup_fields (klass=0x2aa006e20a8) at /home/uweigand/runtime/src/mono/mono/metadata/class-init.c:322
#8  0x000003fffcfe158e in init_sizes_with_info (klass=0x2aa006e20a8, cached_info=0x0) at /home/uweigand/runtime/src/mono/mono/metadata/class-init.c:1754
#9  0x000003fffcfd74d4 in mono_class_init_internal (klass=0x2aa006e20a8) at /home/uweigand/runtime/src/mono/mono/metadata/class-init.c:2804
#10 0x000003fffcfccbba in mono_class_is_subclass_of_internal (klass=0x2aa006e20a8, klassc=0x2aa000d5b98, check_interfaces=0)
    at /home/uweigand/runtime/src/mono/mono/metadata/class.c:3526
#11 0x000003fffd0053b0 in ves_icall_RuntimeTypeHandle_is_subclass_of (childType=0x2aa006e2158, baseType=0x2aa000d5c48) at /home/uweigand/runtime/src/mono/mono/metadata/icall.c:1763

where the is_subclass_of intrinsic is called from managed code here:

System.RuntimeTypeHandle:IsSubclassOf (System.RuntimeType,System.RuntimeType) [{0x2aa001a3810} + 0x92] [/home/uweigand/runtime/src/mono/System.Private.CoreLib/src/System/RuntimeTypeHandle.cs :: 254u]
System.RuntimeType:IsSubclassOf (System.Type) [{0x2aa000ed2d0} + 0x122] [/home/uweigand/runtime/src/mono/System.Private.CoreLib/src/System/RuntimeType.Mono.cs :: 2243u] (0x3fffdfb9af0 0x3fffdfb9c60) 
System.Type:get_IsValueType () [{0x2aa001a3378} + 0x50] [/home/uweigand/runtime/src/libraries/System.Private.CoreLib/src/System/Type.cs :: 112u] (0x3fffdfb9958 0x3fffdfb99b8) [0x2aa00077170 - Microso
System.Type:get_IsClass () [{0x2aa006bcd28} + 0x72] [/home/uweigand/runtime/src/libraries/System.Private.CoreLib/src/System/Type.cs :: 83u] (0x3fffca940d0 0x3fffca94180) [0x2aa00077170 - Microsoft.Do
System.Composition.TypedParts.Discovery.TypeInspector:InspectTypeForPart (System.Reflection.TypeInfo,System.Composition.TypedParts.Discovery.DiscoveredPart&) [{0x2aa00693118} + 0xb2]  (0x3fffca93a38 
System.Composition.TypedParts.TypedPartExportDescriptorProvider:.ctor (System.Collections.Generic.IEnumerable`1<System.Type>,System.Composition.Convention.AttributedModelProvider) [{0x2aa00687a38} +
System.Composition.Hosting.ContainerConfiguration:CreateContainer () [{0x2aa00483f30} + 0x1c6]  (0x3fffcaad480 0x3fffcaad828) [0x2aa00077170 - Microsoft.DotNet.HotReload.Utils.Generator.BuildTool.dll
Microsoft.CodeAnalysis.Host.Mef.MefHostServices:Create (System.Collections.Generic.IEnumerable`1<System.Reflection.Assembly>) [{0x2aa005a1ba8} + 0x102]  (0x3fffcaabb90 0x3fffcaabd38) [0x2aa00077170 -

The caller simply wants to invoke IsClass on the type (which internally checks for IsSubtypeOf the object type). In Mono,
this triggers a call to mono_class_init_internal which wants to instantiate all the member types by loading the defining assemblies if necessary. At this point the failure occurs.

This is different with the CoreCLR runtime, where IsSubtypeOf does not trigger any type resolution, and therefore this error does not occur.

Looking at the history of the Mono code, it seems the call to mono_class_init_internal was introduced by @vargaz here: 58255fa in order to fix https://bugzilla.xamarin.com/41/41466/bug.html .

However, that bug only refers to the interface scenario, which doesn't apply in the case where I see the problem. It seems a simple fix could therefore be to restrict the mono_class_init_internal call to the case where check_interfaces is true. This does in fact resolve the problem I'm seeing. I'll submit a PR proposing that fix shortly.

Author:	uweigand
Assignees:	-
Labels:	area-VM-meta-mono, in pr, untriaged
Milestone:	-