MetadataLoadContext to support dynamic assemblies

[steveharter]

As part of the review for feature #15033 and PR dotnet/corefx#33201, there was concern about the lack of support for assemblies not backed by a physical file.

From the comments @jkotas:

There is no way to load runtime loaded assembly into MetadataLoadContext that it not backed by a physical file. The runtime has AssemblyExtensions.TryGetRawMetadata API to handle this case, but there is no way to give the blob returned by TryGetRawMetadata to MetadataLoadContext. The assemblies not backed by physical file are rare today, but they are very likely going to become more common as part of the single-file project.
I am wondering whether there should be a LoadFrom method that takes MetadataReaderProvider to address both these problems and/or whether we should expose a resolver that can resolve the runtime loaded assemblies for you.

Implementation\design TBD. One approach is to use runtime reflection over the dynamic assembly, and convert those from System.RuntimeType to the MetadataLoadContext types (System.RoType).

[steveharter]

Moving to future.

[steveharter]

There have been no customer asks for this so far, although the scenario is valid.

Due to 5.0 schedule constraints, this is being moved to Future.

[slang25]

I would really like support for this 🙂. I want to load some dynamic assemblies that are built in memory with S.R.Emit, and then load them into MetadataLoadContext for unit testing my load context reflection logic.

[steveharter]

Per offline discussion, the scenario for "dynamic assemblies" can be scoped to being able to use MLC to inspect a single-file app which contains embedded assemblies as resource blobs. To support this there are two general options:

1. Add a helper method (like the existing GetRawMetadata()) to the runtime to obtain the assembly(s) from a single file app as blob(s). These then should be able to be passed to MLC.
2. Add a method to MLC that understands single-file apps. This could layer on (1).

I am wondering whether there should be a LoadFrom method that takes MetadataReaderProvider

Exposing an a MLC.LoadFrom(MetadataReaderProvider) seems straightforward. MLC does already have LoadFromStream and
LoadFromByteArray however (not quite as flexible).

Some prototype thoughts:

• Bridge between GetRawMetadata() and MetadataReader - what is needed to use that blob in the reader?
• Does the blob from GetRawMetadata have the assembly references? Update: No it does not. AssemblyBuilder.Save() would be necessary or a larger feature to be able to consume the in-memory metadata from MLC.
• Does the output from AssemblyBuilder support GetRawMetadata()?

and/or whether we should expose a resolver that can resolve the runtime loaded assemblies for you.

I assume this means an addition to or a replacement for the path-based resolver that we already have to help with dynamically produced assemblies referencing each other without physical files?

I want to load some dynamic assemblies that are built in memory with S.R.Emit, and then load them into
MetadataLoadContext for unit testing my load context reflection logic.

Note that GetRawMetadata() does not work with AssemlbyBuilder output. Supporting AssemblyBuilder.Save() would also address the scenario.

[steveharter]

Moving to future based on priority + schedule. Currently there are no community asks for this.

@vitek-karas do you have thoughts on enabling MLC to work on single-file apps?

[vitek-karas]

I must admit that I don't understand how "dynamic assemblies" relate to single-file, other than both don't have a backing file.

So far we've been trying to avoid introducing APIs to read from the single-file. The main reason being that there are effectively 3 or 4 implementations of single-file already and each is likely to behave differently:

• The single-file feature which can be trigged by specifying /p:PublishSinfleFile=true
• Android app target - which is packaged as a bundle as well (but using different mechanisms)
• iOS app target - which I'm not that familiar with, but it effectively acts similarly
• Blazor WASM - to a degree the assemblies here don't have true file paths either - and I don't know if it's possible to read them as byte streams

On top of that there are full AOT scenarios (NativeAOT, WASM AOT, Android AOT, iOS AOT) where there's no IL to begin with.

On top of that, we're hesitant to allow access to the bundle parts as that would directly lead to requests to allow bundling of random content/data files and access to those - which would effectively create another API to access data alongside of existing File APIs and ResourceManager APIs. And more importantly we're trying to get to a place where well written code would work in single-file or multi-file deployments without changes.

To that end the GetRawMetadata approach feels like the best option:

• It already exists as an API and works for file based assemblies and probably also for single-file assemblies (I didn't try this)
• It is scoped to just assemblies so has not chance of spreading to other content
• It could be made to work even on dynamic assemblies if we really wanted to (do Save behind the scenes and return the metadata part of that stream) - but that's a long shot.

It would still not solve full AOT support - but that is likely never going to work.

[steveharter]

I would really like support for this 🙂. I want to load some dynamic assemblies that are built in memory with S.R.Emit, and then load them into MetadataLoadContext for unit testing my load context reflection logic.

If we implement AssemblyBuilder.Save() it would be possible to load those into MLC.