More asynchronous compiler

[kitsonk]

Refs #2626

We would like the compiler modules asynchronously, hopefully within the runtime of the compiler to make it easier to manage the flow control inside Rust. While we landed #2949 it might not be the long term answer, or at least if further enhancement happens in Rust, we could still do better in the compiler.

After mentioning this need in microsoft/TypeScript#29361 there was a helpful reply by:

You could do that manually by calling ts.preProcessFile(fileContent) on the main file. From that you get typeReferenceDirectives, referencedFiles and importedFiles that you can then use to async download the referenced files. The files can then again be recursivelly analysed with the preProcessFile. At the end, you get a list of all the files needed and their content. That you can then use in LanguageServiceHost.

So this might help us break apart the flow control a bit more and be able to better manage the resolution of modules before we actually transpile the modules.

[ry]

Great hint! This is what we need to fix the loading problem!

Something I've also been thinking - we could try to see if the V8 compiler is able to extract imported files without dying due to syntax errors. If so it could be much faster than ts.preProcessFile() ... but it's unlikely it would work.