F# compiler startup is too slow for a code golf website #9061
Comments
|
Just a hunch, but couldn't you use the compiler services for both C# and F#? That way the compiler can stay in memory, since most time will be spent starting up. For timings, compare how |
|
I experience a similar slow startup times with FSI on Ubuntu 19.10 using the .NET 5 preview running this simple script.
@cartermp do you want a seperate issue for that or would the slow startup time of FSI be covered here too? |
|
I also noticed that For Code Golf, running the compiler as a service is certainly an interesting idea. The service would likely need to run outside of the sandboxed environment that scripts normally run in. Then we could compile an assembly outside of the sandbox and run it inside of one. My understanding is that we wouldn't have to worry about security, beyond the normal concerns, because there's no way to write F# code that runs at compile time. Is this true? Is there any documentation for running the compiler as a service that's compatible with running in a .NET Core 3 Linux environment? I would also be very curious to know where all of the time is actually spent during startup. 3 - 7 seconds is a substantial amount of time. |
|
It's absolutely possible to write F# code that runs at compile time, via a mechanism known as Type Providers. These typically have to be referenced via a nuget though, so would be white-listable in that way if you were worried about it. As far as running the compiler service in a .net core 3 linux environment, the existing VS Code support for F# depends entirely on this feature set. The API docs and tutorials for the FSharp.Compiler.Service are available here, and are derived from the code in this repo which also contains samples. |
|
@SirBogman Unfortunately, there's not a lot to go on here. If you can run But here's a small solution with a C# project that only defines a single interface and an F# project that consumes the default interface member: In the latter the F# compiler has to import another assembly (the C# one) but it's still proportional to the Hello World case since there are numerous assemblies to import for a basic .NET Core project. But you can see that there's MSBuild nonsense happening in the latter one that dominates everything else. So the first step is to get a performance summary to see, once MSBuild is involved, where time is spent. The .NET CLI also does have some overhead where it resolves the correct SDK to call into based on the project you have, I don't know how to profile that. Once you've established how much time is spent in Fsc, you can find out exactly where time is being spent is via a trace, which you can collect with dotnet-track and analyze in a tool that can look at ETL traces. This is the only way to profile it reliably. @realvictorprm That script is expected to be slow. FSI has always had slow startup times since it has to do a lot of things at initialization time to accept arbitrary code and compile/execute it quickly. Additionally, |
|
|
It might also be useful to point out that much of the work being done above is unnecessary if your dependency graph didn't change between compiles. In that case, all you have to do is compile, emit a single dll, and run that. This is how Try .NET is as responsive as it is. The example here shows timings which include round-trips to the compilation service, i.e. most of the time is spent in network I/O. (The example is C# because we don't have F# WASM support in place, but the same strategy should apply regardless.)
|
|
Try .NET is pretty cool. I tried compiling with fsc directly, but I have trouble running the resulting program and don't understand why. This seems to be the minimum set of references needed to build a trivial program. If I leave them out, I get this: When I try to run the resulting assembly, I get this: Here's the code: [<EntryPoint>]
let main argv =
printfn "Hello World from F#!"
argv |> Seq.iteri (printfn "Arg %i: %s")
0When I load the assembly in a custom program and try to run it by reflection, I get this: I'm not sure how to address either of these issues. |
|
Thanks for all of the suggestions. I was able to get this to work well enough for my use case. After discussing with the maintainer of the code golf site, we didn't think it was worth the extra complexity to have a compiler service running outside of the sandbox. None of the other languages need that. So I looked into trying to make it a bit faster. I wrote a small application that wrapped the For a trivial script, it initially took 4 seconds, which is too long for the 7 second timeout. At this point I noticed that after my app compiled the script, every .NET Core SDK assembly was loaded in the AppDomain. I thought this might be contributing to the compile time. I noticed there were still a large number of assemblies I felt were unnecessary for code golf and manually removed them. That reduced the time fro 2.5 to 2.15 seconds. At this point, the compile time is acceptable for my use case, although it's not ideal. There were some assemblies that I wanted to remove, but could not because I think they were needed by FSharpCore: System.Net.Requests, and System.Net.WebClient. Here's the set of assemblies I ended up with, after removing everything that didn't cause an error and that I didn't feel would be useful for code golf. The docker image is just over 100 MB. This issue can probably be closed now, although I would still appreciate any speed improvements in the compiler startup process. Perhaps less time could be spent loading assemblies that aren't strictly required? |
|
You could try to ILMerge the assemblies, so that it only has to load a single big one instead of 24 small ones. |
That's an interesting idea. I tried using Can IL merge be used for .NET Core 3.1.200? |
|
F# is now live on code-golf.io! It has slightly slower startup time than any of the other languages, taking about 2 seconds for compiler startup. |
|
If anyone is interested, the code is here for setting up the application containing the F# compiler, trimming it, and creating a container. |
|
Thanks @SirBogman - bit of a shame that startup time before anything ever reaches |
|
F# is a great language. It's definitely not the best for code golf, but it's still fun, as long as you don't do it in production code. I did capture one performance trace with the method you described. I haven't figured out how to look at it yet though :) |
|
Typically the best tool for looking at the trace is with PerfView on Windows. I'm not sure if there are Linux equivalents. |
|
I'll try that. I had tried Windows Performance Analyzer and I couldn't get it to open the file. |
|
@SirBogman if you're on linux you can use |
|
Thanks, I was able to get PerfView to open the file. It looks like a lot of time was spent in JIT but I'm not sure if I was using ReadyToRun at the time. I'll try to capture a new file and see if there's anything interesting in it. |
Yeah, PublishSingleFile packages all the small files into a zip file, then extracts it on startup into temp and runs it normally.
I used https://github.com/gluck/il-repack the last time. It should work for all pure IL assemblies (not native code), but it would need to be executed before Ready2Run. So the pipeline should be |
With ReadyToRun, with Tiered Compilation (default)Of the two second run time to start the wrapper program, compile a trivial program, and run it, a substantial amount of time appears to be spent in JIT, even though the assemblies were all ReadyToRun (except the dynamic golf1 assembly).
Does anyone have thoughts on why so much time is spent in JIT with ready to run? Could it be related to generic type instantiations that are created at runtime? With ReadyToRun, without Tiered CompilationIt appears to save about 100 msec by disabling Tiered Compilation.
Without ReadyToRunWhen I disabled ReadyToRun, it took about 7 seconds to run the program, as I originally reported. It appears that JIT accounts for the added time.
|
|
Closing out old discussion |
Not sure it was suggested here before, but can you ngen your binaries? That should eliminate most of the JIT time. |
Prejitting of methods with tail calls is not yet supported by Ready To Run, see dotnet/runtime#5857. That's the most likely explanation. cc @jkotas. If you look further down in the jit time report it shows all the methods that are jitted. Perhaps that will show some other patterns. |
|
@AndyAyersMS Thanks, that does seem like the most likely explanation. Hopefully it will be implemented eventually. |
I'm trying to add F# to a code golf website. I recently added Java to that site and I'm in the process of adding C#.
I've been having trouble setting up a .NET Core 3 container with the F# compiler that can compile a trivial script within a couple seconds. The site has a seven second time limit for building/running scripts. For Java and C#, I can compile the script within about a second. For F#, it's taking 6 or more seconds to build the script with
dotnet build --no-restore. I had a similar problem with C# was able to save a few seconds by running the compiler directly. I tried running the F# compiler directly and that only takes three seconds, but I'm having trouble running the created assembly. I also made a custom program that embeds the F# compiler and it takes about 6 seconds to compile and run the trivial script.Hopefully I'm doing something wrong. My three attempts at dockerfiles are here. I'm not sure if opening an issue here is appropriate, but I wasn't sure where I could get suggestions for how to fix this.
Is there anything I can do to make this faster? The scripts will be fairly simple and won't need to use any features beyond the core F# library and a few core .NET libraries.
It seems like it should be possible to compile and run a simple program in significantly less than seven seconds, but it's been eluding me.
I'm pretty sure F# would be better for code golf than C# and Java.
The text was updated successfully, but these errors were encountered: