ClearScript V8 Running as a Calculation Engine

[ninlar]

We are allowing users to write their own custom calculations using JavaScript. It would be typical for the user to have a few unique scripts, but several thousand calculations instanced off of those scripts.

For example, they might have thousands of data streams that they want to run a script against every time a new event arrives for a particular data stream. So, the script remains the same, however, different parameters are passed to the script such as the stream Id. In addition, it is common for the scripts to use our client to fetch events for those streams from the database. So there is a combination of network I/O and CPU bound logic.

I'm trying to think of the most efficient way to invoke potentially thousands of these calculations every minute. I probably don't want to have to use a new V8ScriptEngine. It is probably better to compile once, then invoke the function over and over with the different arguments for each calculation. That being said, if one calculation writes data to a global object, another instance could overwrite it, so I they should probably be isolated. Is there a way to snapshot a V8ScriptEngine and quickly load it back into memory, so that each calculation has its own isolated context.

Second, how do I prevent the thousands of instances from blocking threads while waiting for network I/O? Our clients are all async methods that we expose as host objects; however, calling engine.Invoke to run the function is a blocking call. Would it be better to call Invoke inside of a Task.Run? I want the threads in the thread pool to yield to other calculations while they are waiting for the clients to receive a response.

Any ideas on these two items?

[ClearScriptLib]

Hi @ninlar,

I probably don't want to have to use a new V8ScriptEngine. It is probably better to compile once, then invoke the function over and over with the different arguments for each calculation.

Yes. If a script is to be executed many times, spinning up a new engine for each invocation is almost always the wrong way to go. Invoking an existing script function is the fastest and often the best way.

That being said, if one calculation writes data to a global object, another instance could overwrite it, so I they should probably be isolated.

Using separate engines is the only way to prevent scripts from sharing a global object. With V8, you have several options:

1. Use engines with private runtimes. These are expensive but can execute code concurrently.
2. Use engines that share a runtime. These are much less costly, but only one can execute code at a time.
3. Use a mix of strategies to tailor script performance and resource usage for your application.

For (1), simply construct V8ScriptEngine via new. For (2), construct V8Runtime via new, then call CreateScriptEngine as necessary.

Is there a way to snapshot a V8ScriptEngine and quickly load it back into memory, so that each calculation has its own isolated context.

No, but with a shared runtime, extra V8ScriptEngine instances are relatively cheap.

Second, how do I prevent the thousands of instances from blocking threads while waiting for network I/O?

Your application architecture appears to be fairly sophisticated, and it's difficult to make specific recommendations without knowing the details. So here are some general notes:

• Use .NET's thread pool if you can, but keep in mind V8's concurrency characteristics (see above). Server applications sometimes try to maximize script execution concurrency by maintaining a pool of engines in parallel with the thread pool, but that setup doesn't guarantee that each script will always be executed in the same engine, so scripts can't rely on their modifications to the global object.

• ClearScript optionally supports async interoperability via automatic conversion between .NET tasks and JavaScript promises (see V8ScriptEngineFlags.EnableTaskPromiseConversion. Keep in mind that async JavaScript allows one engine to execute multiple concurrent operations in an interleaved manner.

• If you don't want script code to block threads, don't expose thread-blocking .NET facilities such as synchronous I/O. JavaScript and .NET both support asynchronous techniques.

Please let us know if you have additional questions or thoughts. Thanks!

[ninlar]

Thank you so much for this feedback and insight. I originally tried to tackle this problem with a mixed-mode C++/CLI library around ChakraCore before Edge dropped it. I am amazed at how easy it is to make an object available in script. I think the automatic Task to Promise conversions are brilliant. The runtime constraints are also so useful for us to avoid the "noisy" neighbor problem of one calculation consuming all the resources. This ClearScript.V8 library is absolutely amazing and well designed. Kudos to all of you.

[ninlar]

Oh yeah and the use of dynamic dispatch on the Script object to easily get a reference to a function or object is top-notch. The design of this library is phenomenal.

[ClearScriptLib]

Thank you very much for your kind words! 😊