External task threads - how do they work?

[Zortexxx619]

I'm having a hard time wrapping my head around how external task threads work. The examples mention file IO as an example. does registering an IO thread as an external task thread like that allow enkiTS to use the thread for other tasks when it's idle, or am I misunderstanding its purpose?

[dougbinks]

External task threads can be used for a number of different use cases, their primary purpose is to allow the thread to use enkiTS, and for other threads to be able to add pinned tasks for that thread to run.

The IO example is one such use case. IO access tends to be blocking, so if IO is done on a normal enkiTS thread it would prevent other work from being done - the number of active threads would be less than the number of CPU hardware threads (cores + hyperthreading). If we increase the number of enkiTS threads this would lead to oversubscription issues, so by creating an external thread which only checks for pinned tasks we can keep the number of active threads equal to the number of CPU hardware threads.

For disk IO on modern SSDs this probably isn't that useful, but for network IO it is.

So in this case you would not want enkiTS to use that thread when it's idle, since you should already have as many enkiTS threads (when including the main thread) as there are hardware threads.

An alternative use case is when you don't control the active threads but want to perform tasking. For example you want to write a library which uses multithreading which might be used in Unreal Engine or Godot etc. You can have library users call a function mylibrary.DoWork() which when run on a thread acts as an enkiTS external thread and processes tasks. I should add an example for that.

Hope this makes sense, if not let me know.

[Zortexxx619]

Thanks for the quick response. It's a bit hard to follow and I may have misinterpreted, but if I'm understanding correctly, given n HW threads, n enkiTS threads are created in the default case (including the main/calling thread). Then, external threads are created in addition to those, and by registering a thread as external, it effectively suppresses one of the enkiTS threads to prevent oversubscription until it's done.

If that's the case, wouldn't the external thread still block on IO and cause less than n HW threads to be active?

[dougbinks]

if I'm understanding correctly, given n HW threads, n enkiTS threads are created in the default case (including the main/calling thread)

Yes, unless you request a different number of enkiTS threads this is the default.

Then, external threads are created in addition to those

enkiTS does not create the external threads itself, this is something you need to do if you want to use them. This is why they are external threads.

by registering a thread as external, it effectively suppresses one of the enkiTS threads to prevent oversubscription until it's done.

No, this is somewhat complex and so I will add further explanations below.

A given system can only run as many software threads as there are hardware threads (also called logical processors). The OS will schedule the software threads by preemptively stopping and starting threads to share the hardware threads. A number of mechanisms exist for a thread to block it's own execution until some event occurs - such as mutexes, semaphores, sleep etc.

In general for a given process you do not want to have more threads actively running than there hardware threads, as doing so would cause threads to be switched in and out, which eats up CPU time for no benefit.

However, certain forms of IO result in blocking calls which cause the calling thread to be swapped out by the OS until the event the call is for completes. For example listening for network socket IO. If you perform these functions from an enkiTS thread then for the period of time that function blocks you will not be using the full CPU resources available. One solution to this is to create a specific thread for this work, and by making it an enkiTS thread you can communicate with the thread by sending it pinned tasks to perform as per the example. The approach outlined in the WaitForNewPinnedTasks.cpp example does not cause oversubscription because the thread uses the function WaitForNewPinnedTasks which blocks the thread, allowing the OS to run another thread until a new pinned task is sent to that thread.

There will still be some thread switching, however if this happens only infrequently and if the IO calls block for a long period (many ms) then this approach can be beneficial.

[Zortexxx619]

Ah, I see. That cleared it up, thank you.

Does this work for all sources of blocking, or is it only a subset that are detectable?

[dougbinks]

Does this work for all sources of blocking, or is it only a subset that are detectable?

This approach is primarily designed for blocking calls which wait on an event, leading to the OS swapping out the thread. You then need to profile your application to see if it's worth using this method.

As I mentioned earlier the other main use for external threads is when you have a thread not created by enkiTS from which you want to be able to use enkiTS.