Node.js worker_threads vs child_process.fork performance

[aaronccasanova]

Hello 👋

I'm in the process of prototyping a module for running code transformations and gathering metrics over a large number of text files (anywhere from 5,000 to 50,000). My implementation is heavily inspired by jscodeshift, however, instead of using node:child_process.fork I opt’d for node:worker_threads. When comparing the speed differences between my module and jscodeshift, I found jscodeshift was noticeably faster. After making a few adjustments (such as: batching and sending 50 files at a time to each free worker), I came to the (tentative) conclusion the performance differences are in regards to node:child_process.fork vs node:worker_threads.

In an effort to validate this assumption, I created a repository that benchmarks the performance differences between node:child_process.fork and node:worker_threads (scoped to my particular application). That said, I haven’t done much benchmarking in my career and am very open to feedback on how to increase the accuracy of the results.

Speaking of results. My current benchmarks are consistently showing node:child_process.fork is processing files faster than my node:worker_threads implementation.

I would greatly appreciate any insight as too why this is or what adjustments I can make to my node:worker_threads implementation!

https://github.com/aaronccasanova/node-parallel-benchmarks

[Trott]

If your threads are exhausting their memory, I can imagine forking a separate process (which would have more memory) could be much faster.

[aaronccasanova]

Hey @Trott 👋 I think your on to something!

I spent some time testing different benchmark parameters and started seeing more consistent numbers when sending less files to each free worker:

Breakdown of the above CLI options:

• -b, --benchmark-iterations: Ran the benchmark for each API 10 times
• -p, --process-iterations: Ran arbitrary processing on a text file 300 times (simulates high cpu load)
• -w, --worker-pool-size: Used 4 workers/forks for each benchmark iteration
• -f, --files-count: Processed 100 files for each benchmark iteration
• -c, --files-chunk-size: Sent 10 files at a time to each free worker

Note: The screenshot in my initial Discussion post used 6 workers and sent 50 files at a time to each free worker. Additionally, both of my implementations (workers and forks) process the batched files concurrently (with Promise.all).

However, after putting together some rough numbers of how much memory is being used it doesn't seem like memory would be a factor:

• High memory run: 6 workers | 50 files per worker | 34 KB text file
  • 6 * 50 * 34 = 10,200 KB = 10.2 MB - I'm naively assuming this is the max memory being used
• Low memory run: 4 workers | 10 files per worker | 34 KB text file
  • 4 * 10 * 34 = 1,360 KB = 1.36 MB - I'm naively assuming this is the max memory being used

While 1.36 MB and 10.2 MB is quite a large gap, do you think 10.2 MB is enough memory to cause the performance differences?

[Trott]

This isn't really my wheelhouse, but maybe some of the folks in @nodejs/workers or @nodejs/diagnostics might have insight.

[aaronccasanova]

No worries! Thanks for giving me a new area to explore and cc'ing folks with more insight 🚀

[gireeshpunathil]

the difference between worker and child process in this context are:

• the former is lightweight thread, while the later is a heavyweight process
• the former is easy to spin compared to the CP
• the former will use a CPU slice in the process schedule while the later practically gets a better CPU share, being a full process.

so the latency difference will be a function of how long the task is run, and how much you interact with the main (parent) process. In the absence of any continued interaction, it is a sole function of the computation. Longer the task, the child process will prove to be running faster.

[aaronccasanova]

Thanks for the breakdown @gireeshpunathil! My benchmark implementations do somewhat model operations that could be long-running. Which makes me think the CPU slicing you mentioned could be a factor. However, the default benchmark parameters only take ~0.03 seconds per file on my machine so I'm still a bit skeptical. That said if CPU slicing is the cause I wonder what Deno is doing differently.

[kerolloz]

I am not sure of the internal implementation of worker_threads. child_process.fork is more obvious. You can take a look on the difference between threads and process from an OS point of view. That might help you understand the underlying implementation of these 2 different ways. I suggest reading the chapters from OSTEP book (i.e. https://pages.cs.wisc.edu/~remzi/OSTEP/threads-intro.pdf).

[aaronccasanova]

Hey @kerolloz! Thanks for linking the OSTEP resource, great learnings! The following excerpt definitely stood out to me:

threads share an address space and thus make it easy to share data, and hence are a natural choice when constructing these types of programs. Processes are a more sound choice for logically separate tasks where little sharing of data structures in memory is needed.

The particular application I'm benchmarking doesn't need to share memory for which processes may be the more appropriate tool.

Another interesting excerpt:

As it turns out, there is another common interaction that arises, where one thread must wait for another to complete some action before it continues. This interaction arises, for example, when a process performs a disk I/O and is put to sleep; when the I/O completes, the process needs to be roused from its slumber so it can continue.

My implementations do have some light disk I/O (e.g. reading data.txt files) before performing the arbitrary operations on the file. Which makes me curious if there's an element of "sleeping" and "waking" threads adding time to the node:worker_threads implementation.

[kerolloz]

@aaronccasanova I think it means that all the threads will be idle if one of them is performing a disk I/O. So it doesn't really matter how many threads are running if you're doing extensive I/O operations. It reminds me of an experiment I had with reading files and how I tired to optimize it using concurrency in Node.js and Golang. I wrote a thread about it. You might want to take a look. I hope it helps.

[aaronccasanova]

I think it means that all the threads will be idle if one of them is performing a disk I/O.

While I'm not familiar with node:worker_threads internals, I don't believe this is the case. I recently watched a talk by @jasnell where he describes the anatomy of node.js workers and from what I understand each worker shares the libuv thread pool (4 by default) for disk I/O.

Note: I did play around with UV_THREADPOOL_SIZE and it didn't affect the performance of my particular application.

[mhdawson]

Without having looked at the code/benchmark, have you tried using a worker pool, for example https://github.com/piscinajs/piscina ?

[aaronccasanova]

Hey @mhdawson! Yes, my implementation initializes a worker pool 👍 Also, I had a play with piscina during my initial exploration and the performance was virtually identical to my vanilla node:worker_threads implementation.

[aaronccasanova]

Out of curiosity, I put together a demo with Deno and to my surprise saw results on par with node:child_process.fork:

Definitely curious what they are doing differently and if there are opportunities to improve node:worker_threads performance.

[aaronccasanova]

While the examples thus far may seem insignificant (+-1s), the performance differences become much more noticeable as the load increases e.g. Processing 50,000 files:

API	Files	Average processing time
node:worker_threads	50,000	3m 44s
node:child_process.fork	50,000	2m 40s
deno web workers	50,000	2m 49s

[gireeshpunathil]

one of the thing that we can check is where the CPU is actually spent (or not spent), through the perf tool This may give us some insights.

• node --prof <app> // generates a prof log (isolate-0x232x.log)`
• node --prof-process isolate-0x232x.log // shows where the time is spent

[jdmarshall]

It was asked but I don’t see it answered: did you run an experiment where you gave the worker_threads example 2,3,4x as much memory? Because your fork version has access to more resources than your threads example.

[benjamingr]

@nodejs/performance is this actionable?