Build parallelism versus link parallelism

[tepperly]

How do I balance build parallelism with link parallelism? Let's say I have a desktop with 16 cores. My software application has 160 C++ source files, and it produces 64 linked binary executables (applications & tests). Normally, I build with ninja, and it will have 16 build processes running concurrently (compiles or links).

If I use mold instead ld, it seems that each execution of mold assumes that it can use 16 concurrent threads. When there are 16 links happening concurrently, this results in 256 threads wanting to be active. This brought my desktop to its knees. I don't know if it was a thread issue or exceeding chip memory (resulting in disk swapping). Do you have suggestions for how to manage build parallelism versus link parallelism?

Thanks in advance

[rui314]

ninja tries to run as many processes as the number of available cores to maximize parallelism, with an assumption that each process is single-threaded. That assumption is incorrect for mold because mold is itself parallelized. Having more runnable threads than the number of cores is fine, as long as their total working set memory is smaller than the physical RAM size. If your machine grinds to halt due to memory oversubscription, you may want to try to set the MOLD_JOBS environment variable to 1, so that only one mold instances is active. For more info, see https://github.com/rui314/mold/blob/main/docs/mold.md#environment-variables.

[tepperly]

Thanks! This solved my problem!