From cab8640b01eee30e91f5e72d142f9a35c1aaeb37 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Mon, 9 Sep 2019 08:46:43 -0700 Subject: [PATCH] Create a jobserver with N tokens, not N-1 I recently added `jobserver` support to the `cc` crate and ended up running afoul of a `jobserver` quirk on Windows. Due to how it's implemented, on Windows you can't actually add more than the intial number of tokens to the jobserver (it uses an IPC semaphore). On Unix, however, you can since you're just writing bytes into a pipe. In `cc`, however, I found it convenient to control parallelism by simply releasing a token before the parallel loop, then reacquiring the token after the loop. That way the loop just has to acquire a token for each job it wants to spawn and then release it when the job finishes. This is a bit simpler than trying to juggle the "implicit token" all over the place as well as coordinating its use. It's technically invalid because it allows a brief moment of `N+1` parallelism since we release a token and then do a bit of work to acquire a new token, but that's hopefully not really the end of the world. In any case this commit updates Cargo's creation of a jobserver to create it with `N` tokens instead of `N-1`. The same semantics are preserved where Cargo then immediately acquires one of the tokens, but the difference is that this "implicit token" can be released back to the jobserver pool, unlike before. --- src/cargo/core/compiler/context/mod.rs | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/src/cargo/core/compiler/context/mod.rs b/src/cargo/core/compiler/context/mod.rs index e61fa65d98a..d4acfa73779 100644 --- a/src/cargo/core/compiler/context/mod.rs +++ b/src/cargo/core/compiler/context/mod.rs @@ -85,12 +85,16 @@ impl<'a, 'cfg> Context<'a, 'cfg> { // all share the same jobserver. // // Note that if we don't have a jobserver in our environment then we - // create our own, and we create it with `n-1` tokens because one token - // is ourself, a running process. + // create our own, and we create it with `n` tokens, but immediately + // acquire one, because one token is ourself, a running process. let jobserver = match config.jobserver_from_env() { Some(c) => c.clone(), - None => Client::new(bcx.build_config.jobs as usize - 1) - .chain_err(|| "failed to create jobserver")?, + None => { + let client = Client::new(bcx.build_config.jobs as usize) + .chain_err(|| "failed to create jobserver")?; + client.acquire_raw()?; + client + } }; let pipelining = bcx