parallelize many cmd/dep tests

[jmank88]

This change parallelizes many of the cmd/dep tests, and reduces the running time for go test ./cmd/dep/... on my machine from ~21s to ~18s, with room for more improvement with more parallelism (beyond GOMAXPROCS). With go test -parallel 16 ./cmd/dep/..., I can get it down to ~8s. In fact, most of the -parallel 16 speedup comes from pre-existing parallelism, and runs about ~10s on master.

So this PR offers around ~10% speedup, and it may be worth considering a CI test script change to run with more parallelism for an even better payoff.

[jmank88]

The googlebot appears to be hung. Maybe this comment will trigger it...

[sdboyer]

Seems fine - I love how parallelism works in subtests, but the same approach is so much harder to keep track of in normal tests. Still, an improvement.

I'm not sure what kind of parallelism is really even possible in our build environments. Do you want to take a crack at taking advantage of some GOMAXPROCS changes in this PR, or defer that to a later one?

[jmank88]

I'm not sure what kind of parallelism is really even possible in our build environments.

Yeah I have no idea, though I suspect the payoff here is from the integration tests being low CPU and mostly waiting on the network or whatever else, so even some extra concurrency on a single processor could be effective. On the other hand it may set us back on some of the other tests/packages.

Do you want to take a crack at taking advantage of some GOMAXPROCS changes in this PR, or defer that to a later one?

Let's defer. It might take some experimentation. I'm not sure what the best formula would be for CI, but my gut is to try some factor * number of processors (or GOMAXPROCS, if explicitly set for us). I also wonder if only the integration tests should have their parallelism turned up. If so, then it might be more cleanly done after #609 , if we can target whole packages rather than certain tests with regexps.

[sdboyer]

SGTM!