Force test suites to run serially instead of concurrently WAS: separate configuration for test execution

[gregwebs]

Our example use case is that we always want build parallelism (including building of the test suite).
However, we would like to run the test suites serially (configure as jobs: 1) to avoid contending over shared resources such as the database.

[snoyberg]

I'm concerned about the implied scope of this based on the title: "separate configuration" can mean a lot of things, whereas "don't run tests in parallel" means only one thing. Is there some other example of configuration you want control over like this?

[gregwebs]

Right now I don't have other use cases. One thing I can think of that I have needed before (but probably shouldn't because it is really a hack) is setting flags differently.

[gregwebs]

Original issues was #482. @pseudonom @mboes

[mboes]

The configuration provides a default for all commands. Which can be overridden on the command line for specific commands. While it may be useful to provide per-command defaults in the config, could stack test --jobs=1 do the job for now assuming that forcibly serializing test suite execution is something you view as a temporary hack? Or is pinning jobs: 1 for tests something you want the ability to do forever for some reason?

[gregwebs]

The main problem with that solution is that it serializes the build process but we only want to serialize the test execution.

[snoyberg]

@gregwebs You don't have jetlagged kids, why are you awake right now?!?

I'm in favor of having something like concurrent-tests: BOOL in the YAML file. I doubt this will be the only report we get of test suites that share some resource. I'm actually in the middle of implementing this right now.

[snoyberg]

OK, I've pushed a commit to recognize the concurrent-tests setting in stack.yaml. @gregwebs @pseudonom can you test this and let me know if it resolves the issue? If so, would you be able to update the stack.yaml Wiki page with a description of the feature?

[pseudonom]

I can test this on Monday. I'll update the wiki when I do so.

[gregwebs]

I tested the concurrent-tests setting. It both builds the projects in serial and runs the tests in serial. This is an improvement for us because we can still build in parallel with stack build. We can run stack build && stack test and that is closer to what we want. Even still, the test suites themselves are not built in parallel.

The desired behavior is only to run the tests in serial, but for all building (including of the test suites) to happen in parallel.

[gregwebs]

I tested with a stack version that had the concurrent-tests setting commit: Version 0.1.1.0, Git revision 38fa3cf

It appears that we can also test with the 0.2 release now.

[snoyberg]

Hmm, that's surprising. I'll wait to hear on the results of trying this
with 0.2 before investigating further.

On Sun, Jul 5, 2015 at 6:11 PM Greg Weber notifications@github.com wrote:

I tested with a stack version that had the concurrent-tests setting
commit: Version 0.1.1.0, Git revision 38fa3cf
38fa3cf

It appears that we can also test with the 0.2 release now.

—
Reply to this email directly or view it on GitHub
#492 (comment)
.

[snoyberg]

Last I heard this was working as expected, so closing. If there are still issues, please reopen.