detailed-0.9 test suites (always?) freeze

[ttuegel]

As reported by @coreyoconnor on issue #1810, the test runner for detailed-0.9 tests always freezes. This is probably related to the issue with exitcode-stdio-1.0 tests freezing sometimes due to over-buffering of the test output. I'm opening a new issue because the problems are separate and I'm about to close the other issue.

To reproduce: run cabal test in any package containing detailed test suites.

[coreyoconnor]

A small project that reproduces the issue is: https://github.com/coreyoconnor/cabal-issue-1810

(thanks!)

[snoyberg]

I've noticed that the active package's set suite always freezes as well, including with cabal-install-1.20.0.2. Is this the same issue being reported here, or should I open a separate bug report?

[coreyoconnor]

active also uses QuickCheck. active uses exit-code-stdio-1.0 and not detailed-0.9.
Both vty and active use quickCheckWithResult which uses the default reporting of QuickCheck: withStdioTerminal.
withStdioTerminal doesn't look particularly odd. On setup the buffering is set to line buffering. Each message is hPutStr followed by hFlush.

• https://github.com/nick8325/quickcheck/blob/master/Test/QuickCheck/Text.hs#L98

I'll try using withNullTerminal in vty and test if this changes the behavior.

[ttuegel]

@snoyberg active uses exitcode-stdio-1.0 tests, so that's a different issue.

[ttuegel]

@coreyoconnor Ah, your test runner has QuickCheck printing to stdout. detailed-0.9 type tests are not allowed to use stdout, it is reserved for Cabal's exclusive use. All of your output must be done by returning intermediate Progress results.

[coreyoconnor]

@ttuegel That I did not know. Is this in the docs somewhere? I haven't seen this mentioned in either:

• http://www.haskell.org/cabal/users-guide/developing-packages.html
• http://hackage.haskell.org/package/Cabal-1.20.0.1/docs/Distribution-TestSuite.html#t:TestInstance

Easy enough to test: Modified the test project at

• https://github.com/coreyoconnor/cabal-issue-1810

To have QuickCheck not use stdout. These tests no longer freezes and pass as expected. Cool! This change also worked for the few vty tests tried. Looks like that is the issue then. Feel free to close.
Thanks!