dependency resolution fails on travis ci and within docker containers

[holgerbrandl]

See https://travis-ci.org/holgerbrandl/kscript

It seems related to the stream-redirection settings used in runProcess()

[yschimke]

I was getting hangs running locally.

n.b. You have to consume both streams in different threads or combine them.

https://github.com/holgerbrandl/kscript/blob/d24248e906f49fd0aa02cfb3532999f9fc75a855/src/main/kotlin/kscript/app/AppHelpers.kt#L41-L43

This is a nice library if you want to do something with the output. e.g. collect and show to the user if it fails.

https://github.com/zeroturnaround/zt-exec

[holgerbrandl]

Great to get some feedback on that problem! I tried already to narrow down its cause, and failed.

Scary to hear that this also happens locally. For me so far just within docker (and via docker also in travisci).

Why would both streams need to be processed in different threads? I always thought when using ProcessBuilder there would no need for threading in order to handle the output streams.

zt-exec looks cool, but I'm not sure if this would solve this ticket. Before rewriting kscript to use it, I'd love to understand what's causing the problem.

[yschimke]

n.b. I didn't debug this locally, but was getting consistent hangs (I could see a temporary spike in network activity) when maven was actually downloading results and therefore logging. I ran maven with the generated pom once and it completed the download, then I no longer got hangs when running kscript.

n.b. the reason for needing threads or combining the output is that you have two piped streams to the forked command. pipes have a limited capacity and if the writer writes too much it blocks. Think about something like "cat 1gbfile.txt | nc".

With two streams stderr and stdout used, you have to read both otherwise the command you exec could hang.

https://stackoverflow.com/questions/14165517/processbuilder-forwarding-stdout-and-stderr-of-started-processes-without-blocki

[holgerbrandl]

Indeed, hitting the piped stream's capacity limit seems the most reasonable explanation. I'm working on a fix now. Luckily that fix will also solve #23 which is pretty related to this ticket and easy to fix once we have better control over the spawned process out/err streams.

[holgerbrandl]

Thanks for input again. Fixed in today's v2.2 release.