Node fails to download entire (large) file via http/https when agent: false is used. Fails on 0.10, 0.12, 4.1.1.

[trentm]

When attempting to download a (largish) file via https.request and using agent: false, the end of the file will frequently be missing.

https://gist.github.com/trentm/80409ded30b4e0418e9c has a demo script and sample runs showing it failing on node 0.10.40 and node 4.1.1. In my experience I get a failure between 10% - 80% of the time.
There is a sample run with agent: false commented out which shows successful downloads a dozen or more times in a row.

My sample script just does manual writing of 'data' chunks from the res (response) to a file (fs.createWriteStream): https://gist.github.com/trentm/80409ded30b4e0418e9c#file-download-sans-agent-js-L83-L86
This was me trying to eliminate .pipe() usage as a possible cause.

https://gist.github.com/trentm/5d3c4d9dbcb7c3b8dbaa is an equivalent script (and sample runs) using res.pipe(outStream). It fails frequently as well.

[mscdex]

I'm not seeing what you're describing. I've ran the non-pipe and pipe versions of your script against the master branch with the same url for ~20 mins (with many, many requests during that time period) and did not see any errors.

[trentm]

@mscdex Thanks. You are right. It appears that we (at my office) can only repro this when running over our office network. Yuck. FWIW, I cannot repro when doing (at least as close as I can get) similar with curl:

$ while true; do echo ""; curl -sSf https://nodejs.org/dist/v4.1.1/node-v4.1.1-darwin-x64.tar.gz -o out.file --no-keepalive; \ls -l out.file; done

-rw-r--r--  1 trentm  staff  9544550 Sep 24 15:05 out.file

-rw-r--r--  1 trentm  staff  9544550 Sep 24 15:05 out.file

-rw-r--r--  1 trentm  staff  9544550 Sep 24 15:05 out.file
...

That makes me think it is a node issue, but something to do with a funky interaction with my office network. I'll follow up if I can find something to go on.

[misterdjules]

Possible duplicate of nodejs/node-v0.x-archive#8488.

[jasnell]

is this still an issue?

[jasnell]

Closing as there does not appear to be anything further to do. Can reopen if necessary