file.createReadStream() ends prematurely and events do not seem triggered

[neilcronin]

Hi,

I am attempting to download a gzipped file from google cloud storage via the node library.

My code:

    var fs = require('fs');

    var bucket = require('gcloud').storage({
        keyFilename: '.gcloudKey.json',
        projectId: '<project-id>'
    }).bucket('<bucketname>');

    bucket.file('<filename>').createReadStream({validation: false})
        .pipe(fs.createWriteStream('/Users/neilcronin/wtf.gz'))
        .on('error', function(err) {
            console.log('anywhere');
        })
        .on('response', function(response) {
            console.log('here');
        })
        .on('complete', function() {
            console.log('there');
        })
        .on('close', function() {
            console.log('there');
        });

None of those console messages are emitted, and the data transferred for the same 42MB bucket item is between 500K and 5MB.

Transferring smaller files seems to work, but still none of those events are fired AFAICT.

It appears that the transfer is being interrupted or is bombing out, but I don't see any logging about what is going on.

Node v0.12.7
Gcloud v0.19.0
OS X 10.10.4

Thanks!
-neil

[stephenplusplus]

fs.createWriteStream emits finish when it's done. response and complete should be moved up before fs is piped as those are events from file.createReadStream.

After doing that, can you let me know if that helps matters?

Thanks for upgrading and reporting!

[stephenplusplus]

Also, I like your console.log style... looks exactly like mine :)

[neilcronin]

Thanks, your note about the event ordering and the correct 'finish' event helped trigger the right event.

The code I was running was from the sample docs.

Running this code repeatedly:

    bucket.file('<filename>').createReadStream()
        .on('error', function(err) {
            console.log('event: error');
        })
        .on('response', function(response) {
            console.log('event: response');
        })
        .on('finish', function() {
            console.log('event: finish');
        })
        .on('close', function() {
            console.log('event: close');
        })
        .pipe(fs.createWriteStream('/Users/neilcronin/wtf.gz'))
    ;

Yields:

$ time ./script && ls -lah ~/wtf.gz 
event: response

real    0m7.760s
user    0m1.138s
sys 0m0.157s
-rw-r--r--  1 neilcronin  staff   1.0M Aug 18 14:39 /Users/neilcronin/wtf.gz

$ time ./script && ls -lah ~/wtf.gz 
event: response

real    0m38.556s
user    0m1.609s
sys 0m0.400s
-rw-r--r--  1 neilcronin  staff   463K Aug 18 14:39 /Users/neilcronin/wtf.gz

$ time ./script && ls -lah ~/wtf.gz 
event: response

real    0m7.916s
user    0m1.146s
sys 0m0.157s
-rw-r--r--  1 neilcronin  staff   998K Aug 18 14:40 /Users/neilcronin/wtf.gz

$ time ./script && ls -lah ~/wtf.gz 
event: response

real    0m17.341s
user    0m1.198s
sys 0m0.167s
-rw-r--r--  1 neilcronin  staff   1.1M Aug 18 14:40 /Users/neilcronin/wtf.gz

$ time ./script && ls -lah ~/wtf.gz 
event: response

real    0m7.670s
user    0m1.161s
sys 0m0.165s
-rw-r--r--  1 neilcronin  staff   1.1M Aug 18 14:41 /Users/neilcronin/wtf.gz

So I get varied durations before the call terminates, and a variable amount of data for the same request, and I get no error event or error logging.

The bucket is "Nearline" storage type.

Thanks!
-neil

[stephenplusplus]

Thanks for the details. I'll look into this asap.

[neilcronin]

BTW, I see the same thing on node 10.x and gcloud 0.18.x...

[stephenplusplus]

@neilcronin I sent a PR #813 -- would you mind trying it out?

$ rm -rf node_modules/gcloud
$ npm install stephenplusplus/gcloud-node#spp--core-forward-specific-events

[neilcronin]

@stephenplusplus Yup, I ran it a couple 10s of times and your PR works like a charm. I updated the docs in another PR that you have already seen, I couldn't make a sensible PR between my fork and yours.

[neilcronin]

A++ would file issue again.

[stephenplusplus]

I look forward to it!