Moving from Request to got: Twitter Stream drops for no reason.

[BarryCarlyon]

What would you like to discuss?

I'm in the middle of converting from Request to Got.

But for whatever reason, the Stream just drops with Got but not with request.
The only difference between my code is requiring got instead of request.

Sample code:

const fs = require('fs');
const path = require('path');

const config = JSON.parse(fs.readFileSync(path.join(
    __dirname,
    'config.json'
)));

const crypto  = require('crypto');
const OAuth = require('oauth-1.0a');
const oauth = OAuth({
    consumer: {
            key: config.consumer_key,
            secret: config.consumer_secret
    },
    signature_method: 'HMAC-SHA1',
    hash_function: (baseString, key) => crypto.createHmac('sha1', key).update(baseString).digest('base64')
});

const got = require('got');

let url = ''
    + 'https://stream.twitter.com/1.1/statuses/filter.json'
    + '?follow=' + config.target_id + '&stall_warnings=true';

console.log('Connecting to', url);
got(
    url,
    {
        stream: true,
        method: 'GET',
        headers:  oauth.toHeader(oauth.authorize({url, method: 'GET'}, {
            key: config.access_token_key,
            secret: config.access_token_secret
        }))
    }
)
    .on('request', function(req) {
        //console.log('request', req);
    })

    .on('end', function() {
        console.log('end');
    })
    .on('finish', function() {
        console.log('finish');
    })

    .on('error', function(error) {
        if (error.statusCode) {
            console.log('Error with', error.statusCode, error.statusMessage);
        } else {
            console.log('Error', error);
        }
    })
    .on('response', function(response) {
        console.log('Got Response');

        response.setEncoding('utf8');

        response.on('data', function(chunk) {
            console.log('data', chunk.length, new Date());
            if (chunk == "\r\n") {
                console.log('Ping');
            } else {
                console.log('A Chunk');

                parser.recv(chunk);
            }
        });
    });

At this point I can't see any issues or bugs, just my "got" version stalls for no reason, where as the "request" version keeps going and going without issue. So I'm stuck.

Sure my code is missing the keep alive/auto reconnect logic, but I'm because in testing the Got version only sticks for 5/10 minutes max, where as request version I have days. I'm looking at Got being the issue?

Checklist

• I have read the documentation.

[szmarczak]

just my "got" version stalls for no reason

Duplicate of #223

I'm because in testing the Got version only sticks for 5/10 minutes max, where as request version I have days

I've got trouble understanding this sentence. Can you elaborate please?

I strongly recommend reading the full documentation + this Request migration guide.

[BarryCarlyon]

I'm because in testing the Got version only sticks for 5/10 minutes max, where as request version I have days

Screwed my grammer there.

I run the GOT version and the Request version side by side. The GOT version dies very quickly. But the request version keeps running fine (no stall).

I didn't see anything relevant in the Migration Guide, hence this open issue.

I'll take a loot at 223 and see if it helps. Thank you.

[szmarczak]

The GOT version dies very quickly. But the request version keeps running fine (no stall).

It's definitely because of #223. The main stream is blocking flow. You need to resume it:

    .on('response', function(response) {
        console.log('Got Response');

        response.setEncoding('utf8');

        response.on('data', function(chunk) {
            console.log('data', chunk.length, new Date());
            if (chunk == "\r\n") {
                console.log('Ping');
            } else {
                console.log('A Chunk');

                parser.recv(chunk);
            }
        });
-    });
+    }).resume();

If it doesn't work, please let me know, I'm happy to help :)

[BarryCarlyon]

Finally got around to testing.

Can confirm that .resume() has fixed the issue.

Thank you!

[sindresorhus]

@szmarczak Maybe we should add a tip about this in the readme? Seems like a common mistake that is easy to fall for. Maybe the Tips section with an example?

[szmarczak]

That's right. I'll send a PR.