Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

many async methods not usable for items > 5000 #110

Closed
langmi opened this issue Mar 2, 2012 · 2 comments
Closed

many async methods not usable for items > 5000 #110

langmi opened this issue Mar 2, 2012 · 2 comments

Comments

@langmi
Copy link

langmi commented Mar 2, 2012

async version: 0.1.18
machine: i7, 16GB Ram, OSX

i try to use the async library for queueing async processing of around 10000 items and so far i am always getting the

RangeError: Maximum call stack size exceeded

Exception

to count my poor javascript skills out i just used the example at https://github.com/caolan/async#queue

and enhanced it with

var asyncTest = function() {
for(var i = 0; i < 10000; i++) {
q.push({"counter": i}, function (err) {
console.log('finished processing:' + i);
});
}
};

asyncTest();

it will give the infamous exception

@caolan
Copy link
Owner

caolan commented Mar 2, 2012

I suspect you are using synchronous functions rather than asynchronous ones when processing the queue. What does your worker function look like?

@langmi
Copy link
Author

langmi commented Mar 3, 2012

you are right, and just for reference here the now working version

var async = require("async"); // https://github.com/caolan/async

// create a queue object with concurrency 200
var q = async.queue(function (task, callback) {
    function doIt() {
        console.log("working with:" + task.name);
        callback();
    };
    process.nextTick(doIt);
}, 200);

// assign a callback to call when all task are finished
q.drain = function() {
    console.log('all items have been processed');
}

var asyncTest = function() {
    for(var i = 0; i < 10000; i++) {
        q.push({"name": i}, function (err) {
            console.log("processed:" + this.data.name);
        });
    }
};

asyncTest();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants