fix "RangeError: Maximum call stack size exceeded"

[smbape]

eachOfLimit throws a RangeError: Maximum call stack size exceeded when the collection is too big and the iteratee is synchronous.
The usual workaround is to use setImmediate(next) or http://caolan.github.io/async/docs.html#ensureAsync.

I have a data analysis program. The setImmediate(next) workaround made the whole program slow when there are few new data to analyse.

With a small modification, it was possible to make the program lasting from minutes to seconds.

Here is an exemple of time duration with and without setImmediate

const eachOfLimit = require("async/eachOfLimit");

(function() {
    'use strict';

    const collection = new Array(Math.pow(1024, 2));
    for (let i = 0, len = collection.length; i < len; i++) {
        collection[i] = i + 1;
    }

    const timerInit = Date.now();
    eachOfLimit(collection, 1, (value, i, next) => {
        next();
    }, err => {
        console.log("sync", Date.now() - timerInit, "ms");
    });
}());

(function() {
    'use strict';

    const collection = new Array(Math.pow(1024, 2));
    for (let i = 0, len = collection.length; i < len; i++) {
        collection[i] = i + 1;
    }

    const timerInit = Date.now();
    eachOfLimit(collection, 1, (value, i, next) => {
        setImmediate(next);
    }, err => {
        console.log("async", Date.now() - timerInit, "ms");
    });
}());

[hargasinski]

Thanks for the PR! It looks good, except for a couple of small changes. We implemented something similar for queues in #1334, so I don't see an issue in adding this to eachOfLimit (and the collection methods that use eachOfLimit internally).

[hargasinski]

Could you call done inside of the final callback instead?

[hargasinski]

This is minor, but we already have a forEachOfLimit synchronous test. Could you rename this to something like forEachOfLimit no stackoverflow or forEachOfLimit no call stack size exceed error just to differentiate between the two?

[hargasinski]

Could you also add a similar test for forEachOf and forEachOfSeries to make sure the behaviour is consistent?

[hargasinski]

I don't know why the build initially failed, but it appears to be working now. LGTM.

[aearly]

A clever, simple solution to a common problem.

[megawac]

👍 I was originally going to comment that this might be a misuse of async, but to be honest I've hit this case in my own personal use enough times to think its justified. That this solution is so simple is just icing on the cake.