Infinite Node processes get spawned for invalid sandboxed workers

[VladimirMikulic]

Hi! Recently, I encountered an issue where BullMQ would infinitely spawn node processes until the computer eventually runs out of memory and crashes. Here's the screenshot from my Linux box:

This occurs when sanboxed processor JS file doesn't have/can't reach a valid default export.
(e.g. third party module throws, syntax typo etc.)

Reproduction

1. Input 30+ tasks in Redis/BullMQ
2. Create invalid Worker JS file, simplified example:

nonImportedFnCall();
module.exports = {
  // Your valid worker code
}

3. Open your System Monitor and filter for Node process
4. Start processing tasks and see Node processes spawning infinitely

I'd expect BullMQ to log a message like "Worker JS file could not have been imported" instead of spawning infinite Node processes. (silly mistakes happen, but that shouldn't be the reason to crash the host :)

Thanks!

[manast]

On which version of BullMQ are you getting this behaviour?

[VladimirMikulic]

@manast bullmq@3.4.2

[VladimirMikulic]

@roggervalf the issue is still present in the latest release 3.5.1
Node processes still spawn until they match the number of tasks in queue.
(i.e. 500 items in queue results in 500 Node processes when sandboxed worker.js is invalid / default export not reachable)

Here's my test case with 24 items in queue:
(you can ignore the first two Node processes which are servers)

[manast]

Let's reopen this one to investigate what is going on.