Best Practice Question: Thousands of Jobs vs. One Large Job

[selected-pixel-jameson]

Hello,

Sorry I'm not sure where to post this question so I'm posting it here.

Would it be a better practice to queue up 10k very small jobs or 1 job that processes all 10k requests synchronously?

Thank you!

[manast]

In general it is best to have small, idempotent jobs than large ones, also it allows you to paralellize the job in several workers: https://blog.taskforce.sh/splitting-heavy-jobs-using-bullmq-flows/