Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(bullmq): add manual rate-limit section #1590

Merged
merged 1 commit into from Dec 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
11 changes: 7 additions & 4 deletions README.md
Expand Up @@ -28,7 +28,7 @@
</a>
</p>
<p>
<em>Follow <a href="https://twitter.com/manast">@manast</a> for *important* Bull/BullMQ news and updates!</em>
<em>Follow <a href="https://twitter.com/manast">@manast</a> for *important* Bull/BullMQ/BullMQ-Pro news and updates!</em>
</p>
</div>

Expand Down Expand Up @@ -156,9 +156,12 @@ queueEvents.on('completed', ({ jobId }) => {
console.log('done painting');
});

queueEvents.on('failed', ({ jobId, failedReason }: { jobId: string, failedReason: string }) => {
console.error('error painting', failedReason);
});
queueEvents.on(
'failed',
({ jobId, failedReason }: { jobId: string; failedReason: string }) => {
console.error('error painting', failedReason);
},
);
```

This is just scratching the surface, check all the features and more in the official <a href="https://docs.bullmq.io">documentation</a>
Expand Down
14 changes: 14 additions & 0 deletions docs/gitbook/bullmq-pro/changelog.md
@@ -1,3 +1,17 @@
## [5.1.7](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.6...v5.1.7) (2022-12-16)


### Bug Fixes

* **deps:** upgrade bullmq to 3.4.2 ([#127](https://github.com/taskforcesh/bullmq-pro/issues/127)) ([b70ac2b](https://github.com/taskforcesh/bullmq-pro/commit/b70ac2bb6bc6af096a2980ab77b7009853a3c809)), closes [taskforcesh/bullmq-pro-support#33](https://github.com/taskforcesh/bullmq-pro-support/issues/33)

## [5.1.6](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.5...v5.1.6) (2022-12-15)


### Bug Fixes

* **remove-job:** check groupId is different than false on removed children ([#126](https://github.com/taskforcesh/bullmq-pro/issues/126)) ([efb54cb](https://github.com/taskforcesh/bullmq-pro/commit/efb54cbbd9486a608beace7f975247f5c6995470)), closes [taskforcesh/bullmq-pro-support#32](https://github.com/taskforcesh/bullmq-pro-support/issues/32)

## [5.1.5](https://github.com/taskforcesh/bullmq-pro/compare/v5.1.4...v5.1.5) (2022-12-13)


Expand Down
17 changes: 10 additions & 7 deletions docs/gitbook/bullmq-pro/groups/rate-limiting.md
Expand Up @@ -35,17 +35,20 @@ For this purpose, you can use the worker method **rateLimitGroup** like this:
```typescript
import { WorkerPro } from '@taskforcesh/bullmq-pro';

const worker = new WorkerPro('myQueue', async () => {
const worker = new WorkerPro(
'myQueue',
async job => {
const groupId = job.opts.group.id;
const [isRateLimited, duration] = awaidoExternalCall(groupId);
if(isRateLimited) {
const [isRateLimited, duration] = await doExternalCall(groupId);
if (isRateLimited) {
await worker.rateLimitGroup(job, duration);
// Do not forget to throw this special exception,
// since the job is no longer active after being rate limited.
throw Worker.RateLimitError();
}
}, {
connection
});
},
{
connection,
},
);
```

4 changes: 1 addition & 3 deletions docs/gitbook/guide/queuescheduler.md
@@ -1,7 +1,5 @@
# QueueScheduler



{% hint style="danger" %}
From BullMQ 2.0 and onwards, the QueueScheduler is not needed anymore, so the information below is only valid for older versions.
{% endhint %}
Expand Down Expand Up @@ -31,4 +29,4 @@ It is ok to have as many QueueScheduler instances as you want, just keep in mind

## Read more:

* 💡 [Queue Scheduler API Reference](https://github.com/taskforcesh/bullmq/blob/v1.91.1/docs/gitbook/api/bullmq.queuescheduler.md)
- 💡 [Queue Scheduler API Reference](https://github.com/taskforcesh/bullmq/blob/v1.91.1/docs/gitbook/api/bullmq.queuescheduler.md)
30 changes: 30 additions & 0 deletions docs/gitbook/guide/rate-limiting.md
Expand Up @@ -57,3 +57,33 @@ const scheduler = new QueueScheduler('painter');
// jobs will be rate limited by the value of customerId key:
await queue.add('rate limited paint', { customerId: 'my-customer-id' });
```

### Manual rate-limit

Sometimes is useful to rate-limit a queue manually instead of based on some static options. For example, if you have an API that returns 429 (Too many requests), and you want to rate-limit the queue based on that response.

For this purpose, you can use the worker method **rateLimit** like this:

```typescript
import { Worker } from 'bullmq';

const worker = new Worker(
'myQueue',
async () => {
const [isRateLimited, duration] = await doExternalCall();
if (isRateLimited) {
await worker.rateLimit(duration);
// Do not forget to throw this special exception,
// since the job is no longer active after being rate limited.
throw Worker.RateLimitError();
}
},
{
connection,
},
);
```

## Read more:

- 💡 [Rate Limit API Reference](https://api.docs.bullmq.io/classes/Worker.html#rateLimit)