Skip to content

Commit

Permalink
Initial token bucket implementation (#1314)
Browse files Browse the repository at this point in the history
  • Loading branch information
megawac committed Nov 9, 2016
1 parent 9cc01fd commit 76daaef
Showing 1 changed file with 38 additions and 0 deletions.
38 changes: 38 additions & 0 deletions lib/internal/TokenBucket.js
@@ -0,0 +1,38 @@
import DLL from './DoublyLinkedList';

/**
* An internal implementation of [Token Bucket](https://en.wikipedia.org/wiki/Token_bucket)
* for rate-limiting/traffic shaping. Our token bucket starts with a slight twist from the
* conventional token bucket, in which it starts with bucketSize tokens already available.
*
* @param {Number} bucketSize - the maximum number of items (inclusive) which can be queued in
* a interval of time.
* @param {Number} interval - the period in miliseconds to stop tracking a sent item
*/
export function TokenBucket(bucketSize, interval) {
this.bucketSize = bucketSize;
this.interval = interval;
this.queue = new DLL();
this.queued = 0; // Number of items sent + size of queue
}

// Enqueue an operation to be executed when the rate limit is not exceeded.
TokenBucket.prototype.enqueue = function(operation) {
this.queued++;
if (this.queued <= this.bucketSize) {
operation();
} else {
this.queue.push(operation);
}

// after interval, decrement the queued count and call a queued operation (if bucket is full)
setTimeout(onIntervalComplete, this.interval, this);

This comment has been minimized.

Copy link
@hargasinski

hargasinski Nov 11, 2016

Collaborator

@megawac just wanted to confirm before committing a change. I think this should be something along the lines of:

var intervalNum = Math.ceil(this.queued/this.bucketSize);
setTimeout(onIntervalComplete, intervalNum*this.interval, this);

Otherwise, all of the queued functions will be called on the next interval. I'm thinking of a case where we have 7 functions queued, and bucketSize = 3. We should only be invoking the first 3 queued at the beginning of the next interval.

This comment has been minimized.

Copy link
@megawac

megawac Nov 11, 2016

Author Collaborator

It depends on whether we want to batch calls together or not (should we be processing bucketSize items at a time and waiting for them to finish processing or send them one at a time)

AFAIK conventional TokenBucket manages this by starting the token bucket with a queue size of 0 and it will increase the size of the available queue (number of available tokens) by 1 every this.bucketSize/this.interval until the bucket is saturated with bucketSize available tokens

I think there are pros and cons to both approachs

This comment has been minimized.

Copy link
@hargasinski

hargasinski Nov 14, 2016

Collaborator

AFAIK conventional TokenBucket manages this by starting the token bucket with a queue size of 0 and it will increase the size of the available queue (number of available tokens) by 1 every this.bucketSize/this.interval until the bucket is saturated with bucketSize available tokens

I think I get it, thanks. Sorry about that, I was approaching it more from the processing bucketSize items at a time approach, but I understand the other one too. I'll look into TokenBucket a little more.

}

function onIntervalComplete(bucket) {
bucket.queued--;
if (bucket.queue.length > 0) {
// call first queued operation
(bucket.queue.shift())();
}
}

0 comments on commit 76daaef

Please sign in to comment.