Clarification on the use of Bucket4j to limit concurrent requests.

[MinionAttack]

Hi, I have to implement a solution at work to limit the number of concurrent requests users can make to some API endpoints.

Reading the documentation and looking at examples I see that it can be done with Bucket4j but if I have understood correctly some examples I have seen on the internet of how tryConsume works:

if (bucket.tryConsume(1)) {
    Collections.shuffle(PLANETS);
    return ResponseEntity.ok(PLANETS.get(0));
} else {
    return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS).build();
}

Tries to consume a specified number of tokens from this bucket.

My doubts are when the consumed token is released, if when the whole code is finished executing inside if or as soon as it sees that there is a token available in the bucket and consumes it. To make it more clear I will expose two scenarios:

1. Let's say we have a policy of 5 requests every 10 seconds. In those 10 seconds I receive 5 requests from the same user but they take a long time to finish (we can't determine the time it will take to complete each one), but with this bucket policy after 10 seconds the bucket is refilled again with 5 tokens... therefore the user could generate another 5 requests again and finally exhaust the connection pool.

2. The user makes 5 requests that take 500 ms each, so in total it takes 2 seconds out of 10, the user would have to wait until the next refill to launch more requests. In this case there would be no need to wait.

So my question is whether Bucket4j allows tokens to be released when the request code finishes executing, i.e. when the if block code finishes executing.

if (bucket.tryConsume(1)) {
    // Doing a lot of time-consuming things
    .......
  // Things finished, release the consumed token.
  return ResponseEntity.ok(PLANETS.get(0));
} else {
    return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS).build();
}

In this way, if the user can make 5 requests every X amount of time, as long as the user does not finish the 5 he currently has, the user cannot request more. And if the user finishes them much earlier, the suer could make them again.

I have reviewed the refill strategies and it is not clear to me if these policies support scenario 1 and 2. That the bucket is not refilled until the execution of each request is finished instead of refilling every X amount of time.

I hope I have made myself clear.

Best regards.

[vladimir-bukhtoyarov]

Hello @MinionAttack

Bucket4j does not limit concurrent requests and does not track the requests in progress.

[MinionAttack]

I see, thank you for your reply.