Chained rate limiting policy

[SbiCA]

I was following the rate limiting docs https://learn.microsoft.com/en-us/aspnet/core/performance/rate-limit?view=aspnetcore-8.0#create-chained-limiters which allows to create a GlobalLimiter in a chained way.

I'm missing this on policy level, or is there a way that I might have overlooked?

I would have expected something like this to work

My scenario is the following:

Existing application with lots of endpoints I don't want to enforce a global rate limit immediately since a reasonable default isn't known yet. Therefore I'd like to start lax and make the limits stricter once the load patter per policy is more clear.

Also note that in our case we have/want to distinguish certain workloads via policy because not all endpoints are treated equal e.g. authenticate/non-authenticated and post/get traffic

[SbiCA]

@sebastienros or @danroth27 maybe? I'd like to understand if it's a user issue (highly likely) or a missing feature 😄

[juliuslouw]

I've been looking for this feature as well. We have a couple of endpoints in the solution that legitimately need some type of rate limit due to its cost. The other endpoints are fairly cheap and would be counterproductive to rate limit.
I tried creating multiple policies - i.e., per 1s, per 1m, per 1h, per 1d - and apply all of them to one endpoint, but the EnableRateLimiting attribute constraints prevent this type of solution.
A chained, partitioned rate limiting policy would be very useful.

[MikeNolan678]

Also (unsuccessfully) looking for a similar chained partitioned rate limiting feature. We impose higher limits on some internal endpoints currently, in a .Net Framework project using https://github.com/stefanprodan/WebApiThrottle. I'm moving this API to .Net 8.0, and would like to avoid using another NuGet package, but it looks like similar chaining is not possible unless it's applied to the GlobalLimiter.

[SbiCA]

seems like @JeremyLikness is would be able to tell more (at least that's what I understood from the last community standup 😄 )

[danroth27]

@BrennanConroy Can you take a look at this one?

[SbiCA]

@BrennanConroy did you give a look?

[BrennanConroy]

I'd like to start lax and make the limits stricter once the load patter per policy is more clear

Maybe I'm misunderstanding, but how does a chained limiter solve this? It seems like you want to return a different rate limiter depending on dynamic data.

You can achieve this by changing the name of the partition and returning different settings in the new limiter:

var dynamicData = 0;
if (dynamicData < 1)
{
    return RateLimitPartition.GetNoLimiter("noop");
}
else if (dynamicData < 5)
{
    return RateLimitPartition.GetConcurrencyLimiter("stricter", c =>
    {
        return new ConcurrencyLimiterOptions()
        {
            PermitLimit = 1000,
            QueueLimit = 100
        };
    });
}
else
{
    return RateLimitPartition.GetConcurrencyLimiter("strictest", c =>
    {
        return new ConcurrencyLimiterOptions()
        {
            PermitLimit = 100,
            QueueLimit = 10
        };
    });
}

We have a couple of endpoints in the solution that legitimately need some type of rate limit due to its cost. The other endpoints are fairly cheap and would be counterproductive to rate limit.
I tried creating multiple policies - i.e., per 1s, per 1m, per 1h, per 1d - and apply all of them to one endpoint, but the EnableRateLimiting attribute constraints prevent this type of solution.

This is also weird and maybe a misunderstanding, why are you trying to apply all policies to all endpoints? If one endpoint is cheap, apply the cheap policy to it. If an endpoint is expensive, apply the expensive policy to it.

app.MapGet("/one", () => { }).RequireRateLimiting("cheap");
app.MapGet("/two", () => { }).RequireRateLimiting("expensive");

[SbiCA]

@BrennanConroy yes probably I wasn't clear enough. Let me try to give some more context.

I'd like to use the ChainedRateLimiter on endpoint level but somehow the policy build doesn't allow to use it (or I'm not getting how).

Our scenario is more that the policy should

• Limit X requests per Min
• Limit Y requests per Hour
• Limit Z requests per Day

for a specific group of endpoints (hence we can't use the global rate limiter as shown in the docs)

app.MapGet("/limited-one", () => { }).RequireRateLimiting("Policy-10perMin-20perHour-100perDay");
app.MapGet("/limited-two", () => { }).RequireRateLimiting("Policy-10perMin-20perHour-100perDay");
app.MapGet("/not-limited", () => { });

I hope that helps.

[BrennanConroy]

Sounds like you want dotnet/runtime#98017 which should be "fairly easy" to implement yourself.

[SbiCA]

@BrennanConroy I would say so yes.
But as a user there is one point that I don't get.
Why I'm I allowed to use the following code when using RateLimiterOptions in AddRateLimiter

            // that works 
            limiterOptions.GlobalLimiter = PartitionedRateLimiter.CreateChained<HttpContext>( ... 
            
            // why can't I use the same here? 😕 
            // Option 1) ⬇️ maybe an overload that would allow to use `CreateChained` 
            limiterOptions.AddPolicy("some policy", context =>
            {
                var userId = GetAuthenticatedUserId(context);

                // Option 2) ⬇️  Allow to use `CreatedChained` here
                return RateLimitPartition.GetSlidingWindowLimiter(userId,
                    _ => new SlidingWindowRateLimiterOptions
                    {
                        PermitLimit = rateLimitsEventCreation.PermitLimit,
                        QueueLimit = rateLimitsEventCreation.QueueLimit,
                        Window = rateLimitsEventCreation.TimeSpan,
                        SegmentsPerWindow = rateLimitsEventCreation.SegmentsPerWindow
                    });
            });

That would be my natural way when seeing the current docs and I would try to use it.
My problem with it being "fairly" easy is that:

1. the time it takes to figure that out would take a few hours (many types, what is tied to what builder, options, policies ..)
2. not to speak about coming up with a solution on my own (not having your value insights @BrennanConroy but also that took weeks to get to)

So IMHO that should become part of the default since it's already allowed on global level and it would improve the clarity of the feature