Epic: Rate Limiting

[BrennanConroy]

This issue captures issues related to rate limiting work in .NET 9. Issues and categorizations are subject to change as design and planning is underway.

• Distributed rate limiting
  • Redis Implementation
  • Any new primitives needed?
• More Configuration
  • Changing config at runtime
  • IConfiguration support?
• Missing features
  • Weighted endpoints (e.g. POST /buyItem costs 2 instead of 1 for GET /listItems)
  • Circuit breaker? (X request failures/sec, fail incoming requests to allow service time to self-heal)
• Metrics
  • Any additional metrics needed?
  • At a minimum document how to do manual metric retrieval

[aKzenT]

What is missing from this list is the support of multiple rate limits per endpoint (as requested in #42691 ). E.g. you want to allow your order endpoint to accept a maximum of five orders per IP per minute, but also a maximum of 100 orders per day.

[ShaneCourtrille]

@BrennanConroy One scenario we've run into is the need for rate limiting for the first 3 minutes of startup after which we can safely switch to no limiter. Is switching the rate limiter based on config going to be possible? Or would we just be using the concurrency limiter and changing the permit limit?

[BrennanConroy]

Unfortunately, the majority of this work is being pushed back to .NET 10.

[jakubsemerak]

Also priority queues used inside rate limiters would be pretty nice. In our case we needed to skip the queue line by priority communication to minimize response time as much as possible. So we have replaced standard Queue with PriorityQueue and everything works flawlessly. IMO priority queue inside the limiters would be more universal.

[adityamandaleeka]

Re-titled the issue and moved it to the backlog for now until we find time to work on this area again.

[marekott]

Hi guys, Distributed rate limiting seems like a very useful feature that would prevent many of developers from reinventing the wheel. In our case we are planning to introduce rate limiting on our edge microservice that is implementing YARP. Since it is distributed system there are multiple instances of that microservice varying in number between environments. Current implementation of rate limiting is not sufficient for us since we need to synchronize those limits between instances.

Other developers that need to implement it in their distributed systems will face the same issue. All of us will have to create own implementations based on Redis solving the same problems over and over again. It would be cool if you could find time to introduce this feature :)

[rkargMsft]

As a "right now" solution, there is this package implementing distributed rate limiting using Orleans:
https://github.com/managedcode/Orleans.RateLimiting

This includes exposing the core rate limiting primitives with Orleans as the coordinating state storage and implementations using that for Orleans grain calls as well as using them for ASP.NET Core calls.

[dtila]

Hi guys so from my point of view some work has to be done to have unified design.
Here are my thoughts:

1. A rate limit should allow having OnReject per each limit.
   Expected API:

options.AddXXX("policyName", options =>
 options.OnRejected = .....

Currently this functionality can be achieved by checking policy name by using if statements. However would be nice to have the reject attached to the policy as well.
This is needed in case you want to transmit to the client a response, eventually with the properties from the limit metadata

2. An overload with options.AddRateLimit("name", PartitionedRateLimiter instance) should be added. With the current design there is a discrepancy when you create a policy using options.AddXX and the GlobalLimiter.
   Expected behavior should be that everywhere should be using a PartitionedRateLimiter instance, similar with the current design of GlobalLimiter.
   Also all the existing functions to add policy, should should call this new overload

For example, currently if you want to create a chained policy you use this API:

 var ipRateLimit = PartitionedRateLimiter.Create<HttpContext, string>(CreateIpLimitPartition);
 var concurrencyLimit = PartitionedRateLimiter.Create<HttpContext, string>(CreateConcurrencyLimitPartition);
 options.GlobalLimiter = PartitionedRateLimiter.CreateChained(ipRateLimit, concurrencyLimit);

Using the same API, you should also use call options.Add("mypolicy", result_of_chained_policy)
Currently I could not find any way to achieve such a thing, even analysing the code,