Rate Limiting blocks everything after a while

[filex]

Describe the bug

I have a set up where I use Couper as an egress Gateway to implement a client-side rate limiting.

(I have a group of services that use the same rate-limited upstream API. The services run independently, so I wanted to have central exit point where I can count/throttle requests.)

Because the upstream is very rigid, I use a connection pool size of 1 and a rate limiter.

At first it works fine. Requests are throttled but succeed after queueing for a bit. But it can happen that too many requests pile up and the queue get longer. Sometimes that leads to requests being canceled by the HTTP clients (because they also have their timeouts).

After some time, Couper is in a state where all incoming requests are stuck in an endless block/wait queue. Even if there weren't any requests over a couple of minutes (the rate limiter window is a minute), all new requests wait forever until they timeout.

I cannot reproduce it yet, but it happens again and again. I have to restart Couper to get out of it.

To Reproduce
Steps to reproduce the behavior:

?

1. Which Couper version? Run couper version or docker run coupergateway/couper version

1.12.1 and edge

2. Provide your configuration file *.hcl. Remove sensitive data.

definitions {
  backend "upstream" {
    origin = env.ORIGIN
    max_connections = env.MAX_CONNECTIIONS
    set_request_headers = {
      authorization = request.headers.authorization
    }
    beta_rate_limit {
      mode = "wait"
      period = "60s"
      per_period = env.REQUESTS_PER_MINUTE
      period_window = "fixed"
    }
  }
}