Rate limiting for fun and for profit

[jedahan]

Felt like starting a discussion to make the api wrapper more fun to use. One thing would be (opt-out) ratelimiting.

I searched for the topics ratelimit, limiter, and throttle on npmjs.com. Clicked on a few, and this one seemed the most simple, though its more callback-y than promise-y.

https://github.com/jhurliman/node-rate-limiter#usage

import { RateLimiter }  from 'limiter'
// Allow 150 requests per hour
const limiter = new RateLimiter(150, 'hour')

// err will only be set if we request more than the maximum number of requests set in the constructor
limiter.removeTokens(1, (err, remainingRequests) => {
  if (err) return
  callMyRequestSendingFunction(...)
})

---

I also started writing something to give a little more context to the poe ratelimit headers:

#!/bin/env ts-node

const response = {
    headers: {
        'x-rate-limit-account': '45:60:60,240:240:900',
        'x-rate-limit-account-state': '3:60:0,11:240:0'
    }
}

const ratelimit = (triple: string) => {
    const [requests, interval, timeout] = triple.split(':')
    return {requests, interval, timeout}
}

const RateLimit = {
    decode: (headers: Record<string, string>) => {
        return headers['x-rate-limit-account-state']
            .split(',')
            .map(ratelimit)
    }
}
const limits = RateLimit.decode(response.headers)

console.dir(limits)
/**
 [
  { requests: '3', interval: '60', timeout: '0' },
  { requests: '11', interval: '240', timeout: '0' }
]
**/

---

A few things to think about regarding ergonomics of making this nice for the developer:

• do we just go as fast as possible until 'blocking' on the ratelimit?
• can we have returned promises include expected time to completion?
• can we create some sort of batching api, where we build up a queue of all the requests that would hit poe trade and then await?
• does any of this complexity actually benefit the developer?

My gut feeling is that including a ratelimiter that blocks on await, with an optional timeout, would improve ergonomics.

PoeApi.setRatelimitTimeout(15 * 1000)

// Regular code that makes a bunch of requests...

// The request that would normally hit poe api and throw ratelimit error
const items = PoeApi.character('microns', 'nchiana').getItems()
// 15 seconds go by
items
  .then(items => console.log(items))
  .catch((e instanceof RateLimitTimeout | PoeApiError) => {
    //etc...
  })

[jedahan]

When developing, to get a sense of safety

PoeApi.setMockMode() // or .setBatch()

const items = PoeApi.search('Ritual', { type: 'Ring', mods: ['All T1 Harvest Mods lul', 'Never gonna see these together again'] })
console.log(PoeApi.mockExecute())
/**
*  Expected # of requests: 127
*  Request breakdown:
*   .character: {
*     requests: 1,
*     .items: {
*       requests: 1,
*       .item: {
*          requests: 7,
*         .search {
*           requests: 1
*         }
*     }
**/

[klayveR]

Built-in rate limiting is definitely something I'd like to have as an option in the library. Great code snippets, should definitely come in handy!

As a disclaimer, I'm not terribly familiar with all of this rate limit stuff, so I might get some things wrong or ask stupid questions.

---

Do you have an idea how node-rate-limiter could be used with multiple rules?

It might be easier to write our own implementation which supports multiple buckets for a single rate limiting policy. That way we could make sure that we comply with every rule by checking if each bucket has at least one token. As a bonus we'd avoid using callbacks or promisifying third party methods. I don't think token buckets are too difficult to write. x-rate-limit-policy could be used to differentiate limiters.

Scheduling requests would definitely be very useful, and as you mentioned, there should be a configurable maximum timeout before the rate limiter throws an error.

In the past I used bottleneck. It has built-in scheduling of requests, which I quite like. I thought I could use an instance of Bottleneck for each rule, but that seems very hacky and weird to me. Maybe I'm missing an obvious solution to this problem.

do we just go as fast as possible until 'blocking' on the ratelimit?

I'd say that if the rate limiter says it's fine to make a request, the request should be made without delay/minimum execution time.

can we have returned promises include expected time to completion?

Would be nice to have, but I personally don't think it's a necessity. Might add some weird complexity to return values, I don't think you can easily add information that is immediately accessible to promises (if you can, enlighten me).

---

Do you think the current state should be taken into consideration somehow? I don't know if the state we get in the response should be synced with the available tokens in the rate limiters. I'm asking because the state could always be different from the amount of tokens internally, because other instances could hit the API and the internal token buckets would never know (manual browsing, other tools, ...).