Rate Limiting

[rectifyer]

As of October 27, 2020, we’re enforcing rate limiting for all API apps.

---

Why are rate limits being enforced?

Over the past several months, Trakt performance has been negatively impacted by a huge increase in API traffic and poorly coded apps. In order to stabilize the API and increase performance for everyone, we’re turning on rate limiting. The Trakt API is free to use, and rate limiting will help us keep it that way.

What updates does my app need?

Your app will need to handle the 429 HTTP status code and headers that are sent when the rate limit is exceeded. This might be built into your API library, or you might need to customize your code to handle it. The API docs have more details.

What are the limits?

Name	Verb	Methods	Limit
AUTHED_API_POST_LIMIT	POST, PUT, DELETE	all	1 call per second
AUTHED_API_GET_LIMIT	GET	all	1000 calls every 5 minutes
UNAUTHED_API_GET_LIMIT	GET	all	1000 calls every 5 minutes

All limits are per user.

Moving forward

We plan on adjusting limits until we find a balance of good performance with minimal app impact. The goal is to prevent API abuse, but allow users to use apps normally. We’ll keep the API docs updated with the current rate limits.

---

If you have any questions, please continue the discussion below.

[jonathanantoine]

Hello,

Is there somewhere we can see our current usage of the api ?

Thanks !

[rectifyer]

No there isn't. The above limits are applied per user in your app, not globally.

[kevincador]

The limits are higher than expected! Thanks a lot @rectifyer !

Is it already push in prod? I'm trying to hit the limit (sorry) with some load testing but I can't seem to hit it.

[rectifyer]

@kevincador in the process of emailing all devs, then pushing the code to production. Yeah, we wanted to start here and we'll adjust once we have more info and metrics.

[halkeye]

Fyi, the email starts with

  We are contacting you today because we have learned of a data breach that occurred back in December 2014

It's too late to do anything about it, but good to know if you get support requests

[rectifyer]

@halkeye oh shoot, I'll fix that before sending the rest. That's what I get for using the same email template.

Update: looks like it wasn't in the email itself, but somehow the email preview only. It's been corrected for the remaining emails being sent out.

[rectifyer]

API is updated and now contains the rate limiting and headers.

[Larsg310]

Hi, any updates on the staging server? It would save a lot of calls to the main server when people are testing code a lot.

[rectifyer]

@Larsg310 I'm having to rebuild that server since it got corrupted. Hoping to have that working in the next few days.

[kevincador]

Just checked SERIST, MOVIST and Rippple. Everything seems to be okay. The limits are high enough to be hard to hit. Thanks again!

I just have two notes:

• the 1000 every 300 seconds limit would be better in terms of user experience if it's something like 200 every 60 seconds. Because if you hit the 1000 limit after 60 seconds in the first case you need to wait 4 minutes to retry witch is a lot for a user. If you hit the 200 limit in the second case, you need to wait less than a minute to retry. Then we may hit the limit faster, not sure it's the best idea. Just thinking about the worst case scenarios.
• I'm doing some HEAD requests sometimes to get the real number of items in a list without getting the whole payload and without parsing everything. I don't know if it counts in the rate limit. I would appreciate it if it's not.

[tidusjar]

Is the rate limit applied per API Key or IP?

[rectifyer]

@tidusjar the limit takes into account both the app + user IP

@kevincador that's great to hear it seems to be running as expected. We don't want to impact apps negatively, we just want to stop any API spamming that seems to be happening from some apps (media centers being more likely then mobile apps).

We considered adjusting the limit, but ended up with 1000 over 5 minutes to allow for bursting of API calls. For example, if you connect an app for the first time it might do 300 api calls to get things all setup, but then settle down after that. That is our theory anyway, but we'll continue to monitor and get feedback to see what issues come up. If 1000 api calls are happening in a minute, that seems like code needs to be optimized or cached better.

I believe any API call will be subject to rate limiting, but can you check the headers in your HEAD call to see for sure?

[kevincador]

Based on a quick test, yes the HEAD requests are taken into account.

You're right. The current limits are better for first sync and big one time processing. And yes, I had to trigger several big sync to hit the limit and even after that, I think I only had to wait 1 minute. So all good my side.

Thanks a lot for your time and support!

[michaldrabik]

@rectifyer FYI Not every endpoint seem to return limit info.
Here's just quick one I stumbled upon.

https://api.trakt.tv/shows/104648/seasons?extended=full,episodes

while next_episode has it:

[rudf0rd]

@michaldrabik this is due to it hitting cloudflare and not our servers. Unfortunately, it's just going to be part of it. We should document it though, because I agree, it can be confusing.

[eliehaykal]

Thank you for all the hard work @rectifyer

[JimQ4]

Why did I receive an email about rate limiting on Trakt.tv? I am not a developer.

[rudf0rd]

@JimQ4 You received the email because you've created 4 api keys for the Trakt API. Creating an API key is supposed to be for developers.

[JimQ4]

Don't remeber doing that. Can I delete them. Don't want to cause any issues.Jim Sent from Yahoo Mail on Android On Tue, Oct 27, 2020 at 5:14 PM, Sean<notifications@github.com> wrote: @JimQ4 You received the email because you've created 4 api keys for the Trakt API. Creating an API key is supposed to be for developers. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[jairolopezj]

@JimQ4 You received the email because you've created 4 api keys for the Trakt API. Creating an API key is supposed to be for developers.

I'm in the same situation as @JimQ4: I got the email, and have an API key created that I have never used since I'm not a developer. How can I delete it?
BTW, thanks a lot for the great work!

[rectifyer]

Please email support from your Trakt registered email address, and we can delete the apps for you.

[Kiggerbare]

I'm also not a developer, but I'm pretty sure that I created the API to have Kodi sync what I saw.
That it was the old way to connect your account to the Trakt add-on, before the new way to log in was implemented.

[rudf0rd]

Hello,
I use your API in my API. To explain, I request and record according to the movie, show and list in my API in every 12-18 hours. I made these requests without user login. I make my requests by spreading over time, but it seems too little for me.

You're saying you need more than 2000 api calls per 10 minute interval?

[aburke20]

Ever since the updated API limits, my Trakt Kodi add-on has stopped syncing, and yet it seems that this change shouldn't have affected it, right ?

I've tried logging in and out of the add-on, but still no dice. Am I missing something, is the log-on method still better than using the API ?

Just no clue how to get it all working, and it seems to be in parallel to these changes ...

[MxFlix]

Is there some way to get notified of Rate Limit changes? Because today I happened to check the docs and saw that it has changed (now based on POST and authed/unauthed GET), but I couldn't find any announcement of this happening. Obviously it would be pretty cumbersome to check the docs regularly just to see if the rate limits have silently changed...

[michaldrabik]

@MxFlix Emails were sent around 27 of October.

[MxFlix]

@michaldrabik Correct.

I was, however, not talking about the initial introduction of rate limits in general, but rather that the specifics of rate limits have changed since then. (At the beginning it was 2/1 for /sync/history and 1000/300 elsewhere, as of writing it is 1/1 POST/PUT/DELETE and for unauthed/authed GET 1000/300 each.)
This time, the change was not a very big one, and few apps should be affected by it, but it would still be nice to be notified of rate limit changes, especially as they could no doubt become more significant in the future.

[MaxHasADHD]

@rectifyer In my app Television Time if a user signs into their account, then marks a few episodes as watched I hit the rate limit. Not sure if I'm hitting the actual limit or this is an issue. Replicated it twice for the same content.

Headers
"retry-after" = ( 1 ); "x-ratelimit" = ( "{\"name\":\"AUTHED_API_POST_LIMIT\",\"period\":1,\"limit\":1,\"remaining\":0,\"until\":\"2020-12-07T14:14:18Z\"}" );

[michaldrabik]

@MaxHasADHD Are you making a different API call each time user marks an episode? I mean if the user marks 5 episodes in a quick succesion and each click triggers a different API call for the same POST endoint you will hit the limit for sure.

[MaxHasADHD]

yes, at the moment it's a different POST. I won't be able to get a fix out for a few days though and the app is not running great if users can sync 1 episode at a time.

[michaldrabik]

@MaxHasADHD Too bad :/ I think the only thing you can do is release a fix ASAP. Current limit is 1 call per second:

I agree that Trakt guys should at least give us developers a warning and time to check codebases before making any rate limiting changes in the future.

[MaxHasADHD]

:( well that would explain it. Surprised I haven't heard from any users or get 1 star reviews yet but I found the app unusable this morning. I understand the change, but wish we got an email about this with some time to check our apps.

[rectifyer]

We'll do a better job of announcing any changes to the API rate limits. The first post here has been updated to match the latest in the API docs. This change was made about 3 weeks ago and this was the first issue we've had reported with it.

I don't think there is a need to specifically handle the individual rate limits, but rather handle the 429 and wait the amount of time requested and your apps should be able to automatically handle things.

There was a backend service affecting performance this morning, which might have been what you were actually seeing.

[jonathanantoine]

@rectifyer even when handling 429, it's a pretty low rate limit for a normal user which open an app and set the last x episodes watched as seen.

Do you want us to make users wait x seconds to be sure the episodes are set as seen.

On mobile app, if the app is not in the foreground it can't "make later" the call to the api because it's killed by the OS ( for battery consumption purpose) so we can't even "tell him it's set as seen as the app will deal with it later by itself".

Anyway, I will update my apps to handle 429 and send as much "batch" call as possible but it's a lot of work and having a little more time to develop it and test it later would be nice.

[MaxHasADHD]

Yeah I haven't received any specific reports for this issue, but I did see in a few user logs a lot of failures and when I used the app myself it wasn't usable in the least. Even saw an odd issue where the app got 429 but I saw most of the content on Trakt, and then my app tried again so it got duplicated. Not sure how though. Will test more and see if its working better today, but currently not batching some of the POSTs and will try to get that updated soon

[MaxHasADHD]

@rectifyer is the limit for POST, PUT, and DELETE 1 per second per path, or per method? can I POST to 2 different paths at the same time?

[kevincador]

@rectifyer I have a question regarding Rate Limits. You have a AUTHED and UNAUTHED rate limit but it seems like it is based on the auth headers we set. So basically my question is do you look at the headers to see if we send a token or do you count based on the endpoint?

FYI I'm always injecting the auth token in all my requests if the user is logged in even for endpoints that do not require to be authenticated. I guess I'm not the only one 😅

Also, do you think this would be possible (HEAD methods are not GET or POST methods): #247

I'm looking at every way to handle the rate limits best and reduce the impact on the user experience.

Thanks!

[roody11]

I wonder if i add this api in my node js server side for security issue of my client id, does trakt api support header[ X-Forwaded-For ] .

I'm trying to setup a Javascript clinet side , forwarding it through my server to avoid exposing my client id, but if the server makes the request, wouldn't its IP be the one logged in trakt's rate limiting ?

[mmoore99]

@rectifyer I would like the app I am developing to be able to handle 429 errors when they occur. However, currently when a 429 error occurs I get the following message from the browser:

Since the browser is rejecting the request because of CORS, my application does not get to see the rate limit error and the associated rate limit headers. Does a rate limit error response not contain the "Access-Control-Allow-Origin" header? If not, would it be possible for you to provide it? Or, is there some other way to handle this situation?