Application no longer properly waits for Twitter rate limit reset. Possible change on Twitter's side?

[grug-son-of-thog]

I use this tool to download quite a lot of artists' galleries off Twitter, and over the last couple days, I've noticed an interesting new issue when trying to run my batch file.
The application will iterate through a bunch of different profiles until it reaches a certain point, then start spitting out the following error for every profile:

[twitter][error] An unexpected error occurred: JSONDecodeError - Expecting value: line 1 column 1 (char 0). Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[twitter][debug]
Traceback (most recent call last):
  File "c:\users\mark\appdata\local\programs\python\python38-32\lib\site-packages\gallery_dl\job.py", line 76, in run
    for msg in extractor:
  File "c:\users\mark\appdata\local\programs\python\python38-32\lib\site-packages\gallery_dl\extractor\twitter.py", line 62, in items
    for tweet in self.tweets():
  File "c:\users\mark\appdata\local\programs\python\python38-32\lib\site-packages\gallery_dl\extractor\twitter.py", line 1082, in _pagination_tweets
    data = self._call(endpoint, params)["data"]
  File "c:\users\mark\appdata\local\programs\python\python38-32\lib\site-packages\gallery_dl\extractor\twitter.py", line 964, in _call
    data = response.json()
  File "c:\users\mark\appdata\local\programs\python\python38-32\lib\site-packages\requests\models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File "c:\users\mark\appdata\local\programs\python\python38-32\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "c:\users\mark\appdata\local\programs\python\python38-32\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "c:\users\mark\appdata\local\programs\python\python38-32\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This will continue for approximately 4 minutes, then proper downloads will commence again until the cycle repeats.

The fact that the timing is so consistent within my list of downloaded profiles and I never once see "waiting until ##:##:## for rate limit reset" is what makes me think something has changed on Twitter's side. I could be way off base, but that's my thought process.

I'm running v1.20.6-dev, for context.

[mikf]

I never once see "waiting until ##:##:## for rate limit reset" is what makes me think something has changed on Twitter's side

Something did change, which is why you got these errors, but Twitter's rate limit currently allows for a lot more requests compared to the last time I checked it. A year ago is was ~180 requests per 15 minutes, currently it is at 500.