Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase _MAX_BACKOFF_WAIT_SECS seconds #154

Open
csanders-git opened this issue Dec 17, 2021 · 0 comments
Open

Increase _MAX_BACKOFF_WAIT_SECS seconds #154

csanders-git opened this issue Dec 17, 2021 · 0 comments

Comments

@csanders-git
Copy link

csanders-git commented Dec 17, 2021

next_offset=['[offset1]', '[offset2]']
429
Sleeping for 1.6818238383872655
429
Sleeping for 2.8658061545293907
429
Sleeping for 4.266326646211362
429
Sleeping for 8.58080225730511
429
Sleeping for 16.02056706813436
429
Sleeping for 32.86096999735611
429
Traceback (most recent call last):
  File "/test.py", line 13, in <module>
    res = admin_api.get_authentication_log()
  File "/duo_client/admin.py", line 479, in get_authentication_log
    return list(self.get_authentication_log_iterator(api_version, params))
  File "/duo_client/client.py", line 415, in json_paging_api_call
    (objects, metadata) = self.parse_json_response_and_metadata(response, data)
  File "/duo_client_python/duo_client/client.py", line 491, in parse_json_response_and_metadata
    raise_error('Received %s %s' % (
  File "/duo_client_python/duo_client/client.py", line 477, in raise_error
    raise error
RuntimeError: Received 429 Too Many Requests

As has been demonstrated, It is possible to exceed this timeframe, which doesn't seem to be documented. Adding an additional iteration (aka 64, as opposed to 32) seemed to resolve most but not all observed issues.

ext_offset=['x', 'y']
429
Sleeping for 1.1092953488995574
429
Sleeping for 2.8283432090944345
429
Sleeping for 4.848998406819535
429
Sleeping for 8.35095954541268
429
Sleeping for 16.688679086577334
429
Sleeping for 32.90772754486507
429
Sleeping for 64.0423301176081
429
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    res = admin_api.get_authentication_log()
  File "/duo_client/admin.py", line 479, in get_authentication_log
    return list(self.get_authentication_log_iterator(api_version, params))
  File "/duo_client/client.py", line 415, in json_paging_api_call
    next_offset = metadata.get('next_offset', None)
  File "/duo_client/client.py", line 491, in parse_json_response_and_metadata
    response.status,
  File "/duo_client/client.py", line 477, in raise_error
    if not isinstance(data, six.text_type):
RuntimeError: Received 429 Too Many Requests

The ideal situation is not to have a max iterator if this is not supported within the documentation, and allow for infinite attempts given that stopping in the middle of an iterator may break requests that take a long time to complete (like for auth_logs).

For code reference see

wait_secs > self._MAX_BACKOFF_WAIT_SECS):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant