Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retries on RPC token bucket limit has been exceeded #395

Merged
merged 2 commits into from
Oct 10, 2023
Merged

Conversation

nfx
Copy link
Contributor

@nfx nfx commented Oct 10, 2023

Changes

Another case when the platform is not sending a proper HTTP 429 response. This time for the SCIM user list:
image

Tests

  • make test run locally
  • make fmt applied
  • relevant integration tests applied

Signed-off-by: Serge Smertin <259697+nfx@users.noreply.github.com>
@nfx nfx added the chore anything that makes the code better label Oct 10, 2023
@nfx nfx requested a review from a team October 10, 2023 11:30
@nfx nfx enabled auto-merge October 10, 2023 11:30
@pietern
Copy link
Contributor

pietern commented Oct 10, 2023

@nfx @mgyucht Do we have a canonical format for these, or is this just a message substring?

It would be nice to replicate these conditions across SDKs, as we do for the auth permutations.

@mgyucht
Copy link
Contributor

mgyucht commented Oct 10, 2023

@pietern I'm filing a follow-up with the platform team to fix the underlying issue.

@nfx nfx added this pull request to the merge queue Oct 10, 2023
@codecov-commenter
Copy link

codecov-commenter commented Oct 10, 2023

Codecov Report

All modified lines are covered by tests ✅

Files Coverage Δ
databricks/sdk/core.py 80.36% <ø> (ø)

📢 Thoughts on this report? Let us know!.

Merged via the queue into main with commit afeff49 Oct 10, 2023
8 checks passed
@nfx nfx deleted the fix/more-retries branch October 10, 2023 15:03
pietern added a commit that referenced this pull request Oct 12, 2023
* Added Python 3.12 to project classifiers ([#381](#381)).
* Fix serialization issues for generated resources ([#382](#382)).
* Fix select spark version in staging ([#388](#388)).
* Adjust token expiry window to 40 seconds because of Azure ([#392](#392)).
* Add retries on `RPC token bucket limit has been exceeded` ([#395](#395)).
* Regenerate to fix template drift ([#398](#398)).
* Update OpenAPI spec to 12 Oct 2023 ([#399](#399)).

Internal:

* GitHub OIDC publishing ([#386](#386)).
* Move Release Pipeline to OIDC ([#387](#387)).

API Changes:

 * Changed `download()` method for [a.billable_usage](https://databricks-sdk-py.readthedocs.io/en/latest/account/billable_usage.html) account-level service to start returning `databricks.sdk.service.billing.DownloadResponse` dataclass.
 * Added `databricks.sdk.service.billing.DownloadResponse` dataclass.
 * Changed `delete()` method for [a.account_storage_credentials](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_storage_credentials.html) account-level service with new required argument order.
 * Changed `get()` method for [a.account_storage_credentials](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_storage_credentials.html) account-level service with new required argument order.
 * Changed `update()` method for [a.account_storage_credentials](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_storage_credentials.html) account-level service with new required argument order.
 * Added `get_bindings()` method for [w.workspace_bindings](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/workspace_bindings.html) workspace-level service.
 * Added `update_bindings()` method for [w.workspace_bindings](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/workspace_bindings.html) workspace-level service.
 * Removed `name` field for `databricks.sdk.service.catalog.AccountsUpdateStorageCredential`.
 * Added `storage_credential_name` field for `databricks.sdk.service.catalog.AccountsUpdateStorageCredential`.
 * Removed `name` field for `databricks.sdk.service.catalog.DeleteAccountStorageCredentialRequest`.
 * Added `storage_credential_name` field for `databricks.sdk.service.catalog.DeleteAccountStorageCredentialRequest`.
 * Removed `name` field for `databricks.sdk.service.catalog.GetAccountStorageCredentialRequest`.
 * Added `storage_credential_name` field for `databricks.sdk.service.catalog.GetAccountStorageCredentialRequest`.
 * Added `owner` field for `databricks.sdk.service.catalog.UpdateConnection`.
 * Added `databricks.sdk.service.catalog.GetBindingsRequest` dataclass.
 * Added `databricks.sdk.service.catalog.UpdateWorkspaceBindingsParameters` dataclass.
 * Added `databricks.sdk.service.catalog.WorkspaceBinding` dataclass.
 * Added `databricks.sdk.service.catalog.WorkspaceBindingBindingType` dataclass.
 * Added `databricks.sdk.service.catalog.WorkspaceBindingsResponse` dataclass.
 * Added `spec` field for `databricks.sdk.service.compute.ClusterDetails`.
 * Added `apply_policy_default_values` field for `databricks.sdk.service.compute.ClusterSpec`.
 * Removed `aws_attributes` field for `databricks.sdk.service.compute.EditInstancePool`.
 * Removed `azure_attributes` field for `databricks.sdk.service.compute.EditInstancePool`.
 * Removed `disk_spec` field for `databricks.sdk.service.compute.EditInstancePool`.
 * Removed `enable_elastic_disk` field for `databricks.sdk.service.compute.EditInstancePool`.
 * Removed `gcp_attributes` field for `databricks.sdk.service.compute.EditInstancePool`.
 * Removed `preloaded_docker_images` field for `databricks.sdk.service.compute.EditInstancePool`.
 * Removed `preloaded_spark_versions` field for `databricks.sdk.service.compute.EditInstancePool`.
 * Added `deployment` field for `databricks.sdk.service.jobs.CreateJob`.
 * Added `ui_state` field for `databricks.sdk.service.jobs.CreateJob`.
 * Added `deployment` field for `databricks.sdk.service.jobs.JobSettings`.
 * Added `ui_state` field for `databricks.sdk.service.jobs.JobSettings`.
 * Removed `condition_task` field for `databricks.sdk.service.jobs.RunOutput`.
 * Added `webhook_notifications` field for `databricks.sdk.service.jobs.Task`.
 * Added `databricks.sdk.service.jobs.CreateJobUiState` dataclass.
 * Added `databricks.sdk.service.jobs.JobDeployment` dataclass.
 * Added `databricks.sdk.service.jobs.JobDeploymentKind` dataclass.
 * Added `databricks.sdk.service.jobs.JobSettingsUiState` dataclass.
 * Added `workload_type` field for `databricks.sdk.service.serving.ServedModelInput`.
 * Added `workload_type` field for `databricks.sdk.service.serving.ServedModelOutput`.
 * Removed [a.account_network_policy](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_network_policy.html) account-level service.
 * Changed `list()` method for [w.ip_access_lists](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/ip_access_lists.html) workspace-level service to return `databricks.sdk.service.settings.ListIpAccessListResponse` dataclass.
 * Removed `databricks.sdk.service.settings.AccountNetworkPolicyMessage` dataclass.
 * Removed `databricks.sdk.service.settings.DeleteAccountNetworkPolicyRequest` dataclass.
 * Removed `databricks.sdk.service.settings.DeleteAccountNetworkPolicyResponse` dataclass.
 * Removed `ip_access_lists` field for `databricks.sdk.service.settings.GetIpAccessListResponse`.
 * Added `ip_access_list` field for `databricks.sdk.service.settings.GetIpAccessListResponse`.
 * Removed `databricks.sdk.service.settings.ReadAccountNetworkPolicyRequest` dataclass.
 * Removed `databricks.sdk.service.settings.UpdateAccountNetworkPolicyRequest` dataclass.
 * Added `databricks.sdk.service.settings.ListIpAccessListResponse` dataclass.

OpenAPI SHA: 493a76554afd3afdd15dc858773d01643f80352a, Date: 2023-10-12
@pietern pietern mentioned this pull request Oct 12, 2023
github-merge-queue bot pushed a commit that referenced this pull request Oct 12, 2023
* Added Python 3.12 to project classifiers
([#381](#381)).
* Fix serialization issues for generated resources
([#382](#382)).
* Fix select spark version in staging
([#388](#388)).
* Adjust token expiry window to 40 seconds because of Azure
([#392](#392)).
* Add retries on `RPC token bucket limit has been exceeded`
([#395](#395)).
* Regenerate to fix template drift
([#398](#398)).
* Update OpenAPI spec to 12 Oct 2023
([#399](#399)).

Internal:

* GitHub OIDC publishing
([#386](#386)).
* Move Release Pipeline to OIDC
([#387](#387)).

API Changes:

* Changed `download()` method for
[a.billable_usage](https://databricks-sdk-py.readthedocs.io/en/latest/account/billable_usage.html)
account-level service to start returning
`databricks.sdk.service.billing.DownloadResponse` dataclass.
 * Added `databricks.sdk.service.billing.DownloadResponse` dataclass.
* Changed `delete()` method for
[a.account_storage_credentials](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_storage_credentials.html)
account-level service with new required argument order.
* Changed `get()` method for
[a.account_storage_credentials](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_storage_credentials.html)
account-level service with new required argument order.
* Changed `update()` method for
[a.account_storage_credentials](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_storage_credentials.html)
account-level service with new required argument order.
* Added `get_bindings()` method for
[w.workspace_bindings](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/workspace_bindings.html)
workspace-level service.
* Added `update_bindings()` method for
[w.workspace_bindings](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/workspace_bindings.html)
workspace-level service.
* Removed `name` field for
`databricks.sdk.service.catalog.AccountsUpdateStorageCredential`.
* Added `storage_credential_name` field for
`databricks.sdk.service.catalog.AccountsUpdateStorageCredential`.
* Removed `name` field for
`databricks.sdk.service.catalog.DeleteAccountStorageCredentialRequest`.
* Added `storage_credential_name` field for
`databricks.sdk.service.catalog.DeleteAccountStorageCredentialRequest`.
* Removed `name` field for
`databricks.sdk.service.catalog.GetAccountStorageCredentialRequest`.
* Added `storage_credential_name` field for
`databricks.sdk.service.catalog.GetAccountStorageCredentialRequest`.
* Added `owner` field for
`databricks.sdk.service.catalog.UpdateConnection`.
 * Added `databricks.sdk.service.catalog.GetBindingsRequest` dataclass.
* Added
`databricks.sdk.service.catalog.UpdateWorkspaceBindingsParameters`
dataclass.
 * Added `databricks.sdk.service.catalog.WorkspaceBinding` dataclass.
* Added `databricks.sdk.service.catalog.WorkspaceBindingBindingType`
dataclass.
* Added `databricks.sdk.service.catalog.WorkspaceBindingsResponse`
dataclass.
* Added `spec` field for
`databricks.sdk.service.compute.ClusterDetails`.
* Added `apply_policy_default_values` field for
`databricks.sdk.service.compute.ClusterSpec`.
* Removed `aws_attributes` field for
`databricks.sdk.service.compute.EditInstancePool`.
* Removed `azure_attributes` field for
`databricks.sdk.service.compute.EditInstancePool`.
* Removed `disk_spec` field for
`databricks.sdk.service.compute.EditInstancePool`.
* Removed `enable_elastic_disk` field for
`databricks.sdk.service.compute.EditInstancePool`.
* Removed `gcp_attributes` field for
`databricks.sdk.service.compute.EditInstancePool`.
* Removed `preloaded_docker_images` field for
`databricks.sdk.service.compute.EditInstancePool`.
* Removed `preloaded_spark_versions` field for
`databricks.sdk.service.compute.EditInstancePool`.
 * Added `deployment` field for `databricks.sdk.service.jobs.CreateJob`.
 * Added `ui_state` field for `databricks.sdk.service.jobs.CreateJob`.
* Added `deployment` field for
`databricks.sdk.service.jobs.JobSettings`.
 * Added `ui_state` field for `databricks.sdk.service.jobs.JobSettings`.
* Removed `condition_task` field for
`databricks.sdk.service.jobs.RunOutput`.
* Added `webhook_notifications` field for
`databricks.sdk.service.jobs.Task`.
 * Added `databricks.sdk.service.jobs.CreateJobUiState` dataclass.
 * Added `databricks.sdk.service.jobs.JobDeployment` dataclass.
 * Added `databricks.sdk.service.jobs.JobDeploymentKind` dataclass.
 * Added `databricks.sdk.service.jobs.JobSettingsUiState` dataclass.
* Added `workload_type` field for
`databricks.sdk.service.serving.ServedModelInput`.
* Added `workload_type` field for
`databricks.sdk.service.serving.ServedModelOutput`.
* Removed
[a.account_network_policy](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_network_policy.html)
account-level service.
* Changed `list()` method for
[w.ip_access_lists](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/ip_access_lists.html)
workspace-level service to return
`databricks.sdk.service.settings.ListIpAccessListResponse` dataclass.
* Removed `databricks.sdk.service.settings.AccountNetworkPolicyMessage`
dataclass.
* Removed
`databricks.sdk.service.settings.DeleteAccountNetworkPolicyRequest`
dataclass.
* Removed
`databricks.sdk.service.settings.DeleteAccountNetworkPolicyResponse`
dataclass.
* Removed `ip_access_lists` field for
`databricks.sdk.service.settings.GetIpAccessListResponse`.
* Added `ip_access_list` field for
`databricks.sdk.service.settings.GetIpAccessListResponse`.
* Removed
`databricks.sdk.service.settings.ReadAccountNetworkPolicyRequest`
dataclass.
* Removed
`databricks.sdk.service.settings.UpdateAccountNetworkPolicyRequest`
dataclass.
* Added `databricks.sdk.service.settings.ListIpAccessListResponse`
dataclass.

OpenAPI SHA: 493a76554afd3afdd15dc858773d01643f80352a, Date: 2023-10-12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore anything that makes the code better
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants