Skip to content

Commit

Permalink
Support older versions of urllib3 and Databricks Runtime with regar…
Browse files Browse the repository at this point in the history
…ds to `DEFAULT_METHOD_WHITELIST` change to `DEFAULT_ALLOWED_METHODS` (#240)

## Changes
This PR changes the SDK to tolerate urllib3 < 1.26.0. In these versions
of urllib3, Retry class's `DEFAULT_ALLOWED_METHODS` was called
`DEFAULT_METHOD_WHITELIST`, and the parameter in Retry's constructor was
changed from `method_whitelist` to `allowed_methods`. With this change,
users using older versions of DBR (like the 7.3 LTS) can still use the
current version of the SDK, even though the version of urllib3 is old.

This change can be reverted when we no longer need to support urllib3 <
1.26.0 (i.e. after DBR 10.4 LTS, which is EOL on [March 18,
2025](https://docs.databricks.com/release-notes/runtime/releases.html)).

## Test

- [x] Manually tested on DBR 7.3LTS:
![Old_DBR_test_-_Databricks](https://github.com/databricks/databricks-sdk-py/assets/1850319/62822f6c-5b6a-46f6-aa30-26584f2e7f6b)
- [x] Manually tested on DBR 13.1: 

![Old_DBR_test_-_Databricks_2](https://github.com/databricks/databricks-sdk-py/assets/1850319/da0f4ac9-5170-4fd7-bfee-662fcc9214a0)

Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com>
  • Loading branch information
mgyucht and nfx authored Jul 18, 2023
1 parent a0dd2e8 commit 25ff035
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion databricks/sdk/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -871,13 +871,20 @@ def __init__(self, cfg: Config = None):
self._debug_truncate_bytes = cfg.debug_truncate_bytes if cfg.debug_truncate_bytes else 96
self._user_agent_base = cfg.user_agent

# Since urllib3 v1.26.0, Retry.DEFAULT_METHOD_WHITELIST is deprecated in favor of
# Retry.DEFAULT_ALLOWED_METHODS. We need to support both versions.
if 'DEFAULT_ALLOWED_METHODS' in dir(Retry):
retry_kwargs = {'allowed_methods': {"POST"} | set(Retry.DEFAULT_ALLOWED_METHODS)}
else:
retry_kwargs = {'method_whitelist': {"POST"} | set(Retry.DEFAULT_METHOD_WHITELIST)}

retry_strategy = Retry(
total=6,
backoff_factor=1,
status_forcelist=[429],
allowed_methods={"POST"} | set(Retry.DEFAULT_ALLOWED_METHODS),
respect_retry_after_header=True,
raise_on_status=False, # return original response when retries have been exhausted
**retry_kwargs,
)

self._session = requests.Session()
Expand Down

0 comments on commit 25ff035

Please sign in to comment.