Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: 🐛 forward secure in StorageHelper for s3 #1205

Merged

Conversation

odulcy-mindee
Copy link
Contributor

Related Issue \ discussion

After b78e9b9, secure is now default to True. We need to forward secure key otherwise, even if it has been set to False, it switches back to True.

Patch Description

At

self._conf = copy(self._s3_configurations.get_config_by_uri(url))

secure key is correctly fetched from the config file.

When self._conf.update is called right after, secure is automatically changed to True.

Testing Instructions

Install latest version of clearml (1.14.3). Try with secure: false in clearml.conf under aws.s3.credentials.
You should get a similar error:

[...]

File "/usr/local/lib/***3.8/site-packages/botocore/retryhandler.py", line 233, in __call__
    return self._check_caught_exception(
  File "/usr/local/lib/***3.8/site-packages/botocore/retryhandler.py", line 376, in _check_caught_exception
    raise caught_exception
  File "/usr/local/lib/***3.8/site-packages/botocore/endpoint.py", line 249, in _do_get_response
    http_response = self._send(request)
  File "/usr/local/lib/***3.8/site-packages/botocore/endpoint.py", line 321, in _send
    return self.http_session.send(request)
  File "/usr/local/lib/***3.8/site-packages/botocore/httpsession.py", line 466, in send
    raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://X.X.X.X:9000/bucket/.clearml.bb4a79eb-bdbb-4c34-8e88-6b450ddc9def.test [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1131)

During handling of the above exception, another exception occurred:

[...]

  File "/usr/local/lib/***3.8/site-packages/clearml/task.py", line 609, in init
    task.output_uri = str(cls.__default_output_uri)
  File "/usr/local/lib/***3.8/site-packages/clearml/task.py", line 1207, in output_uri
    helper.check_write_permissions(value)
  File "/usr/local/lib/***3.8/site-packages/clearml/storage/helper.py", line 2789, in check_write_permissions
    raise ValueError("Insufficient permissions (delete failed) for {}".format(base_url))
ValueError: Insufficient permissions (delete failed) for s3://X.X.X.X:9000/bucket/

@jkhenning jkhenning merged commit 2e08468 into allegroai:master Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants