Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pickling support to HttpError class #571

Merged
merged 4 commits into from
Jul 25, 2023
Merged

Conversation

tasansal
Copy link
Contributor

The HttpError class was updated to include support for pickling. A new private attribute _error_response was added for potential pickle use, and the __reduce__ method was added to deconstruct the HttpError when pickled. This update allows HttpErrors to be serialized and deserialized, i.e. in the case of multiprocessing use.

Closes #570.

The HttpError class was updated to include support for pickling. A new private attribute _error_response was added for potential pickle use, and the __reduce__ method was added to deconstruct the HttpError when pickled. This update allows HttpErrors to be serialized and deserialized.
@tasansal tasansal changed the title Add pickling support to HttpError class Add pickling support to HttpError class Jul 25, 2023
@martindurant
Copy link
Member

I suppose a test would be nice

@tasansal
Copy link
Contributor Author

True, will write one and push.

Altay Sansal added 3 commits July 25, 2023 13:33
Two new tests were added to the 'test_retry.py' script for improving the reliability and robustness of the code. The 'test_pickle_serialization' test was implemented to check the pickle serialization process of the HttpError class. The 'test_multiprocessing_error_handling' was added to validate how the application handles exceptions when running on multiple processes.
@tasansal
Copy link
Contributor Author

@martindurant I think its good to go now. Can you do one more review?

@martindurant martindurant merged commit 4541e40 into fsspec:main Jul 25, 2023
5 checks passed
@tasansal
Copy link
Contributor Author

@martindurant would it be possible to release this soon? Thank you very much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Strange behavior with HTTPError and multiprocessing
2 participants