Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make writing cluster state dumps to buckets easier #5659

Closed
gjoseph92 opened this issue Jan 13, 2022 · 0 comments · Fixed by #5863
Closed

Make writing cluster state dumps to buckets easier #5659

gjoseph92 opened this issue Jan 13, 2022 · 0 comments · Fixed by #5863
Assignees
@gjoseph92
Labels
diagnostics good second issue Clearly described, educational, but less trivial than "good first issue".

Comments

@gjoseph92
Copy link
Collaborator

Client.dump_cluster_state is a useful method for debugging. But having to download all the state to your local machine can be slow. And as we consider having more automation around it (auto-dump on exception, on close, etc.), being able to write the dump directly to cloud/network storage would be useful.

Add support for dump_cluster_state to write using fsspec. We should support passing in any string that fsspec.open can handle, as well as a storage_options dict kwarg.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gjoseph92 gjoseph92

Labels
diagnostics good second issue Clearly described, educational, but less trivial than "good first issue".
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support dumping cluster state to URL gjoseph92/distributed
1 participant
@gjoseph92