Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

remote-externalities: parallelize value download #12044

Closed
kianenigma opened this issue Aug 16, 2022 · 1 comment · Fixed by #12537
Closed

remote-externalities: parallelize value download #12044

kianenigma opened this issue Aug 16, 2022 · 1 comment · Fixed by #12537
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. Z2-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase.

Comments

@kianenigma
Copy link
Contributor

The way this tool works now is that we first download all of they keys, in a paginated way. Then, we start requesting the values of the corresponding keys in batches. My guess is that this process can still be optimized by parallelizing the requests.
@kianenigma kianenigma added I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. Z2-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase. labels Aug 16, 2022
@ggwpez
Copy link
Member

ggwpez commented Aug 17, 2022

You can also play with the PAGE and BATCH_SIZE parameters to see if there is a sweet-spot.

@kianenigma kianenigma mentioned this issue Oct 20, 2022
Merged
2 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. Z2-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase.
Projects
(Nominated) Proof of Stake
Status: ✅ Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Try-runtime Revamp and Facelift paritytech/substrate
2 participants
@kianenigma @ggwpez