Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

profile block processor to improve sync performance #1142

Open
Tracked by #1790
TalDerei opened this issue May 22, 2024 · 4 comments
Open
Tracked by #1790

profile block processor to improve sync performance #1142

TalDerei opened this issue May 22, 2024 · 4 comments
Assignees
Labels
documentation Improvements or additions to documentation performance Related to speed improvements sync

Comments

@TalDerei
Copy link
Contributor

TalDerei commented May 22, 2024

profile the block processor and sync mechanism in the web, tracked by https://docs.google.com/document/d/1OCVsyi4qn5DlJmdoY9x2mk0dMgcLF577hlC0TD_fpjo/edit

@TalDerei TalDerei added the performance Related to speed improvements label May 22, 2024
@TalDerei TalDerei self-assigned this May 22, 2024
@github-project-automation github-project-automation bot moved this to 🗄️ Backlog in Penumbra web May 22, 2024
@TalDerei TalDerei moved this from 🗄️ Backlog to 🏗 In progress in Penumbra web May 22, 2024
@grod220 grod220 removed the mainnet label May 29, 2024
@TalDerei TalDerei added the documentation Improvements or additions to documentation label Jun 7, 2024
@grod220 grod220 added this to Labs web Jun 20, 2024
@grod220 grod220 moved this to 🏗 In progress in Labs web Jun 20, 2024
@TalDerei TalDerei moved this from 🏗 In progress to 🗄️ Backlog in Penumbra web Jun 26, 2024
@TalDerei TalDerei moved this from 🏗 In progress to 🗄️ Backlog in Labs web Jun 26, 2024
@TalDerei TalDerei added the sync label Jul 25, 2024
@TalDerei
Copy link
Contributor Author

TalDerei commented Aug 19, 2024

expanding benchmarking scope to profile the cost of poesidon hashing for TCT related operations.

@TalDerei
Copy link
Contributor Author

TalDerei commented Aug 27, 2024

benchmarking profile for syncing 800k blocks:

  • baseline: 280s
  • skip trial decryption: 200s (~25% speedup)
  • skip trial decryption and poseidon hashing: 100s (~75% speedup)
  • streaming 800k blocks in async block querier: depends on internet connection (experiencing average 30s+ on hotspot)
  • binary serialization and wasm decoding: ~5s

as the chain state grows, there’s a strong indication that the bottleneck will increasingly converge towards bandwidth (downloading every compact block) rather than compute for fresh wallets; imagining what it will be a year from now.

@TalDerei
Copy link
Contributor Author

TalDerei commented Sep 2, 2024

screenshot_2024-08-27_at_11 43 46___am

@TalDerei
Copy link
Contributor Author

TalDerei commented Sep 17, 2024

minimizing storage accesses and disk i/o will be profiled and investigated as other potential low-hanging fruit – related to #1814.

update: ~8% improvement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation performance Related to speed improvements sync
Projects
Status: 🗄️ Backlog
Development

No branches or pull requests

2 participants