Use pnet_datalink instead of all of pnet #429
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pnet_transport takes a long time to build. It's been especially painful from within a docker container for reasons I don't care to understand. pnet_datalink is the only part of pnet we're using so booting the rest.
Codecov Report
@@ Coverage Diff @@
## master #429 +/- ##
==========================================
+ Coverage 91.31% 91.34% +0.02%
==========================================
Files 36 36
Lines 3547 3547
==========================================
+ Hits 3239 3240 +1
+ Misses 308 307 -1
Continue to review full report at Codecov.
|
godmodegalactus
pushed a commit
to blockworks-foundation/solana
that referenced
this pull request
Jan 5, 2024
Use cluster info functions for tpu (solana-labs#345) (solana-labs#347) Use git rev-parse for git sha Remove blacklisted tx from message_hash_to_transaction (Backport solana-labs#374) (solana-labs#376) Updates scripts for easy local setup. (solana-labs#383) Backports sim_bundle improvements (solana-labs#407) backports clone derivation 416 (solana-labs#417) Backport solana-labs#419: add upsert to AccountOverrides (solana-labs#420) backports solana-labs#430 v1.16: update jito-programs sha (solana-labs#431) [JIT-1661] Faster Autosnapshot (solana-labs#406) Fix Buildkite warnings (solana-labs#437) Backport solana-labs#446 to v1.16 (solana-labs#447) backport 428, runtime plugin (solana-labs#429) v1.16: Backport solana-labs#449 (solana-labs#450)
willhickey
pushed a commit
that referenced
this pull request
Mar 29, 2024
#429) implements weighted shuffle using N-ary tree (#259) This is port of firedancer's implementation of weighted shuffle: https://github.com/firedancer-io/firedancer/blob/3401bfc26/src/ballet/wsample/fd_wsample.c anza-xyz#185 implemented weighted shuffle using binary tree. Though asymptotically a binary tree has better performance, compared to a Fenwick tree, it has less cache locality resulting in smaller improvements and in particular slower WeightedShuffle::new. In order to improve cache locality and reduce the overheads of traversing the tree, this commit instead uses a generalized N-ary tree with fanout of 16, showing significant improvements in both WeightedShuffle::new and WeightedShuffle::shuffle. With 4000 weights: N-ary tree (fanout 16): test bench_weighted_shuffle_new ... bench: 36,244 ns/iter (+/- 243) test bench_weighted_shuffle_shuffle ... bench: 149,082 ns/iter (+/- 1,474) Binary tree: test bench_weighted_shuffle_new ... bench: 58,514 ns/iter (+/- 229) test bench_weighted_shuffle_shuffle ... bench: 269,961 ns/iter (+/- 16,446) Fenwick tree: test bench_weighted_shuffle_new ... bench: 39,413 ns/iter (+/- 179) test bench_weighted_shuffle_shuffle ... bench: 364,771 ns/iter (+/- 2,078) The improvements become even more significant as there are more items to shuffle. With 20_000 weights: N-ary tree (fanout 16): test bench_weighted_shuffle_new ... bench: 200,659 ns/iter (+/- 4,395) test bench_weighted_shuffle_shuffle ... bench: 941,928 ns/iter (+/- 26,492) Binary tree: test bench_weighted_shuffle_new ... bench: 881,114 ns/iter (+/- 12,343) test bench_weighted_shuffle_shuffle ... bench: 1,822,257 ns/iter (+/- 12,772) Fenwick tree: test bench_weighted_shuffle_new ... bench: 276,936 ns/iter (+/- 14,692) test bench_weighted_shuffle_shuffle ... bench: 2,644,713 ns/iter (+/- 49,252) (cherry picked from commit 30eecd6) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
lijunwangs
pushed a commit
to lijunwangs/solana
that referenced
this pull request
Apr 7, 2024
…na-labs#259) (solana-labs#429) implements weighted shuffle using N-ary tree (solana-labs#259) This is port of firedancer's implementation of weighted shuffle: https://github.com/firedancer-io/firedancer/blob/3401bfc26/src/ballet/wsample/fd_wsample.c anza-xyz#185 implemented weighted shuffle using binary tree. Though asymptotically a binary tree has better performance, compared to a Fenwick tree, it has less cache locality resulting in smaller improvements and in particular slower WeightedShuffle::new. In order to improve cache locality and reduce the overheads of traversing the tree, this commit instead uses a generalized N-ary tree with fanout of 16, showing significant improvements in both WeightedShuffle::new and WeightedShuffle::shuffle. With 4000 weights: N-ary tree (fanout 16): test bench_weighted_shuffle_new ... bench: 36,244 ns/iter (+/- 243) test bench_weighted_shuffle_shuffle ... bench: 149,082 ns/iter (+/- 1,474) Binary tree: test bench_weighted_shuffle_new ... bench: 58,514 ns/iter (+/- 229) test bench_weighted_shuffle_shuffle ... bench: 269,961 ns/iter (+/- 16,446) Fenwick tree: test bench_weighted_shuffle_new ... bench: 39,413 ns/iter (+/- 179) test bench_weighted_shuffle_shuffle ... bench: 364,771 ns/iter (+/- 2,078) The improvements become even more significant as there are more items to shuffle. With 20_000 weights: N-ary tree (fanout 16): test bench_weighted_shuffle_new ... bench: 200,659 ns/iter (+/- 4,395) test bench_weighted_shuffle_shuffle ... bench: 941,928 ns/iter (+/- 26,492) Binary tree: test bench_weighted_shuffle_new ... bench: 881,114 ns/iter (+/- 12,343) test bench_weighted_shuffle_shuffle ... bench: 1,822,257 ns/iter (+/- 12,772) Fenwick tree: test bench_weighted_shuffle_new ... bench: 276,936 ns/iter (+/- 14,692) test bench_weighted_shuffle_shuffle ... bench: 2,644,713 ns/iter (+/- 49,252) (cherry picked from commit 30eecd6) Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pnet_transport takes a long time to build. It's been especially
painful from within a docker container for reasons I don't care
to understand. pnet_datalink is the only part of pnet we're using
so booting the rest.