More shred filtering #8975
More shred filtering #8975
Conversation
mergify
bot
dismissed
aeyakovenko’s stale review
Pull request has been modified.
sakridge
force-pushed
the
shred-filter
branch
3 times, most recently
from
275b8cb to
758578e
Compare
|
tests to show filtering? |
core/src/shred_fetch_stage.rs
Outdated
| if let Ok(index) = limited_deserialize::<u32>(&p.data[index_start..index_end]) { | ||
| if index < MAX_DATA_SHREDS_PER_SLOT as u32 { | ||
| if let Some((slot, index)) = Self::get_slot_index(p, &mut index_overrun) { | ||
| if slot > last_root && slot < (last_slot + 2 * slots_per_epoch) { |
There was a problem hiding this comment.
Can we add a comment for why we picked 2 * slots_per_epoch
There was a problem hiding this comment.
Seemed like a reasonable number :)
|
|
||
| pub struct ShredFetchStage { | ||
| thread_hdls: Vec<JoinHandle<()>>, | ||
| } | ||
|
|
||
| impl ShredFetchStage { | ||
| fn get_slot_index(p: &Packet, index_overrun: &mut usize) -> Option<(u64, u32)> { |
There was a problem hiding this comment.
let's add a small test for this so we don't change shred format without breaking this.
There was a problem hiding this comment.
Can this be a utility function in the shreds module?
core/src/shred_fetch_stage.rs
Outdated
| let slot_received = shreds_received.entry(slot).or_insert_with(|| { | ||
| BitVec::new_fill(false, MAX_DATA_SHREDS_PER_SLOT as u64) | ||
| }); | ||
| if !slot_received.get(index.into()) { | ||
| p.meta.discard = false; |
There was a problem hiding this comment.
isn't the discard flag false by default?
There was a problem hiding this comment.
No, it's set to true by default earlier in the loop.
https://github.com/solana-labs/solana/pull/8975/files#diff-6e3721caf0a126d8a8852e4f448fc0e7R84
This file has no tests.. why start now? ... /s |
Thread bank_forks into shred fetch
Thread bank_forks into shred fetch (cherry picked from commit 453f5ce) # Conflicts: # core/src/shred_fetch_stage.rs
Thread bank_forks into shred fetch
Thread bank_forks into shred fetch and filter by slot.
Problem
Shred replay dos attack can overwhelm shred processing stages downstream of shred fetch.
Summary of Changes
Introduce a filter for shreds at the same slot/index which is cleared periodically. Also, more agressively filter slots which are not relevant like ones below the current root or more than 2 epochs away.
Fixes #8861