send duplicate shred proofs for merkle root conflicts #33889
Conversation
AshwinSekar
force-pushed
the
merkle-root-dup-proof
branch
5 times, most recently
from
c82b3a2
to
d0b2c9f
Compare
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #33889 +/- ##
========================================
Coverage 81.9% 81.9%
========================================
Files 811 811
Lines 219583 219972 +389
========================================
+ Hits 179935 180295 +360
- Misses 39648 39677 +29 |
AshwinSekar
force-pushed
the
merkle-root-dup-proof
branch
from
d0b2c9f
to
69f495d
Compare
| @@ -30,7 +31,7 @@ pub struct DuplicateShred { | |||
| pub(crate) wallclock: u64, | |||
| pub(crate) slot: Slot, | |||
| _unused: u32, | |||
| shred_type: ShredType, | |||
| _unused_shred_type: ShredType, | |||
There was a problem hiding this comment.
now that we can have a conflict between data and coding shreds, this field is useless.
AshwinSekar
force-pushed
the
merkle-root-dup-proof
branch
from
69f495d
to
410f6ed
Compare
AshwinSekar
force-pushed
the
merkle-root-dup-proof
branch
from
410f6ed
to
196b7e1
Compare
| just_inserted_shreds, | ||
| slot, | ||
| last_index.unwrap(), | ||
| u32::try_from(last_index.unwrap()).unwrap(), |
There was a problem hiding this comment.
this is annoying, it seems in order to use u32 indices everywhere, we need to update the slot_meta to use u32 for last_index, consumed, received. then we can also update the shred columns to use u32
|
This is 800+ LOC change. Also, would be better to separate the change writing merkle roots to blockstore from the change consuming/reading from blockstore. |
|
sure - it ended up being bigger than i originally thought 😅 |
Problem
Data shred + coding shred that are part of the same FEC set with different merkle roots are not marked duplicate.
Summary of Changes
Detect this scenario, store the proof in blockstore and notify the cluster.
Fixes #33644