(#52) Optimize unspent utxo #150
Conversation
This implementation of sparse arrays is intended to store 256 elements at most. It relies on 256-bit bitmap that stores indexes of elements that actually exist and uses popcount to restore indexes in the underlying vector. SparseArray implements an iterator.
This wrapper allows to share underlying data storage until it is required to shrink. Shrinks and additions to the storage trigger actual clone operations.
sparse-array/src/sparse_array.rs
Outdated
| #[derive(Clone, Debug, PartialEq, Eq)] | ||
| pub struct SparseArray<V> { | ||
| index: BitmapIndex, | ||
| data: Vec<V>, |
There was a problem hiding this comment.
this can be reduced to a Box<[V]> to save 8 bytes
There was a problem hiding this comment.
I'm not sure about that. Will we still be able to add and remove elements at arbitrary positions?
There was a problem hiding this comment.
no, the Box<[V]> is for when the array is immutable/frozen. when in the tree, it's useful to reduce the cost of the array by 8 bytes by leaf. But once you want to update it, you can thaw it into a Vec with into_vec
There was a problem hiding this comment.
This will change the interface a bit. Instead of set(&mut self, idx: u8, value: V) and remove(&mut self, idx: u8) -> Option<V> we will have set(self, idx: u8, value: V) -> Self and remove(self, idx: u8) -> (Self, Option<V>).
Anyway, I think that's ok for an immutable struct.
There was a problem hiding this comment.
yes, I think that's fine. if you want to have a mutable interface, I usually like the pattern of having a Builder type which use data structure that are more friendly to modification like Vec in this case.
|
@vincenthz I used |
|
@vincenthz I suggest moving this
into a separate issue and then create a reusable bitmap that will be used by both |
|
yes, we can move the special cases to another low priority issue. Otherwise everything has been addressed and now ready for merge, but I'm less than conviced about the latest commit to move from |
eugene-babichenko
force-pushed
the
optimize-unspent-utxo
branch
from
0a020ba
to
ef7dfa4
Compare
|
dropped this one |
|
nice one ! |
SparseArray.FastSparse) on top of sparse array that allows to share data between its instances and thus allows for faster clones.Data under
FastSparseArrayis shared untilsetorshrinkfunction is called. It is also possible to avoid copying the underlying structure when callingset, but actual use cases do not require that.TBD
POPCNTinstruction)TZCNT)Resolves #52