Commitlog offset index #1671
Commitlog offset index #1671
Conversation
Shubham8287
force-pushed
the
shub/commitlog-index
branch
12 times, most recently
from
2a50dc3
to
982a76e
Compare
crates/commitlog/src/index/mod.rs
Outdated
| /// - `IndexError::OutOfMemory`: Append after index file is already full. | ||
| fn append(&mut self, key: Key, value: u64) -> Result<(), IndexError>; | ||
|
|
||
| /// Asynchronously flushes any pending changes to the index file |
There was a problem hiding this comment.
We might want to make this return io::Result (because that is, in fact, the only error which can occur).
Also worth explaining that an Ok doesn't indicate whether or not the flush succeeded, an Err result only says that it definitely did not succeed.
| } | ||
| } | ||
|
|
||
| pub type TxOffsetIndex = IndexFileMut<TxOffset>; |
There was a problem hiding this comment.
This is a concrete type. Do you plan to support anything other than memory maps? We can also have anonymous maps, not backed by a file.
If there's no immediate plan for a different implementation, I don't think we actually need the IndexRead, IndexWrite traits.
Otherwise, the Repo trait will need an associated type type OffsetIndex: IndexRead + IndexWrite and:
fn get_offset_index(&self, offset: TxOffset, cap: u64) -> io::Result<Self::OffsetIndex>
crates/commitlog/src/repo/mod.rs
Outdated
| @@ -17,6 +18,23 @@ pub use fs::Fs; | |||
| #[cfg(test)] | |||
| pub use mem::Memory; | |||
|
|
|||
| #[derive(Debug, Default, Clone, Copy)] | |||
| pub struct TxOffset(u64); | |||
There was a problem hiding this comment.
Can this not just be a type alias?
If we feel like it's useful, we should also change the other Repo methods which take an offset.
There was a problem hiding this comment.
Made it alias, newtype is not useful right now but may be when we use different Indexes like timestamps based.
Though, It will require bunch of mechanical change, can be done separately or when needed.
Shubham8287
added
release-1.0
and removed
release-any
Co-authored-by: Kim Altintop <kim@eagain.io> Signed-off-by: Shubham Mishra <shubham@clockworklabs.io>
Shubham8287
force-pushed
the
shub/commitlog-index
branch
from
e79dcb9
to
013bfdf
Compare
Description of Changes
Implements Commitlog offset proposal -https://github.com/clockworklabs/SpacetimeDBPrivate/pull/867
Reader integration is left to do in another PR.
API and ABI breaking changes
N/A
Testing