panic on index out of bounds

[GregoryConrad]

I was trying to figure out the maximum key & value size redb supports since I didn't see it specifically documented anywhere (side question: what is the maximum k/v size redb supports?), so I decided to just figure it out for myself. Unfortunately, when I attempted to, I think I hit an edge case:

$ cargo run --release
   Compiling libc v0.2.155
   Compiling redb v2.1.1
   Compiling thing v0.1.0 (/Users/gconrad/Downloads/thing)
    Finished `release` profile [optimized] target(s) in 3.33s
     Running `target/release/thing`
thread 'main' panicked at /Users/gconrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/redb-2.1.1/src/tree_store/page_store/region.rs:73:28:
index out of bounds: the len is 21 but the index is 21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at /Users/gconrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/redb-2.1.1/src/tree_store/page_store/page_manager.rs:1054:43:
called `Result::unwrap()` on an `Err` value: PoisonError { .. }
stack backtrace:
   0:        0x1001f3a04 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h01b2beffade888b2
   1:        0x100209884 - core::fmt::write::hbadb443a71b75f23
   2:        0x1001f1d80 - std::io::Write::write_fmt::hc09d7755e3ead5f0
   3:        0x1001f385c - std::sys_common::backtrace::print::h28349e5c25acbac7
   4:        0x1001f4c1c - std::panicking::default_hook::{{closure}}::hd24b6196784d991e
   5:        0x1001f4900 - std::panicking::default_hook::hfcec80a2720c8c73
   6:        0x1001f5510 - std::panicking::rust_panic_with_hook::h84760468187ddc85
   7:        0x1001f4efc - std::panicking::begin_panic_handler::{{closure}}::he666a5eb600a7203
   8:        0x1001f3e88 - std::sys_common::backtrace::__rust_end_short_backtrace::h592f44d2bf9f843f
   9:        0x1001f4c74 - _rust_begin_unwind
  10:        0x100211444 - core::panicking::panic_fmt::h98bbf7bdf4994454
  11:        0x100211740 - core::result::unwrap_failed::h8e3b933261dd7fec
  12:        0x1001d8b44 - <redb::tree_store::page_store::page_manager::TransactionalMemory as core::ops::drop::Drop>::drop::hfe991fcf7f46db61
  13:        0x10018dcd0 - alloc::sync::Arc<T,A>::drop_slow::h51e838c18fe31e94
  14:        0x10018fe98 - core::ptr::drop_in_place<redb::db::Database>::h37fb11f6bf320a0a
  15:        0x100191df0 - thing::main::h4f7a964ed7cb9a42
  16:        0x10018f3a8 - std::sys_common::backtrace::__rust_begin_short_backtrace::h4b5c857910f82c09
  17:        0x10018f3c0 - std::rt::lang_start::{{closure}}::h580f3206fe5f58f9
  18:        0x1001f01c4 - std::rt::lang_start_internal::h39923ab4c3913741
  19:        0x100191e30 - _main
thread 'main' panicked at library/core/src/panicking.rs:164:5:
panic in a destructor during cleanup
thread caused non-unwinding panic. aborting.
fish: Job 1, 'cargo run --release' terminated by signal SIGABRT (Abort)

Thankfully, this is extremely easy to reproduce. Copy the below into a fresh cargo project that depends upon redb 2.1.1.

use redb::{Database, TableDefinition};

const TABLE: TableDefinition<&[u8], &[u8]> = TableDefinition::new("items");

fn main() {
    let db = Database::create("items.redb").unwrap();

    let txn = db.begin_write().unwrap();
    let mut table = txn.open_table(TABLE).unwrap();

    let key = (0..3_000_000_000usize)
        .map(|x| (x % 255usize) as u8)
        .collect::<Vec<_>>();
    let value = (0..2_000_000_000usize)
        .rev()
        .map(|x| (x % 255usize) as u8)
        .collect::<Vec<_>>();

    table
        .insert(key.as_slice(), value.as_slice())
        .expect("insert should work");

    drop(table);
    txn.commit().unwrap();

    let txn = db.begin_read().unwrap();
    let table = txn.open_table(TABLE).unwrap();

    table.get(key.as_slice()).expect("read should work");
}

[cberner]

The maximum size is 3GB for a key or value:

redb/src/error.rs

Lines 13 to 14 in 19fc5b5

	/// The value being inserted exceeds the maximum of 3GiB
	ValueTooLarge(usize),

But ya this crash makes sense. I didn't consider the fact that someone might insert a 3GB key and a 3GB value, and that will overflow the storage limit of a page. I'll have to think a bit on the best way to fix this. The maximum for a key + value is unspecified, but is somewhere around 4GB minus some overhead

[GregoryConrad]

Thanks for the info! Appreciate the quick turnaround