Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Update SHiP to work with RocksDB #9635

Merged
merged 35 commits into from
Nov 11, 2020
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
1c78b5f
move some useful common code from combined_database.cpp
victorj8 Oct 22, 2020
8f0ae9b
ship - rocksdb (wip)
victorj8 Oct 23, 2020
1badb85
Merge remote-tracking branch 'origin/develop' into rocksdb_ship
brianjohnson5972 Oct 24, 2020
de4fe84
ship - rocksdb integrationg (wip)
victorj8 Oct 26, 2020
f65fa4f
Merge branch 'develop' into rocksdb_ship
victorj8 Oct 26, 2020
763767e
modify key_value_v0 to erase database field and add payer
victorj8 Oct 26, 2020
42eadad
update abieos
victorj8 Oct 26, 2020
47edcd4
some naming changes
victorj8 Oct 26, 2020
1c3e2a7
Refactor kv snapshots to use a writer that will be available for ship…
brianjohnson5972 Oct 23, 2020
54c8f83
Improved kv snapshots test to have multiple contracts' data written t…
brianjohnson5972 Oct 23, 2020
94216bd
Moved KV/DB prefix flag to db_combined and made them both a single char.
brianjohnson5972 Oct 27, 2020
c8e1485
Added next_sub_key to return the very next key.
brianjohnson5972 Oct 27, 2020
5c8b84e
Added lambda to rocksdb_contract_kv_table_writer to stop processing b…
brianjohnson5972 Oct 27, 2020
f143313
Refactored db writer to separate table collection logic.
brianjohnson5972 Oct 27, 2020
54e10d3
Fixed rebase errors.
brianjohnson5972 Oct 27, 2020
7a798b2
More rebase fixes.
brianjohnson5972 Oct 27, 2020
ad5fe59
Another rebase error.
brianjohnson5972 Oct 27, 2020
8c0e579
Correctly wrapping kv_receiver.
brianjohnson5972 Oct 27, 2020
0fb929a
fix problem with chainbase - rocksdb tester setup
victorj8 Oct 27, 2020
3a423c4
Added method to process an individual rocksdb entry.
brianjohnson5972 Oct 28, 2020
ae9f577
ship - rocksdb (wip)
victorj8 Oct 29, 2020
0832136
rocksdb - ship (wip) all current unit tests passing
victorj8 Oct 29, 2020
8448936
ship - rocksdb refactoring rocksdb receiver and unit test updates
victorj8 Oct 31, 2020
c7fb44f
update of state history delta tests with rocksdb and update a comment
victorj8 Oct 31, 2020
88fa62e
Merge branch 'develop' into rocksdb_ship
victorj8 Oct 31, 2020
8604b47
change #warning to TODO
victorj8 Oct 31, 2020
bfbd460
fix db_previous_i64 method
victorj8 Nov 2, 2020
87c1e8e
add unit tests with both kv and multi-index
victorj8 Nov 2, 2020
017795e
Merge branch 'develop' into rocksdb_ship
victorj8 Nov 3, 2020
971dc82
fix extra comma on json
victorj8 Nov 5, 2020
13ab1b4
make section of test_deltas_contract test independent of the order we…
victorj8 Nov 5, 2020
afb2900
Merge branch 'develop' into rocksdb_ship
victorj8 Nov 9, 2020
a4f0e13
address comments on PR, refactor rocksdb receiver code and add/improv…
victorj8 Nov 10, 2020
8a46c4f
address comments on the PR
victorj8 Nov 10, 2020
c8cf8d8
Changed todo comment into a #warning.
brianjohnson5972 Nov 11, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion libraries/abieos
14 changes: 2 additions & 12 deletions libraries/chain/backing_store/db_key_value_format.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -83,16 +83,6 @@ namespace eosio { namespace chain { namespace backing_store { namespace db_key_v
return { std::get<0>(intermittent), std::get<1>(intermittent), key_loc, kt };
}

char get_rocksdb_contract_db_prefix_as_char() {
const auto prefix = make_rocksdb_contract_db_prefix();
// current design for rocksdb key format doesn't account for writing a variable width db prefix, so no
// point in designing code to write out a 1 char field as a varied length field.
EOS_ASSERT(prefix.size() == 1, bad_composite_key_exception,
"DB intrinsic key-value has been writing out the db_prefix as a defined 1 byte wide, it cannot be "
"changed to a new size without design changes");
return prefix[0];
}

// NOTE: very limited use till redesign
constexpr uint64_t db_type_and_code_size = detail::prefix_size<eosio::session::shared_bytes>() - detail::prefix_size<b1::chain_kv::bytes>(); // 1 (db type) + 8 (contract)
static_assert(db_type_and_code_size == sizeof(char) + sizeof(name), "Some assumptions on formatting have been broken");
Expand Down Expand Up @@ -215,7 +205,7 @@ namespace eosio { namespace chain { namespace backing_store { namespace db_key_v
}

eosio::session::shared_bytes create_full_key(const b1::chain_kv::bytes& composite_key, name code) {
static const char db_type_prefix = detail::get_rocksdb_contract_db_prefix_as_char();
static const char db_type_prefix = make_rocksdb_contract_db_prefix();
b1::chain_kv::bytes code_as_bytes;
b1::chain_kv::append_key(code_as_bytes, code.to_uint64_t());
auto ret = eosio::session::make_shared_bytes<std::string_view, 3>({std::string_view{&db_type_prefix, 1},
Expand Down Expand Up @@ -244,7 +234,7 @@ namespace eosio { namespace chain { namespace backing_store { namespace db_key_v
}

full_key_data parse_full_key(const eosio::session::shared_bytes& full_key) {
static const char db_type_prefix = detail::get_rocksdb_contract_db_prefix_as_char();
static const char db_type_prefix = make_rocksdb_contract_db_prefix();
EOS_ASSERT( full_key.size() >= 1, db_rocksdb_invalid_operation_exception,
"parse_full_key was passed an empty key.");
full_key_data data;
Expand Down
349 changes: 83 additions & 266 deletions libraries/chain/combined_database.cpp

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions libraries/chain/controller.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -727,6 +727,7 @@ struct controller_impl {
reversible_blocks.add_index<reversible_block_index>();

controller_index_set::add_indices(db);
db.add_index<kv_index>();
contract_database_index_set::add_indices(db);

authorization.add_indices();
Expand Down
Loading