Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use version of chainbase which implements a slab allocator for small size buffers. #1070

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
+20 −9
Open

Use version of chainbase which implements a slab allocator for small size buffers. #1070

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c7e8066
Fix some incorrect allocator types mismatch.
greg7mdp Dec 4, 2024
e1438ca
Log used db size after loading the snapshot.
greg7mdp Dec 6, 2024
3f6b52c
Add call to preallocate for large tables.
greg7mdp Dec 11, 2024
d24f759
Disable preallocation which does not seem to make a significant diffe…
greg7mdp Dec 13, 2024
a1452c5
Update to latest chainbase tip (of branch `gh_1049`)
greg7mdp Dec 13, 2024
bacbc3e
Update chainbase to branch tip.
greg7mdp Dec 16, 2024
726553b
Update chainbase to branch tip.
greg7mdp Dec 16, 2024
b827aca
Merge branch 'main' of github.com:AntelopeIO/spring into gh_1049
greg7mdp Dec 16, 2024
ee5964b
Update to chainbase branch tip.
greg7mdp Dec 16, 2024
0d9a8e4
Update chainbase to branch tip.
greg7mdp Dec 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion libraries/chain/controller.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1936,8 +1936,12 @@ struct controller_impl {
}
}
});

auto snapshot_load_time = (fc::time_point::now() - snapshot_load_start_time).to_seconds();
ilog( "Finished initialization from snapshot (snapshot load time was ${t}s)", ("t", snapshot_load_time) );
auto db_size = db.get_segment_manager()->get_size();
auto free_size = db.get_segment_manager()->get_free_memory();

ilog( "Finished initialization from snapshot (snapshot load time was ${t}s, db size used is ${s} bytes)", ("t", snapshot_load_time)("s", db_size - free_size) );
} catch (boost::interprocess::bad_alloc& e) {
elog( "Failed initialization from snapshot - db storage not configured to have enough storage for the provided snapshot, please increase and retry snapshot" );
shutdown();
Expand Down Expand Up @@ -2191,6 +2195,7 @@ struct controller_impl {
section.read_row(rows_for_this_tid, db);
read_row_count.fetch_add(2u, std::memory_order_relaxed);

// utils_t::preallocate(db, rows_for_this_tid.value);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this commented out?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this would help, but upon testing it didn't seem to make a difference, so I commented it out. I thought that maybe we'd want to experiment further in the future which is why I left it commented out. In any case I probably should add a comment that it doesn't appear to make a difference.

for(size_t idx = 0; idx < rows_for_this_tid.value; idx++) {
utils_t::create(db, [this, &section, &more, &t_id](auto& row) {
row.t_id = t_id;
Expand Down
4 changes: 4 additions & 0 deletions libraries/chain/include/eosio/chain/database_utils.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@ namespace eosio::chain {
static void create( chainbase::database& db, F cons ) {
db.create<typename index_t::value_type>(cons);
}

static void preallocate( chainbase::database& db, size_t num ) {
db.preallocate<typename index_t::value_type>(num);
}
};

template<typename Index>
Expand Down
9 changes: 5 additions & 4 deletions libraries/chain/include/eosio/chain/protocol_state_object.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,11 @@ namespace eosio { namespace chain {
class protocol_state_object : public chainbase::object<protocol_state_object_type, protocol_state_object>
{
public:
template<typename Constructor>
protocol_state_object(Constructor&& c, chainbase::constructor_tag) :
id(0),
whitelisted_intrinsics(*activated_protocol_features.get_allocator(this)) {
template <typename Constructor>
protocol_state_object(Constructor&& c, chainbase::constructor_tag)
: id(0)
, whitelisted_intrinsics(
chainbase::make_allocator<whitelisted_intrinsics_type::value_type>(&whitelisted_intrinsics)) {
c(*this);
}

Expand Down
2 changes: 1 addition & 1 deletion libraries/chainbase
Open in desktop
Submodule chainbase updated 10 files
+10 −0 include/chainbase/chainbase.hpp
+46 −21 include/chainbase/chainbase_node_allocator.hpp
+10 −5 include/chainbase/environment.hpp
+45 −5 include/chainbase/pinnable_mapped_file.hpp
+1 −1 include/chainbase/shared_cow_string.hpp
+1 −1 include/chainbase/shared_cow_vector.hpp
+207 −0 include/chainbase/small_size_allocator.hpp
+4 −0 include/chainbase/undo_index.hpp
+16 −3 src/pinnable_mapped_file.cpp
+24 −16 test/test.cpp
7 changes: 4 additions & 3 deletions unittests/test_chainbase_types.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ BOOST_AUTO_TEST_CASE(chainbase_type_segment_alloc) {
fs::path temp = temp_dir.path() / "pinnable_mapped_file_101";

pinnable_mapped_file pmf(temp, true, 1024 * 1024, false, pinnable_mapped_file::map_mode::mapped);
chainbase::allocator<book> alloc(pmf.get_segment_manager());
auto seg_mgr = pmf.get_segment_manager();
auto alloc = chainbase::make_allocator<book>(seg_mgr);
bip_vector<book, chainbase::allocator<book>> v(alloc);
bip_vector<book, chainbase::allocator<book>> v2(alloc);

Expand All @@ -82,6 +83,6 @@ BOOST_AUTO_TEST_CASE(chainbase_type_segment_alloc) {
auto a2 = v2[1].authors[0].get_allocator();

// check that objects inside the vectors are allocated within the pinnable_mapped_file segment
BOOST_REQUIRE(a && (chainbase::allocator<book>)(*a) == alloc);
BOOST_REQUIRE(a2 && (chainbase::allocator<book>)(*a2) == alloc);
BOOST_REQUIRE(a && *a == chainbase::make_allocator<shared_string::allocator_type::value_type>(seg_mgr));
BOOST_REQUIRE(a2 && *a2 == chainbase::make_allocator<shared_string_vector::allocator_type::value_type>(seg_mgr));
}
Loading