feat: improve lmdb dynamic growth #6242
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sdbondi
reviewed
Apr 2, 2024
base_layer/core/src/validation/aggregate_body/aggregate_body_chain_validator.rs
Outdated
Show resolved
Hide resolved
ghpbot-tari-project
added
P-acks_required
Test Results (CI) 3 files 120 suites 32m 42s ⏱️ Results for commit 9f7801b. ♻️ This comment has been updated with latest results. |
sdbondi
reviewed
Apr 2, 2024
Test Results (Integration tests) 2 files 11 suites 24m 44s ⏱️ For more details on these failures, see this check. Results for commit 9f7801b. ♻️ This comment has been updated with latest results. |
hansieodendaal
force-pushed
the
ho_improve_lmdb_dynamic_growth
branch
2 times, most recently
from
a8746c6
to
c8b2204
Compare
|
Previous commit: 2024-04-03 05:59:18.524515400 [tari_core::chain_storage::lmdb_db::helpers] TRACE lmdb_replace - 184 MB, serialize check in 148.30ms, serialize in 457.40ms
2024-04-03 05:59:18.592395600 [tari_core::chain_storage::lmdb_db::lmdb] TRACE lmdb_replace - 184 MB, lmdb write in 67.82ms
2024-04-03 05:59:19.209934000 [tari_core::chain_storage::lmdb_db::helpers] TRACE lmdb_replace - 184 MB, serialize check in 144.04ms, serialize in 460.26ms
2024-04-03 05:59:19.209983300 [tari_core::chain_storage::lmdb_db::lmdb_db] TRACE Inserted 184 MB with key '08000000' into 'tip_utxo_smt' (size 908576) in 1.29sLatest commit: 2024-04-03 08:54:38.424826600 [tari_core::chain_storage::lmdb_db::helpers] TRACE lmdb_replace - 184 MB, serialize check in 11.00µs, serialize in 458.09ms
2024-04-03 08:54:38.491925800 [tari_core::chain_storage::lmdb_db::lmdb] TRACE lmdb_replace - 184 MB, lmdb write in 67.04ms
2024-04-03 08:54:38.501529800 [tari_core::chain_storage::lmdb_db::lmdb_db] TRACE Inserted ~194 MB with key '08000000' into 'tip_utxo_smt' (size 908601) in 534.84ms |
Improved the LMDB resizing during block sync of many consecutive full blocks; this is required due to how the SMT works currently as it is replaced for every new block. Added SMT database write profiling measurements.
SWvheerden
previously approved these changes
Apr 3, 2024
ghpbot-tari-project
removed
the
P-reviews_required
hansieodendaal
force-pushed
the
ho_improve_lmdb_dynamic_growth
branch
from
c8b2204
to
2587601
Compare
hansieodendaal
force-pushed
the
ho_improve_lmdb_dynamic_growth
branch
from
2587601
to
9f7801b
Compare
ghpbot-tari-project
added
the
P-reviews_required
sdbondi
added a commit
to sdbondi/tari
that referenced
this pull request
Apr 15, 2024
* development: fix!: avoid `Encryptable` domain collisions (tari-project#6275) ci(fix): docker image build fix and ci improvements (tari-project#6270) feat: keep smt memory (tari-project#6265) feat: show warning when GRPC method is disallowed (tari-project#6246) fix(chat): metadata panic (tari-project#6247) feat: add monerod detection as an option to the merge mining proxy (tari-project#6248) chore(deps): bump h2 from 0.3.24 to 0.3.26 (tari-project#6250) feat: improve lmdb dynamic growth (tari-project#6242) feat: allow wallet type from db to have preference (tari-project#6245) feat: prevent mempool panic (tari-project#6239) ci: bump nightly version (tari-project#6241) feat: add validation for zero confirmation block sync (tari-project#6237) feat: new template with coinbase call (tari-project#6226) feat: improve wallet sql queries (tari-project#6232) chore: remove ahash as dependancy (tari-project#6238) feat: add dynamic growth to lmdb (tari-project#6231) chore(deps): bump borsh from 0.10.3 to 1.0.0 in /applications/minotari_ledger_wallet (tari-project#6236)
sdbondi
added a commit
to sdbondi/tari
that referenced
this pull request
Apr 15, 2024
* development: fix!: avoid `Encryptable` domain collisions (tari-project#6275) ci(fix): docker image build fix and ci improvements (tari-project#6270) feat: keep smt memory (tari-project#6265) feat: show warning when GRPC method is disallowed (tari-project#6246) fix(chat): metadata panic (tari-project#6247) feat: add monerod detection as an option to the merge mining proxy (tari-project#6248) chore(deps): bump h2 from 0.3.24 to 0.3.26 (tari-project#6250) feat: improve lmdb dynamic growth (tari-project#6242) feat: allow wallet type from db to have preference (tari-project#6245) feat: prevent mempool panic (tari-project#6239) ci: bump nightly version (tari-project#6241) feat: add validation for zero confirmation block sync (tari-project#6237) feat: new template with coinbase call (tari-project#6226) feat: improve wallet sql queries (tari-project#6232) chore: remove ahash as dependancy (tari-project#6238) feat: add dynamic growth to lmdb (tari-project#6231) chore(deps): bump borsh from 0.10.3 to 1.0.0 in /applications/minotari_ledger_wallet (tari-project#6236)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Motivation and Context
See above.
How Has This Been Tested?
esmeralda.fn insert_tip_smt(&self, txn: &WriteTransaction<'_>, smt: &OutputSmt)always succeeded with these settings.What process can a PR reviewer use to test or verify this change?
Code review
Breaking Changes