Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: increase max dbs #4381

Merged
merged 2 commits into from
Sep 24, 2023
Merged

chore: increase max dbs #4381

merged 2 commits into from
Sep 24, 2023

Conversation

onbjerg
Copy link
Member

@onbjerg onbjerg commented Aug 28, 2023

Currently it's not possible to create custom tables if you re-use reth because MDBX needs to be hinted how many tables ("databases") there can ever exist.

This is a breaking change because:

Controls the maximum number of named databases for the environment.

By default only unnamed key-value database could used and appropriate value should set by MDBX_opt_max_db to using any more named subDB(s). To reduce overhead, use the minimum sufficient value. This option may only set after mdbx_env_create() and before mdbx_env_open().

Link

As a follow up, an example of using a custom table could be added. However, it is important to note that this would only work if the database is completely within the control of the developer, i.e. it cannot be a sidecar process because of MDBX limitations on 1 write tx per environment.

@onbjerg onbjerg added C-debt Refactor of code section that is hard to understand or maintain A-db Related to the database S-breaking This PR includes a breaking change labels Aug 28, 2023
@codecov
Copy link

codecov bot commented Aug 28, 2023

Codecov Report

Merging #4381 (aa50e96) into breaking-changes (72b211e) will increase coverage by 0.00%.
Report is 187 commits behind head on breaking-changes.
The diff coverage is 66.66%.

❗ Current head aa50e96 differs from pull request most recent head 85ed637. Consider uploading reports for the commit 85ed637 to get more accurate results

Impacted file tree graph

Files Changed Coverage Δ
crates/storage/db/src/implementation/mdbx/mod.rs 97.44% <66.66%> (+<0.01%) ⬆️

... and 12 files with indirect coverage changes

Flag Coverage Δ
integration-tests 16.69% <0.00%> (-0.02%) ⬇️
unit-tests 63.69% <66.66%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
reth binary 26.01% <ø> (ø)
blockchain tree 82.56% <ø> (ø)
pipeline 90.07% <ø> (ø)
storage (db) 74.73% <66.66%> (+<0.01%) ⬆️
trie 94.88% <ø> (ø)
txpool 47.23% <ø> (-0.50%) ⬇️
networking 77.66% <ø> (+0.15%) ⬆️
rpc 57.96% <ø> (-0.01%) ⬇️
consensus 63.53% <ø> (ø)
revm 31.89% <ø> (ø)
payload builder 6.78% <ø> (ø)
primitives 86.30% <ø> (ø)

@gakonst
Copy link
Member

gakonst commented Aug 31, 2023

Do we have any insight on what the overhead looks like with more tables? Maybe negl?

@shekhirin shekhirin changed the base branch from main to breaking-changes September 20, 2023 13:12
@gakonst
Copy link
Member

gakonst commented Sep 20, 2023

OK w merging this in the breaking-changes branch

@onbjerg onbjerg merged commit 7f80219 into breaking-changes Sep 24, 2023
13 of 14 checks passed
@onbjerg onbjerg deleted the onbjerg/max-dbs branch September 24, 2023 22:06
rkrasiuk pushed a commit that referenced this pull request Oct 17, 2023
rkrasiuk pushed a commit that referenced this pull request Oct 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-db Related to the database C-debt Refactor of code section that is hard to understand or maintain S-breaking This PR includes a breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants