Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bump rocksdb to v7.7.3 #43

Merged
merged 2 commits into from
Dec 1, 2022

Conversation

zhangsoledad
Copy link
Member

No description provided.

@zhangsoledad
Copy link
Member Author

Fixes Compaction pending bytes calculation amplification issue that caused write stall, as discussed in: facebook/rocksdb#9423

@zhangsoledad
Copy link
Member Author

Added a benchmark on transaction read uncommitted key and compared several versions

7.3.1
transaction/transaction Get/10000
time: [39.205 ms 39.358 ms 39.551 ms]
change: [-1.8864% -1.3310% -0.7882%] (p = 0.00 < 0.05)


7.5.3
transaction/transaction Get/10000
time: [38.957 ms 39.072 ms 39.193 ms]
change: [-0.0799% +0.3521% +0.7801%] (p = 0.12 > 0.05)

transaction/transaction GetPinned/10000
time: [38.426 ms 38.550 ms 38.677 ms]
change: [+0.0409% +0.5335% +0.9862%] (p = 0.02 < 0.05)


7.7.3

transaction/transaction Get/10000
time: [38.937 ms 39.040 ms 39.145 ms]
change: [-0.1309% +0.2760% +0.6731%] (p = 0.19 > 0.05)
No change in performance detected.

transaction/transaction GetPinned/10000
time: [38.266 ms 38.373 ms 38.493 ms]
change: [-1.1004% -0.4852% +0.0421%] (p = 0.10 > 0.05)
No change in performance detected.

transaction get_pinned has some improvement in scenarios with larger values

@zhangsoledad
Copy link
Member Author

zhangsoledad commented Dec 1, 2022

Also compared with rocksdb's built-in bench_tool

./db_bench --benchmarks="fillseq,overwrite,fill100K,readrandom,deleterandom,readwhilewriting,readrandomwriterandom,seekrandom,seekrandomwhilewriting,newiterator,randomtransaction,fillseekseq"
7.3.1

fillseq      :       1.905 micros/op 524969 ops/sec 1.905 seconds 1000000 operations;   58.1 MB/s
overwrite    :       3.144 micros/op 318045 ops/sec 3.144 seconds 1000000 operations;   35.2 MB/s

fill100K     :     462.245 micros/op 2163 ops/sec 462.245 seconds 1000000 operations;  206.3 MB/s (1000 ops)
readrandom   :       0.483 micros/op 2072195 ops/sec 0.483 seconds 1000000 operations;  202.4 MB/s (1024 of 1000000 found)

deleterandom :       3.151 micros/op 317385 ops/sec 3.151 seconds 1000000 operations;
readwhilewriting :       1.763 micros/op 567089 ops/sec 1.763 seconds 1000000 operations;   26.8 MB/s (185526 of 1000000 found)

readrandomwriterandom :       5.732 micros/op 174448 ops/sec 5.732 seconds 1000000 operations; ( reads:900000 writes:100000 total:1000000 found:384754)
seekrandom   :       7.795 micros/op 128289 ops/sec 7.795 seconds 1000000 operations; (455638 of 1000000 found)

seekrandomwhilewriting :      15.864 micros/op 63037 ops/sec 15.864 seconds 1000000 operations; (838183 of 1000000 found)
newiterator  :       0.988 micros/op 1012307 ops/sec 0.988 seconds 1000000 operations;

randomtransaction :      18.218 micros/op 54892 ops/sec 18.218 seconds 1000000 operations;    1.3 MB/s ( batches:1000000 )
fillseekseq  :       3.553 micros/op 281449 ops/sec 7.106 seconds 2000000 operations;
7.7.3

fillseq      :       1.967 micros/op 508267 ops/sec 1.967 seconds 1000000 operations;   56.2 MB/s
overwrite    :       3.206 micros/op 311940 ops/sec 3.206 seconds 1000000 operations;   34.5 MB/s

fill100K     :     461.919 micros/op 2164 ops/sec 461.919 seconds 1000000 operations;  206.5 MB/s (1000 ops)
readrandom   :       0.505 micros/op 1981002 ops/sec 0.505 seconds 1000000 operations;  193.5 MB/s (1024 of 1000000 found)

deleterandom :       2.975 micros/op 336164 ops/sec 2.975 seconds 1000000 operations;
readwhilewriting :       1.361 micros/op 734806 ops/sec 1.361 seconds 1000000 operations;   34.7 MB/s (166016 of 1000000 found)

readrandomwriterandom :       4.577 micros/op 218482 ops/sec 4.577 seconds 1000000 operations; ( reads:900000 writes:100000 total:1000000 found:319268)
seekrandom   :       7.364 micros/op 135788 ops/sec 7.364 seconds 1000000 operations; (387709 of 1000000 found)

seekrandomwhilewriting :      15.719 micros/op 63618 ops/sec 15.719 seconds 1000000 operations; (816989 of 1000000 found)
newiterator  :       1.154 micros/op 866509 ops/sec 1.154 seconds 1000000 operations;

randomtransaction :      18.806 micros/op 53173 ops/sec 18.806 seconds 1000000 operations;    1.2 MB/s ( batches:1000000 )
fillseekseq  :       3.498 micros/op 285866 ops/sec 6.996 seconds 2000000 operations;

Little difference, with an improvement in concurrent scenarios

@zhangsoledad zhangsoledad merged commit 26066bb into nervosnetwork:txn Dec 1, 2022
@zhangsoledad zhangsoledad deleted the zhangsoledad/773 branch December 1, 2022 08:01
bors bot added a commit to nervosnetwork/ckb that referenced this pull request Dec 6, 2022
3733: refactor: eliminate chainstore lifecycle r=zhangsoledad a=zhangsoledad

<!--
Thank you for contributing to nervosnetwork/ckb!

If you haven't already, please read [CONTRIBUTING](https://github.com/nervosnetwork/ckb/blob/develop/CONTRIBUTING.md) document.

If you're unsure about anything, just ask; somebody should be along to answer within a day or two.

PR Title Format:
1. module [, module2, module3]: what's changed
2. *: what's changed
-->

### What is changed and how it works?


The new version of rocksdb supports transaction get_pinned, which can be used to eliminate the lifecycle on `ChainStore`, and also brings some fixes and performance improvements

nervosnetwork/rust-rocksdb#43


### Check List <!--REMOVE the items that are not applicable-->

Tests <!-- At least one of them must be included. -->

- Unit test
- Integration test


### Release note <!-- Choose from None, Title Only and Note. Bugfixes or new features need a release note. -->

```release-note
Title Only: Include only the PR title in the release note.
```



Co-authored-by: zhangsoledad <787953403@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants