Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem: memiavl leaf node separation optimization is not implemented #1021

Merged
merged 9 commits into from
May 8, 2023

Conversation

yihuang
Copy link
Collaborator

@yihuang yihuang commented May 5, 2023

Closes: #1016

20% improvement in RandomGet benchmark.

$ go test -run=^$ -bench="RandomGet/memiavl-disk" -benchmem ./ -count=4 -tags nativebyteorder
$ benchstat /tmp/before.txt /tmp/after.txt
goos: darwin
goarch: amd64
pkg: github.com/crypto-org-chain/cronos/memiavl
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
                          │ /tmp/before.txt │           /tmp/after.txt            │
                          │     sec/op      │    sec/op     vs base               │
RandomGet/memiavl-disk-12      175.8n ± ∞ ¹   138.2n ± ∞ ¹  -21.37% (p=0.029 n=4)
¹ need >= 6 samples for confidence interval at level 0.95

                          │ /tmp/before.txt │         /tmp/after.txt         │
                          │      B/op       │    B/op      vs base           │
RandomGet/memiavl-disk-12       0.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=1.000 n=4) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                          │ /tmp/before.txt │         /tmp/after.txt         │
                          │    allocs/op    │  allocs/op   vs base           │
RandomGet/memiavl-disk-12       0.000 ± ∞ ¹   0.000 ± ∞ ¹  ~ (p=1.000 n=4) ²
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

👮🏻👮🏻👮🏻 !!!! REFERENCE THE PROBLEM YOUR ARE SOLVING IN THE PR TITLE AND DESCRIBE YOUR SOLUTION HERE !!!! DO NOT FORGET !!!! 👮🏻👮🏻👮🏻

PR Checklist:

  • Have you read the CONTRIBUTING.md?
  • Does your PR follow the C4 patch requirements?
  • Have you rebased your work on top of the latest master?
  • Have you checked your code compiles? (make)
  • Have you included tests for any non-trivial functionality?
  • Have you checked your code passes the unit tests? (make test)
  • Have you checked your code formatting is correct? (go fmt)
  • Have you checked your basic code style is fine? (golangci-lint run)
  • If you added any dependencies, have you checked they do not contain any known vulnerabilities? (go list -json -m all | nancy sleuth)
  • If your changes affect the client infrastructure, have you run the integration test?
  • If your changes affect public APIs, does your PR follow the C4 evolution of public contracts?
  • If your code changes public APIs, have you incremented the crate version numbers and documented your changes in the CHANGELOG.md?
  • If you are contributing for the first time, please read the agreement in CONTRIBUTING.md now and add a comment to this pull request stating that your PR is in accordance with the Developer's Certificate of Origin.

Thank you for your code, it's appreciated! :)

@yihuang yihuang requested a review from a team as a code owner May 5, 2023 15:38
@yihuang yihuang requested review from mmsqe and calvinaco and removed request for a team May 5, 2023 15:38
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gosec found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

@codecov
Copy link

codecov bot commented May 5, 2023

Codecov Report

Merging #1021 (088108a) into main (6765a82) will increase coverage by 0.38%.
The diff coverage is 75.41%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1021      +/-   ##
==========================================
+ Coverage   45.32%   45.71%   +0.38%     
==========================================
  Files          80       80              
  Lines        6718     6895     +177     
==========================================
+ Hits         3045     3152     +107     
- Misses       3354     3411      +57     
- Partials      319      332      +13     
Impacted Files Coverage Δ
memiavl/snapshot.go 54.64% <60.95%> (-1.22%) ⬇️
memiavl/import.go 64.44% <66.66%> (-0.20%) ⬇️
memiavl/export.go 65.42% <67.69%> (-8.27%) ⬇️
memiavl/persisted_node.go 95.20% <94.11%> (-4.80%) ⬇️
memiavl/iterator.go 79.71% <100.00%> (ø)
memiavl/layout_little_endian.go 100.00% <100.00%> (ø)
memiavl/mem_node.go 95.08% <100.00%> (ø)
memiavl/mmap.go 68.75% <100.00%> (+8.75%) ⬆️
memiavl/node.go 73.43% <100.00%> (-0.41%) ⬇️

ch: make(chan *iavl.ExportNode, exportBufferSize),
cancel: cancel,
}
go exporter.export(ctx)

Check notice

Code scanning / CodeQL

Spawning a Go routine

Spawning a Go routine may be a possible source of non-determinism
memiavl/benchmark_test.go Outdated Show resolved Hide resolved
memiavl/mem_node.go Show resolved Hide resolved
yihuang and others added 2 commits May 8, 2023 09:57
Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Signed-off-by: yihuang <huang@crypto.com>
@yihuang yihuang enabled auto-merge May 8, 2023 02:00
@yihuang yihuang added this pull request to the merge queue May 8, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks May 8, 2023
@yihuang yihuang added this pull request to the merge queue May 8, 2023
Merged via the queue into crypto-org-chain:main with commit 81d6a4b May 8, 2023
@yihuang yihuang deleted the bsearch branch May 8, 2023 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Problem: memiavl persisted node queries is not optimal
2 participants