Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benches for the merkle package #794

Closed
wants to merge 57 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
deb2cf5
root of transactions by stateless merkledb
bianyuanop Jan 19, 2024
0804930
make root generation a function
bianyuanop Jan 19, 2024
eb41d9a
preallocate memory for merkle array and consumebytes flag
bianyuanop Jan 19, 2024
77c7952
remove wrong merkleroot docstring
bianyuanop Jan 19, 2024
6dc40e9
add <*.code-workspace> to .gitignore and remove it from git commit
bianyuanop Jan 20, 2024
09b0b6b
move root generation func to merkle package, tx root by items of [txI…
bianyuanop Jan 20, 2024
7586f40
comments about appending outputs to slice of txID
bianyuanop Jan 20, 2024
f009424
rebase & blk marshal/unmarshal & merkleroot to ids.ID
bianyuanop Jan 28, 2024
2eae8da
write benches for the merkle package
iFrostizz Mar 18, 2024
cd10842
use crypto/rand, fix var name, report allocs
iFrostizz Mar 22, 2024
df2344f
put the 10k bench back
iFrostizz Mar 22, 2024
df5fec1
Merge branch 'main' into merkle_bench
iFrostizz Mar 22, 2024
8167c70
pass config by parameter
iFrostizz Mar 31, 2024
5800fb5
Revert "rebase & blk marshal/unmarshal & merkleroot to ids.ID"
iFrostizz Apr 15, 2024
df97603
resurrect block.go
iFrostizz Apr 15, 2024
2fa9847
Merge branch 'main' into merkle_bench
iFrostizz Apr 15, 2024
f604320
apply review changes: call root on view, nits
iFrostizz Apr 22, 2024
0c42f30
Merge branch 'main' into merkle_bench
iFrostizz Apr 22, 2024
86ca814
Merge branch 'main' into merkle_bench
iFrostizz Apr 26, 2024
40774f3
lint
iFrostizz Apr 26, 2024
c5565dd
fix lints
iFrostizz Apr 26, 2024
b86e5be
Cached state values in the program state (#840)
iFrostizz Apr 29, 2024
a781c03
root of transactions by stateless merkledb
iFrostizz Apr 29, 2024
76be5dd
make root generation a function
iFrostizz Apr 29, 2024
b87d002
add <*.code-workspace> to .gitignore and remove it from git commit
iFrostizz Apr 29, 2024
987761a
move root generation func to merkle package, tx root by items of [txI…
iFrostizz Apr 29, 2024
e42d9f9
Revert "rebase & blk marshal/unmarshal & merkleroot to ids.ID"
iFrostizz Apr 29, 2024
941bd93
sync with main
iFrostizz Apr 29, 2024
c8868ca
lint
iFrostizz Apr 29, 2024
87d1213
preallocate memory for merkle array and consumebytes flag
iFrostizz Apr 29, 2024
13e10c3
remove wrong merkleroot docstring
bianyuanop Jan 19, 2024
f12ba39
add <*.code-workspace> to .gitignore and remove it from git commit
iFrostizz Apr 29, 2024
d046582
move root generation func to merkle package, tx root by items of [txI…
iFrostizz Apr 29, 2024
4b0f9b2
rebase & blk marshal/unmarshal & merkleroot to ids.ID
iFrostizz Apr 29, 2024
46719d3
write benches for the merkle package
iFrostizz Mar 18, 2024
2543ce8
use crypto/rand, fix var name, report allocs
iFrostizz Mar 22, 2024
6451527
put the 10k bench back
iFrostizz Mar 22, 2024
a25cae6
pass config by parameter
iFrostizz Mar 31, 2024
5d861d4
Revert "rebase & blk marshal/unmarshal & merkleroot to ids.ID"
iFrostizz Apr 29, 2024
9c1b7cb
apply review changes: call root on view, nits
iFrostizz Apr 22, 2024
45ffd21
lint
iFrostizz Apr 26, 2024
e6f44fe
fix lints
iFrostizz Apr 26, 2024
3f3c06b
root of transactions by stateless merkledb
iFrostizz Apr 29, 2024
5dd3d08
make root generation a function
iFrostizz Apr 29, 2024
aeedd2f
add <*.code-workspace> to .gitignore and remove it from git commit
iFrostizz Apr 29, 2024
ab71cb5
move root generation func to merkle package, tx root by items of [txI…
iFrostizz Apr 29, 2024
9e3491b
Revert "rebase & blk marshal/unmarshal & merkleroot to ids.ID"
iFrostizz Apr 29, 2024
87e75c9
sync with main
iFrostizz Apr 29, 2024
b503be3
lint
iFrostizz Apr 29, 2024
e294b4d
Merge branch 'merkle_bench' of github.com:iFrostizz/hypersdk into mer…
iFrostizz Apr 29, 2024
d46cdbd
sync main
iFrostizz Apr 29, 2024
c28ce15
Merge branch 'main' into merkle_bench
iFrostizz Apr 30, 2024
50eac97
remove diff tag
iFrostizz May 3, 2024
633a597
Merge remote-tracking branch 'origin/main' into merkle_bench
iFrostizz May 3, 2024
fe90cfe
Merge branch 'main' into merkle_bench
iFrostizz May 3, 2024
4fe496c
Merge branch 'main' into merkle_bench
iFrostizz May 30, 2024
22d71c7
don't hash key twice
iFrostizz May 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions merkle/merkle.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// Copyright (C) 2024, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package merkle

import (
"context"

"github.com/ava-labs/avalanchego/database"
"github.com/ava-labs/avalanchego/database/memdb"
"github.com/ava-labs/avalanchego/ids"
"github.com/ava-labs/avalanchego/x/merkledb"
)

// Generate merkle root for a set of items
func GenerateMerkleRoot(ctx context.Context, config merkledb.Config, merkleItems [][]byte, consumeBytes bool) (ids.ID, merkledb.MerkleDB, error) {
batchOps := make([]database.BatchOp, 0, len(merkleItems))

for _, item := range merkleItems {
batchOps = append(batchOps, database.BatchOp{
Key: item,
Value: item,
})
}

db, err := merkledb.New(ctx, memdb.New(), config)
if err != nil {
return ids.Empty, nil, err
}

view, err := db.NewView(ctx, merkledb.ViewChanges{BatchOps: batchOps, ConsumeBytes: consumeBytes})
if err != nil {
return ids.Empty, nil, err
}

root, err := view.GetMerkleRoot(ctx)
if err != nil {
return ids.Empty, nil, err
}

return root, db, nil
}
64 changes: 64 additions & 0 deletions merkle/merkle_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Copyright (C) 2024, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package merkle

import (
"context"
"crypto/rand"
"strconv"
"testing"

"github.com/ava-labs/avalanchego/ids"
"github.com/ava-labs/avalanchego/trace"
"github.com/ava-labs/avalanchego/utils/units"
"github.com/ava-labs/avalanchego/x/merkledb"
"github.com/stretchr/testify/require"
)

var (
resRoot ids.ID
resDB merkledb.MerkleDB
resErr error
)

func BenchmarkMerkleTxRoot(b *testing.B) {
for _, size := range []int{10, 100, 1000, 10000} {
iFrostizz marked this conversation as resolved.
Show resolved Hide resolved
ctx := context.TODO()
tracer := trace.Noop
merkleItems := make([][]byte, 0, size)
for i := 0; i < size; i++ {
item := make([]byte, 32)
_, err := rand.Read(item)
require.NoError(b, err)
merkleItems = append(merkleItems, item)
}

var root ids.ID
var db merkledb.MerkleDB
var err error

defaultConfig := merkledb.Config{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really want to merge this until this config is largely removed. We need a "single instance" trie that is merkleDB-compatible for #950

BranchFactor: merkledb.BranchFactor16,
HistoryLength: 100,
IntermediateNodeCacheSize: units.MiB,
ValueNodeCacheSize: units.MiB,
Tracer: tracer,
}

b.Run(strconv.Itoa(size), func(b *testing.B) {
for n := 0; n < b.N; n++ {
iFrostizz marked this conversation as resolved.
Show resolved Hide resolved
for i := 0; i < size; i++ {
root, db, err = GenerateMerkleRoot(ctx, defaultConfig, merkleItems, false)
}
}
})

// avoid compiler optimizations to cancel out the bench
resRoot = root
resDB = db
resErr = err
}

b.ReportAllocs()
}
Loading