Skip to content

Commit

Permalink
Rust module API rework (#1660)
Browse files Browse the repository at this point in the history
  • Loading branch information
@coolreader18
coolreader18 authored Sep 27, 2024
1 parent 1e31b23 commit 5375842
Show file tree
Hide file tree
Showing 141 changed files with 2,200 additions and 2,286 deletions.
1 change: 1 addition & 0 deletions Cargo.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

2 comments on commit 5375842

@github-actions
Copy link

@github-actions github-actions bot commented on 5375842 Sep 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 5395 5409 -0.26% 5471 5569 -1.76%
sqlite 5555 5555 0.00% 6043 5971 1.21%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 2 string 117797 117832 -0.03% 118161 118548 -0.33%
stdb_raw u32_u64_str no_index 64 128 1 u64 75385 75399 -0.02% 75659 76069 -0.54%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 24072 24068 0.02% 24444 24636 -0.78%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 23021 23035 -0.06% 23399 23577 -0.75%
sqlite u32_u64_str no_index 64 128 2 string 144677 144677 0.00% 146211 146087 0.08%
sqlite u32_u64_str no_index 64 128 1 u64 124026 124026 0.00% 125346 125234 0.09%
sqlite u32_u64_str btree_each_column 64 128 2 string 134476 134494 -0.01% 136134 136084 0.04%
sqlite u32_u64_str btree_each_column 64 128 1 u64 131343 131343 0.00% 132863 132831 0.02%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 901946 902752 -0.09% 953552 958002 -0.46%
stdb_raw u32_u64_str btree_each_column 64 128 1054498 1056772 -0.22% 1088774 1097812 -0.82%
sqlite u32_u64_str unique_0 64 128 398292 398292 0.00% 414896 415568 -0.16%
sqlite u32_u64_str btree_each_column 64 128 983609 983609 0.00% 1020479 1023511 -0.30%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 152681 152695 -0.01% 152719 152853 -0.09%
stdb_raw u32_u64_str unique_0 64 15706 15720 -0.09% 15744 15878 -0.84%
sqlite u32_u64_str unique_0 1024 1068223 1068223 0.00% 1071583 1071623 -0.00%
sqlite u32_u64_str unique_0 64 76209 76209 0.00% 77251 77331 -0.10%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47118 47118 0.00% 49804 49804 0.00%
64 bsatn 25716 25716 0.00% 28028 27994 0.12%
16 json 12062 12062 0.00% 14000 14000 0.00%
16 bsatn 8117 8117 0.00% 9545 9511 0.36%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 20675427 20714193 -0.19% 21486975 21464241 0.11%
stdb_raw u32_u64_str unique_0 64 128 1307741 1307357 0.03% 1363041 1359219 0.28%
sqlite u32_u64_str unique_0 1024 1024 1802091 1802091 0.00% 1811427 1811253 0.01%
sqlite u32_u64_str unique_0 64 128 128437 128437 0.00% 131167 131263 -0.07%
On-disk benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 5405 5419 -0.26% 5485 5583 -1.76%
sqlite 5613 5603 0.18% 6213 6193 0.32%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 2 string 117807 117821 -0.01% 118135 118581 -0.38%
stdb_raw u32_u64_str no_index 64 128 1 u64 75395 75409 -0.02% 75665 76067 -0.53%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 24064 24079 -0.06% 24436 24643 -0.84%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 23031 23045 -0.06% 23401 23603 -0.86%
sqlite u32_u64_str no_index 64 128 2 string 146598 146598 0.00% 148472 148332 0.09%
sqlite u32_u64_str no_index 64 128 1 u64 125965 125947 0.01% 127729 127535 0.15%
sqlite u32_u64_str btree_each_column 64 128 2 string 136598 136598 0.00% 138794 138674 0.09%
sqlite u32_u64_str btree_each_column 64 128 1 u64 133439 133439 0.00% 135433 135265 0.12%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 853144 851804 0.16% 906772 906228 0.06%
stdb_raw u32_u64_str btree_each_column 64 128 1003623 1004541 -0.09% 1068585 1076947 -0.78%
sqlite u32_u64_str unique_0 64 128 415829 415829 0.00% 431723 432339 -0.14%
sqlite u32_u64_str btree_each_column 64 128 1021870 1021870 0.00% 1057898 1060852 -0.28%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 152691 152705 -0.01% 152725 152855 -0.09%
stdb_raw u32_u64_str unique_0 64 15716 15730 -0.09% 15750 15880 -0.82%
sqlite u32_u64_str unique_0 1024 1071297 1071291 0.00% 1075043 1075081 -0.00%
sqlite u32_u64_str unique_0 64 77981 77981 0.00% 79263 79375 -0.14%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47118 47118 0.00% 49804 49804 0.00%
64 bsatn 25716 25716 0.00% 28028 27994 0.12%
16 json 12062 12062 0.00% 14000 14000 0.00%
16 bsatn 8117 8117 0.00% 9545 9511 0.36%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 19417013 19417210 -0.00% 20310857 20233794 0.38%
stdb_raw u32_u64_str unique_0 64 128 1260539 1259757 0.06% 1346175 1341037 0.38%
sqlite u32_u64_str unique_0 1024 1024 1809652 1809652 0.00% 1818240 1818254 -0.00%
sqlite u32_u64_str unique_0 64 128 132563 132563 0.00% 135537 135525 0.01%

@github-actions
Copy link

@github-actions github-actions bot commented on 5375842 Sep 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Criterion benchmark results

Criterion benchmark report

YOU SHOULD PROBABLY IGNORE THESE RESULTS.

Criterion is a wall time based benchmarking system that is extremely noisy when run on CI. We collect these results for longitudinal analysis, but they are not reliable for comparing individual PRs.

Go look at the callgrind report instead.

empty

db on disk new latency old latency new throughput old throughput
sqlite 💿 409.8±2.28ns 417.8±2.63ns - -
sqlite 🧠 402.8±1.24ns 408.8±1.96ns - -
stdb_raw 💿 625.7±0.23ns 625.6±0.68ns - -
stdb_raw 🧠 624.1±0.45ns 625.7±0.68ns - -

insert_1

db on disk schema indices preload new latency old latency new throughput old throughput

insert_bulk

db on disk schema indices preload count new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str btree_each_column 2048 256 585.5±0.86µs 587.8±1.05µs 1707 tx/sec 1701 tx/sec
sqlite 💿 u32_u64_str unique_0 2048 256 153.1±0.24µs 154.1±9.47µs 6.4 Ktx/sec 6.3 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 2048 256 472.6±15.25µs 466.3±0.62µs 2.1 Ktx/sec 2.1 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 2048 256 135.9±0.57µs 133.5±0.55µs 7.2 Ktx/sec 7.3 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 2048 256 451.7±0.46µs 449.8±0.51µs 2.2 Ktx/sec 2.2 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 2048 256 122.6±0.93µs 122.6±1.45µs 8.0 Ktx/sec 8.0 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 2048 256 365.3±0.36µs 364.0±0.89µs 2.7 Ktx/sec 2.7 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 2048 256 104.3±0.66µs 103.0±0.69µs 9.4 Ktx/sec 9.5 Ktx/sec
stdb_raw 💿 u32_u64_str btree_each_column 2048 256 597.2±22.51µs 579.4±31.29µs 1674 tx/sec 1725 tx/sec
stdb_raw 💿 u32_u64_str unique_0 2048 256 469.2±35.67µs 492.4±26.05µs 2.1 Ktx/sec 2030 tx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 2048 256 338.0±9.91µs 379.3±8.15µs 2.9 Ktx/sec 2.6 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 2048 256 302.2±6.67µs 350.8±13.10µs 3.2 Ktx/sec 2.8 Ktx/sec
stdb_raw 🧠 u32_u64_str btree_each_column 2048 256 309.1±0.24µs 313.0±0.89µs 3.2 Ktx/sec 3.1 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 2048 256 240.9±0.14µs 244.7±0.18µs 4.1 Ktx/sec 4.0 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 2048 256 245.6±0.15µs 251.4±0.29µs 4.0 Ktx/sec 3.9 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 2048 256 221.1±0.20µs 222.7±0.24µs 4.4 Ktx/sec 4.4 Ktx/sec

iterate

db on disk schema indices new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str unique_0 23.2±0.21µs 20.8±0.16µs 42.1 Ktx/sec 47.0 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 21.9±0.13µs 20.1±0.05µs 44.6 Ktx/sec 48.7 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 20.6±0.20µs 18.3±0.25µs 47.5 Ktx/sec 53.3 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 19.3±0.10µs 17.6±0.06µs 50.6 Ktx/sec 55.6 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 4.7±0.00µs 4.7±0.00µs 206.1 Ktx/sec 206.9 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 4.6±0.00µs 4.6±0.00µs 210.8 Ktx/sec 211.1 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 4.7±0.00µs 4.7±0.00µs 206.2 Ktx/sec 206.9 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 4.6±0.00µs 4.6±0.00µs 211.0 Ktx/sec 211.0 Ktx/sec

find_unique

db on disk key type preload new latency old latency new throughput old throughput

filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string index 2048 256 70.7±0.25µs 66.6±0.14µs 13.8 Ktx/sec 14.7 Ktx/sec
sqlite 💿 u64 index 2048 256 67.6±0.25µs 65.5±0.12µs 14.4 Ktx/sec 14.9 Ktx/sec
sqlite 🧠 string index 2048 256 66.4±0.33µs 63.9±0.12µs 14.7 Ktx/sec 15.3 Ktx/sec
sqlite 🧠 u64 index 2048 256 61.9±0.28µs 59.9±0.05µs 15.8 Ktx/sec 16.3 Ktx/sec
stdb_raw 💿 string index 2048 256 4.8±0.00µs 4.8±0.00µs 201.6 Ktx/sec 202.2 Ktx/sec
stdb_raw 💿 u64 index 2048 256 4.8±0.00µs 4.8±0.00µs 204.0 Ktx/sec 205.1 Ktx/sec
stdb_raw 🧠 string index 2048 256 4.8±0.00µs 4.8±0.00µs 202.1 Ktx/sec 202.4 Ktx/sec
stdb_raw 🧠 u64 index 2048 256 4.8±0.00µs 4.8±0.00µs 204.2 Ktx/sec 205.3 Ktx/sec

serialize

schema format count new latency old latency new throughput old throughput
u32_u64_str bflatn_to_bsatn_fast_path 100 3.3±0.01µs 3.3±0.01µs 29.0 Mtx/sec 28.7 Mtx/sec
u32_u64_str bflatn_to_bsatn_slow_path 100 3.4±0.00µs 3.8±0.11µs 27.8 Mtx/sec 25.2 Mtx/sec
u32_u64_str bsatn 100 2.3±0.00µs 2.3±0.01µs 41.1 Mtx/sec 42.2 Mtx/sec
u32_u64_str bsatn 100 40.5±0.04ns 40.4±0.10ns 2.3 Gtx/sec 2.3 Gtx/sec
u32_u64_str json 100 4.6±0.05µs 4.8±0.02µs 20.6 Mtx/sec 19.9 Mtx/sec
u32_u64_str json 100 6.8±0.05µs 6.9±0.11µs 14.1 Mtx/sec 13.9 Mtx/sec
u32_u64_str product_value 100 1015.3±1.53ns 1015.1±1.08ns 93.9 Mtx/sec 93.9 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_fast_path 100 1131.0±3.35ns 1132.7±3.10ns 84.3 Mtx/sec 84.2 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_slow_path 100 2.8±0.01µs 2.8±0.01µs 34.0 Mtx/sec 34.4 Mtx/sec
u32_u64_u64 bsatn 100 1613.7±31.02ns 1545.7±14.74ns 59.1 Mtx/sec 61.7 Mtx/sec
u32_u64_u64 bsatn 100 39.7±0.04ns 39.3±0.04ns 2.3 Gtx/sec 2.4 Gtx/sec
u32_u64_u64 json 100 3.2±0.03µs 3.3±0.15µs 30.1 Mtx/sec 28.5 Mtx/sec
u32_u64_u64 json 100 4.6±0.01µs 4.8±0.03µs 20.6 Mtx/sec 20.0 Mtx/sec
u32_u64_u64 product_value 100 1013.2±1.16ns 1013.7±1.16ns 94.1 Mtx/sec 94.1 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_fast_path 100 899.1±1.48ns 894.6±2.71ns 106.1 Mtx/sec 106.6 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_slow_path 100 2.8±0.04µs 2.8±0.00µs 34.1 Mtx/sec 34.3 Mtx/sec
u64_u64_u32 bsatn 100 1641.1±40.20ns 1542.4±8.83ns 58.1 Mtx/sec 61.8 Mtx/sec
u64_u64_u32 bsatn 100 706.4±0.66ns 749.4±7.06ns 135.0 Mtx/sec 127.3 Mtx/sec
u64_u64_u32 json 100 3.2±0.10µs 3.6±0.03µs 29.8 Mtx/sec 26.3 Mtx/sec
u64_u64_u32 json 100 4.7±0.02µs 4.7±0.04µs 20.2 Mtx/sec 20.3 Mtx/sec
u64_u64_u32 product_value 100 1015.9±0.81ns 1018.2±1.18ns 93.9 Mtx/sec 93.7 Mtx/sec

stdb_module_large_arguments

arg size new latency old latency new throughput old throughput
64KiB 108.1±11.46µs 108.4±8.18µs - -

stdb_module_print_bulk

line count new latency old latency new throughput old throughput
1 51.3±5.18µs 55.3±5.69µs - -
100 603.0±4.86µs 606.6±4.59µs - -
1000 5.3±0.61ms 5.1±0.78ms - -

remaining

name new latency old latency new throughput old throughput
special/db_game/circles/load=10 272.2±3.60µs 272.6±3.13µs - -
special/db_game/circles/load=100 275.7±3.47µs 271.2±4.81µs - -
special/db_game/ia_loop/load=10 0.0±0.00ns 0.0±0.00ns - -
special/db_game/ia_loop/load=100 0.0±0.00ns 0.0±0.00ns - -
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 55.7±0.08µs 54.2±0.19µs 17.5 Ktx/sec 18.0 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 47.5±0.19µs 46.5±0.06µs 20.5 Ktx/sec 21.0 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 40.7±0.25µs 39.1±0.44µs 24.0 Ktx/sec 25.0 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 36.1±0.26µs 35.7±0.15µs 27.0 Ktx/sec 27.4 Ktx/sec
stdb_module/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 1249.6±13.41µs 1246.4±24.64µs 800 tx/sec 802 tx/sec
stdb_module/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 993.8±10.01µs 988.0±13.00µs 1006 tx/sec 1012 tx/sec
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 641.8±22.56µs 626.4±23.69µs 1558 tx/sec 1596 tx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 491.4±6.25µs 487.6±14.00µs 2034 tx/sec 2.0 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 374.2±0.68µs 378.6±0.97µs 2.6 Ktx/sec 2.6 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 337.1±0.35µs 336.3±0.49µs 2.9 Ktx/sec 2.9 Ktx/sec

Please sign in to comment.