Skip to content

Commit

Permalink
Upgrade wasmtime dependency (#1857)
Browse files Browse the repository at this point in the history
  • Loading branch information
@coolreader18
coolreader18 authored Oct 16, 2024
1 parent 576b12e commit ce3d6ed
Show file tree
Hide file tree
Showing 3 changed files with 316 additions and 244 deletions.
Loading

2 comments on commit ce3d6ed

@github-actions
Copy link

@github-actions github-actions bot commented on ce3d6ed Oct 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Criterion benchmark results

Criterion benchmark report

YOU SHOULD PROBABLY IGNORE THESE RESULTS.

Criterion is a wall time based benchmarking system that is extremely noisy when run on CI. We collect these results for longitudinal analysis, but they are not reliable for comparing individual PRs.

Go look at the callgrind report instead.

empty

db on disk new latency old latency new throughput old throughput
sqlite 💿 412.9±2.50ns 414.7±3.55ns - -
sqlite 🧠 408.1±1.31ns 409.7±1.61ns - -
stdb_raw 💿 651.6±0.98ns 631.7±1.11ns - -
stdb_raw 🧠 651.4±1.79ns 632.4±0.76ns - -

insert_1

db on disk schema indices preload new latency old latency new throughput old throughput

insert_bulk

db on disk schema indices preload count new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str btree_each_column 2048 256 587.7±0.80µs 583.3±0.91µs 1701 tx/sec 1714 tx/sec
sqlite 💿 u32_u64_str unique_0 2048 256 147.4±0.69µs 151.8±1.70µs 6.6 Ktx/sec 6.4 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 2048 256 467.9±0.68µs 466.5±0.53µs 2.1 Ktx/sec 2.1 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 2048 256 135.3±0.59µs 134.3±0.42µs 7.2 Ktx/sec 7.3 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 2048 256 448.6±0.75µs 448.9±1.76µs 2.2 Ktx/sec 2.2 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 2048 256 121.5±1.41µs 124.0±0.47µs 8.0 Ktx/sec 7.9 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 2048 256 365.2±0.51µs 368.4±0.27µs 2.7 Ktx/sec 2.7 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 2048 256 103.6±0.35µs 105.7±0.38µs 9.4 Ktx/sec 9.2 Ktx/sec
stdb_raw 💿 u32_u64_str btree_each_column 2048 256 643.6±28.29µs 623.1±17.52µs 1553 tx/sec 1604 tx/sec
stdb_raw 💿 u32_u64_str unique_0 2048 256 451.1±49.86µs 417.6±13.20µs 2.2 Ktx/sec 2.3 Ktx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 2048 256 381.8±6.37µs 311.0±42.26µs 2.6 Ktx/sec 3.1 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 2048 256 350.7±15.76µs 358.9±7.64µs 2.8 Ktx/sec 2.7 Ktx/sec
stdb_raw 🧠 u32_u64_str btree_each_column 2048 256 309.1±0.25µs 308.9±0.29µs 3.2 Ktx/sec 3.2 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 2048 256 238.7±0.29µs 240.0±0.12µs 4.1 Ktx/sec 4.1 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 2048 256 245.2±0.14µs 246.1±0.45µs 4.0 Ktx/sec 4.0 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 2048 256 219.8±0.26µs 221.9±0.60µs 4.4 Ktx/sec 4.4 Ktx/sec

iterate

db on disk schema indices new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str unique_0 23.2±0.39µs 22.8±0.16µs 42.1 Ktx/sec 42.9 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 21.5±0.45µs 20.9±0.16µs 45.5 Ktx/sec 46.7 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 20.2±0.19µs 20.2±0.13µs 48.3 Ktx/sec 48.2 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 19.2±0.26µs 18.6±0.32µs 50.8 Ktx/sec 52.5 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 4.8±0.00µs 4.7±0.00µs 204.0 Ktx/sec 206.0 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 4.7±0.00µs 4.6±0.00µs 209.8 Ktx/sec 210.8 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 4.8±0.00µs 4.7±0.00µs 204.1 Ktx/sec 205.9 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 4.7±0.00µs 4.6±0.00µs 209.6 Ktx/sec 210.9 Ktx/sec

find_unique

db on disk key type preload new latency old latency new throughput old throughput

filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string index 2048 256 67.8±0.12µs 68.1±0.16µs 14.4 Ktx/sec 14.3 Ktx/sec
sqlite 💿 u64 index 2048 256 66.9±0.30µs 65.2±0.34µs 14.6 Ktx/sec 15.0 Ktx/sec
sqlite 🧠 string index 2048 256 64.4±0.08µs 64.7±0.11µs 15.2 Ktx/sec 15.1 Ktx/sec
sqlite 🧠 u64 index 2048 256 61.0±0.12µs 59.7±0.12µs 16.0 Ktx/sec 16.4 Ktx/sec
stdb_raw 💿 string index 2048 256 4.9±0.00µs 4.9±0.00µs 197.7 Ktx/sec 198.6 Ktx/sec
stdb_raw 💿 u64 index 2048 256 4.8±0.00µs 4.7±0.00µs 203.9 Ktx/sec 208.0 Ktx/sec
stdb_raw 🧠 string index 2048 256 5.0±0.00µs 4.9±0.01µs 197.3 Ktx/sec 198.6 Ktx/sec
stdb_raw 🧠 u64 index 2048 256 4.8±0.00µs 4.7±0.01µs 203.9 Ktx/sec 207.9 Ktx/sec

serialize

schema format count new latency old latency new throughput old throughput
u32_u64_str bflatn_to_bsatn_fast_path 100 3.3±0.01µs 3.3±0.09µs 29.3 Mtx/sec 29.1 Mtx/sec
u32_u64_str bflatn_to_bsatn_slow_path 100 3.4±0.01µs 3.4±0.00µs 28.0 Mtx/sec 28.2 Mtx/sec
u32_u64_str bsatn 100 2.4±0.00µs 2.5±0.04µs 39.1 Mtx/sec 38.8 Mtx/sec
u32_u64_str bsatn 100 40.5±0.10ns 40.5±0.09ns 2.3 Gtx/sec 2.3 Gtx/sec
u32_u64_str json 100 5.7±0.02µs 4.8±0.03µs 16.7 Mtx/sec 19.7 Mtx/sec
u32_u64_str json 100 7.3±0.06µs 8.6±0.01µs 13.1 Mtx/sec 11.0 Mtx/sec
u32_u64_str product_value 100 1016.6±6.67ns 1015.2±3.40ns 93.8 Mtx/sec 93.9 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_fast_path 100 1133.2±1.18ns 1185.0±15.96ns 84.2 Mtx/sec 80.5 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_slow_path 100 2.8±0.00µs 2.8±0.00µs 34.1 Mtx/sec 34.2 Mtx/sec
u32_u64_u64 bsatn 100 1773.9±24.73ns 1736.5±30.34ns 53.8 Mtx/sec 54.9 Mtx/sec
u32_u64_u64 bsatn 100 39.7±0.15ns 39.4±0.07ns 2.3 Gtx/sec 2.4 Gtx/sec
u32_u64_u64 json 100 3.1±0.01µs 3.4±0.03µs 30.8 Mtx/sec 28.1 Mtx/sec
u32_u64_u64 json 100 5.1±0.00µs 4.8±0.03µs 18.7 Mtx/sec 19.8 Mtx/sec
u32_u64_u64 product_value 100 1013.3±1.39ns 1012.7±0.97ns 94.1 Mtx/sec 94.2 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_fast_path 100 925.4±10.53ns 914.9±7.73ns 103.1 Mtx/sec 104.2 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_slow_path 100 2.8±0.00µs 2.8±0.00µs 34.1 Mtx/sec 34.1 Mtx/sec
u64_u64_u32 bsatn 100 1756.7±30.60ns 1843.5±49.57ns 54.3 Mtx/sec 51.7 Mtx/sec
u64_u64_u32 bsatn 100 943.2±2.39ns 756.2±0.34ns 101.1 Mtx/sec 126.1 Mtx/sec
u64_u64_u32 json 100 3.9±0.01µs 3.2±0.03µs 24.7 Mtx/sec 30.1 Mtx/sec
u64_u64_u32 json 100 5.3±0.01µs 4.7±0.01µs 18.1 Mtx/sec 20.5 Mtx/sec
u64_u64_u32 product_value 100 1017.5±0.57ns 1015.0±0.79ns 93.7 Mtx/sec 94.0 Mtx/sec

stdb_module_large_arguments

arg size new latency old latency new throughput old throughput
64KiB 101.7±8.12µs 108.0±9.68µs - -

stdb_module_print_bulk

line count new latency old latency new throughput old throughput
1 58.2±4.87µs 51.9±7.85µs - -
100 591.9±6.90µs 597.7±8.26µs - -
1000 4.3±0.96ms 4.5±0.99ms - -

remaining

name new latency old latency new throughput old throughput
special/db_game/circles/load=10 298.0±2.41µs 290.8±2.35µs - -
special/db_game/circles/load=100 300.2±4.17µs 288.4±3.05µs - -
special/db_game/ia_loop/load=10 0.0±0.00ns 0.0±0.00ns - -
special/db_game/ia_loop/load=100 0.0±0.00ns 0.0±0.00ns - -
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 55.1±0.34µs 53.9±0.40µs 17.7 Ktx/sec 18.1 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 46.8±0.19µs 46.5±0.22µs 20.9 Ktx/sec 21.0 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 41.8±0.29µs 38.9±0.34µs 23.4 Ktx/sec 25.1 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 36.7±0.20µs 35.6±0.17µs 26.6 Ktx/sec 27.4 Ktx/sec
stdb_module/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 1297.0±22.60µs 1298.6±15.75µs 771 tx/sec 770 tx/sec
stdb_module/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 1034.8±8.87µs 1041.4±5.34µs 966 tx/sec 960 tx/sec
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 611.2±40.03µs 541.3±20.64µs 1636 tx/sec 1847 tx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 486.5±7.67µs 492.2±8.82µs 2.0 Ktx/sec 2031 tx/sec
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 375.3±0.72µs 374.9±0.48µs 2.6 Ktx/sec 2.6 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 335.2±0.38µs 337.0±0.33µs 2.9 Ktx/sec 2.9 Ktx/sec

@github-actions
Copy link

@github-actions github-actions bot commented on ce3d6ed Oct 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 5386 5386 0.00% 5462 5486 -0.44%
sqlite 5575 5575 0.00% 5975 5987 -0.20%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 75579 75577 0.00% 76081 75981 0.13%
stdb_raw u32_u64_str no_index 64 128 2 string 118076 118074 0.00% 118648 118658 -0.01%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 24095 24068 0.11% 24567 24464 0.42%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 23063 23036 0.12% 23449 23352 0.42%
sqlite u32_u64_str no_index 64 128 2 string 144663 144661 0.00% 146053 146249 -0.13%
sqlite u32_u64_str no_index 64 128 1 u64 124013 124011 0.00% 125205 125295 -0.07%
sqlite u32_u64_str btree_each_column 64 128 1 u64 131340 131338 0.00% 132864 132726 0.10%
sqlite u32_u64_str btree_each_column 64 128 2 string 134462 134460 0.00% 136184 136092 0.07%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 900421 902512 -0.23% 924107 924682 -0.06%
stdb_raw u32_u64_str btree_each_column 64 128 1050416 1048713 0.16% 1119976 1088675 2.88%
sqlite u32_u64_str unique_0 64 128 398284 398284 0.00% 413300 417858 -1.09%
sqlite u32_u64_str btree_each_column 64 128 983611 983611 0.00% 1022231 1030111 -0.76%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 152706 152706 0.00% 152764 152784 -0.01%
stdb_raw u32_u64_str unique_0 64 15731 15731 0.00% 15785 15797 -0.08%
sqlite u32_u64_str unique_0 1024 1067233 1067233 0.00% 1070511 1070657 -0.01%
sqlite u32_u64_str unique_0 64 76179 76179 0.00% 77195 77289 -0.12%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47182 47182 0.00% 49770 49736 0.07%
64 bsatn 25716 25716 0.00% 27858 27960 -0.36%
16 bsatn 8117 8117 0.00% 9341 9477 -1.44%
16 json 12078 12078 0.00% 13914 13880 0.24%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 20751010 20506515 1.19% 21422270 21285041 0.64%
stdb_raw u32_u64_str unique_0 64 128 1310896 1307491 0.26% 1386866 1392603 -0.41%
sqlite u32_u64_str unique_0 1024 1024 1802089 1802083 0.00% 1811175 1811283 -0.01%
sqlite u32_u64_str unique_0 64 128 128429 128429 0.00% 131185 131223 -0.03%
On-disk benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 5396 5396 0.00% 5480 5496 -0.29%
sqlite 5617 5617 0.00% 6085 6097 -0.20%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 75589 75587 0.00% 76087 75995 0.12%
stdb_raw u32_u64_str no_index 64 128 2 string 118107 118084 0.02% 118651 118772 -0.10%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 24105 24078 0.11% 24569 24510 0.24%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 23073 23046 0.12% 23459 23358 0.43%
sqlite u32_u64_str no_index 64 128 1 u64 125934 125932 0.00% 127386 127472 -0.07%
sqlite u32_u64_str no_index 64 128 2 string 146584 146582 0.00% 148218 148370 -0.10%
sqlite u32_u64_str btree_each_column 64 128 2 string 136584 136582 0.00% 138772 138684 0.06%
sqlite u32_u64_str btree_each_column 64 128 1 u64 133426 133424 0.00% 135364 135262 0.08%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 849927 852256 -0.27% 903613 904216 -0.07%
stdb_raw u32_u64_str btree_each_column 64 128 996977 1000613 -0.36% 1035961 1068567 -3.05%
sqlite u32_u64_str unique_0 64 128 415821 415821 0.00% 430275 434705 -1.02%
sqlite u32_u64_str btree_each_column 64 128 1021862 1021862 0.00% 1059774 1066916 -0.67%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 152716 152716 0.00% 152762 152782 -0.01%
stdb_raw u32_u64_str unique_0 64 15741 15741 0.00% 15787 15807 -0.13%
sqlite u32_u64_str unique_0 1024 1070301 1070301 0.00% 1074085 1074055 0.00%
sqlite u32_u64_str unique_0 64 77951 77951 0.00% 79231 79249 -0.02%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47182 47182 0.00% 49770 49736 0.07%
64 bsatn 25716 25716 0.00% 27858 27960 -0.36%
16 bsatn 8117 8117 0.00% 9341 9477 -1.44%
16 json 12078 12078 0.00% 13914 13880 0.24%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 19441634 19429585 0.06% 20164854 20289121 -0.61%
stdb_raw u32_u64_str unique_0 64 128 1262862 1260911 0.15% 1339556 1344117 -0.34%
sqlite u32_u64_str unique_0 1024 1024 1809644 1809644 0.00% 1818190 1818188 0.00%
sqlite u32_u64_str unique_0 64 128 132555 132555 0.00% 135407 135401 0.00%

Please sign in to comment.