Skip to content

Commit

Permalink
Make ARM64 and AMD64 docker builds more resilient (#1844)
Browse files Browse the repository at this point in the history 
Co-authored-by: John Detter <no-reply@boppygames.gg>
  • Loading branch information
@jdetter
jdetter and John Detter authored Oct 11, 2024
1 parent 208751b commit 66ea22f
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .github/workflows/docker.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ jobs:
steps:
- name: Install jq
run: sudo apt-get install jq -y
- name: Prune stale references
run: git remote prune origin
- name: Checkout
uses: actions/checkout@v3
- name: Docker meta
Expand Down Expand Up @@ -75,6 +77,8 @@ jobs:
steps:
- name: Install jq
run: sudo apt-get install jq -y
- name: Prune stale references
run: git remote prune origin
- name: Checkout
uses: actions/checkout@v3
- name: Docker meta
Expand Down

2 comments on commit 66ea22f

@github-actions
Copy link

@github-actions github-actions bot commented on 66ea22f Oct 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Criterion benchmark results

Criterion benchmark report

YOU SHOULD PROBABLY IGNORE THESE RESULTS.

Criterion is a wall time based benchmarking system that is extremely noisy when run on CI. We collect these results for longitudinal analysis, but they are not reliable for comparing individual PRs.

Go look at the callgrind report instead.

empty

db on disk new latency old latency new throughput old throughput
sqlite 💿 414.9±2.48ns 417.1±2.11ns - -
sqlite 🧠 410.3±2.45ns 405.0±2.03ns - -
stdb_raw 💿 625.5±0.83ns 625.8±0.69ns - -
stdb_raw 🧠 622.8±0.50ns 623.5±3.91ns - -

insert_1

db on disk schema indices preload new latency old latency new throughput old throughput

insert_bulk

db on disk schema indices preload count new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str btree_each_column 2048 256 589.3±0.53µs 587.0±0.87µs 1696 tx/sec 1703 tx/sec
sqlite 💿 u32_u64_str unique_0 2048 256 152.4±0.36µs 152.6±0.76µs 6.4 Ktx/sec 6.4 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 2048 256 471.4±1.56µs 468.9±0.88µs 2.1 Ktx/sec 2.1 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 2048 256 134.8±0.99µs 137.2±0.41µs 7.2 Ktx/sec 7.1 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 2048 256 453.4±0.62µs 451.9±0.58µs 2.2 Ktx/sec 2.2 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 2048 256 123.5±0.48µs 124.2±0.52µs 7.9 Ktx/sec 7.9 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 2048 256 369.9±0.44µs 372.9±0.28µs 2.6 Ktx/sec 2.6 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 2048 256 104.3±0.33µs 108.1±0.67µs 9.4 Ktx/sec 9.0 Ktx/sec
stdb_raw 💿 u32_u64_str btree_each_column 2048 256 583.0±17.31µs 574.8±49.03µs 1715 tx/sec 1739 tx/sec
stdb_raw 💿 u32_u64_str unique_0 2048 256 506.9±18.18µs 496.1±19.11µs 1972 tx/sec 2015 tx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 2048 256 371.7±11.90µs 372.0±16.31µs 2.6 Ktx/sec 2.6 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 2048 256 352.3±5.47µs 348.1±7.60µs 2.8 Ktx/sec 2.8 Ktx/sec
stdb_raw 🧠 u32_u64_str btree_each_column 2048 256 305.3±0.21µs 305.5±0.15µs 3.2 Ktx/sec 3.2 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 2048 256 240.1±0.15µs 238.4±0.24µs 4.1 Ktx/sec 4.1 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 2048 256 241.5±0.24µs 244.9±0.32µs 4.0 Ktx/sec 4.0 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 2048 256 218.5±0.25µs 220.2±0.08µs 4.5 Ktx/sec 4.4 Ktx/sec

iterate

db on disk schema indices new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str unique_0 23.5±0.17µs 24.3±0.08µs 41.6 Ktx/sec 40.2 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 21.7±0.12µs 21.6±0.09µs 45.1 Ktx/sec 45.2 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 20.8±0.08µs 21.3±0.15µs 47.0 Ktx/sec 45.8 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 19.1±0.13µs 19.4±0.05µs 51.2 Ktx/sec 50.3 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 4.7±0.00µs 4.7±0.00µs 206.0 Ktx/sec 206.8 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 4.6±0.00µs 4.6±0.00µs 210.6 Ktx/sec 211.2 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 4.7±0.00µs 4.7±0.00µs 205.6 Ktx/sec 206.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 4.6±0.00µs 4.6±0.00µs 210.6 Ktx/sec 211.1 Ktx/sec

find_unique

db on disk key type preload new latency old latency new throughput old throughput

filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string index 2048 256 68.2±0.41µs 69.1±0.34µs 14.3 Ktx/sec 14.1 Ktx/sec
sqlite 💿 u64 index 2048 256 64.3±0.12µs 65.7±0.11µs 15.2 Ktx/sec 14.9 Ktx/sec
sqlite 🧠 string index 2048 256 65.4±0.15µs 66.1±0.19µs 14.9 Ktx/sec 14.8 Ktx/sec
sqlite 🧠 u64 index 2048 256 59.5±0.42µs 60.7±0.19µs 16.4 Ktx/sec 16.1 Ktx/sec
stdb_raw 💿 string index 2048 256 4.9±0.00µs 4.9±0.00µs 198.9 Ktx/sec 197.4 Ktx/sec
stdb_raw 💿 u64 index 2048 256 4.8±0.00µs 4.7±0.00µs 204.0 Ktx/sec 208.2 Ktx/sec
stdb_raw 🧠 string index 2048 256 4.9±0.00µs 4.9±0.00µs 199.0 Ktx/sec 197.7 Ktx/sec
stdb_raw 🧠 u64 index 2048 256 4.8±0.00µs 4.7±0.00µs 203.8 Ktx/sec 208.3 Ktx/sec

serialize

schema format count new latency old latency new throughput old throughput
u32_u64_str bflatn_to_bsatn_fast_path 100 3.3±0.01µs 3.3±0.01µs 29.3 Mtx/sec 29.2 Mtx/sec
u32_u64_str bflatn_to_bsatn_slow_path 100 3.0±0.01µs 3.0±0.00µs 31.7 Mtx/sec 31.7 Mtx/sec
u32_u64_str bsatn 100 2.4±0.02µs 2.4±0.03µs 39.6 Mtx/sec 39.8 Mtx/sec
u32_u64_str bsatn 100 40.4±0.11ns 40.5±0.10ns 2.3 Gtx/sec 2.3 Gtx/sec
u32_u64_str json 100 4.8±0.03µs 4.8±0.03µs 19.8 Mtx/sec 20.0 Mtx/sec
u32_u64_str json 100 7.1±0.06µs 7.3±0.06µs 13.4 Mtx/sec 13.2 Mtx/sec
u32_u64_str product_value 100 1016.4±7.74ns 1014.1±0.80ns 93.8 Mtx/sec 94.0 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_fast_path 100 1191.3±3.71ns 1158.0±4.17ns 80.1 Mtx/sec 82.4 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_slow_path 100 2.4±0.00µs 2.4±0.00µs 39.2 Mtx/sec 39.2 Mtx/sec
u32_u64_u64 bsatn 100 1779.1±53.76ns 1685.3±29.41ns 53.6 Mtx/sec 56.6 Mtx/sec
u32_u64_u64 bsatn 100 39.4±0.11ns 39.4±0.10ns 2.4 Gtx/sec 2.4 Gtx/sec
u32_u64_u64 json 100 3.1±0.02µs 3.2±0.10µs 30.3 Mtx/sec 29.5 Mtx/sec
u32_u64_u64 json 100 4.9±0.06µs 4.9±0.03µs 19.3 Mtx/sec 19.3 Mtx/sec
u32_u64_u64 product_value 100 1013.8±1.42ns 1011.9±1.58ns 94.1 Mtx/sec 94.2 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_fast_path 100 926.3±10.93ns 923.4±1.70ns 103.0 Mtx/sec 103.3 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_slow_path 100 2.5±0.01µs 2.4±0.00µs 38.9 Mtx/sec 39.0 Mtx/sec
u64_u64_u32 bsatn 100 1690.0±31.99ns 1695.6±31.18ns 56.4 Mtx/sec 56.2 Mtx/sec
u64_u64_u32 bsatn 100 723.8±0.70ns 723.8±0.53ns 131.8 Mtx/sec 131.8 Mtx/sec
u64_u64_u32 json 100 3.3±0.03µs 3.3±0.02µs 29.3 Mtx/sec 28.7 Mtx/sec
u64_u64_u32 json 100 4.9±0.01µs 5.1±0.18µs 19.6 Mtx/sec 18.9 Mtx/sec
u64_u64_u32 product_value 100 1015.9±0.95ns 1015.5±0.83ns 93.9 Mtx/sec 93.9 Mtx/sec

stdb_module_large_arguments

arg size new latency old latency new throughput old throughput
64KiB 115.5±6.34µs 111.2±8.38µs - -

stdb_module_print_bulk

line count new latency old latency new throughput old throughput
1 56.4±5.46µs 55.6±5.99µs - -
100 597.6±19.52µs 595.4±5.45µs - -
1000 4.5±0.96ms 5.3±0.03ms - -

remaining

name new latency old latency new throughput old throughput
special/db_game/circles/load=10 290.8±3.19µs 307.3±5.78µs - -
special/db_game/circles/load=100 291.7±3.50µs 316.4±8.72µs - -
special/db_game/ia_loop/load=10 0.0±0.00ns 0.0±0.00ns - -
special/db_game/ia_loop/load=100 0.0±0.00ns 0.0±0.00ns - -
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 55.4±0.32µs 56.4±0.20µs 17.6 Ktx/sec 17.3 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 47.8±0.10µs 48.0±0.41µs 20.4 Ktx/sec 20.3 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 41.7±0.25µs 41.7±0.10µs 23.4 Ktx/sec 23.4 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 36.4±0.23µs 37.1±0.04µs 26.8 Ktx/sec 26.3 Ktx/sec
stdb_module/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 1279.3±9.18µs 1275.2±23.65µs 781 tx/sec 784 tx/sec
stdb_module/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 1032.0±17.36µs 1035.4±5.96µs 968 tx/sec 965 tx/sec
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 647.6±18.24µs 655.7±13.78µs 1544 tx/sec 1525 tx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 497.1±8.13µs 487.7±15.11µs 2011 tx/sec 2.0 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 380.0±0.40µs 376.8±0.45µs 2.6 Ktx/sec 2.6 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 342.1±1.06µs 341.7±0.55µs 2.9 Ktx/sec 2.9 Ktx/sec

@github-actions
Copy link

@github-actions github-actions bot commented on 66ea22f Oct 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 5395 5395 0.00% 5441 5441 0.00%
sqlite 5509 5509 0.00% 5871 5871 0.00%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 75388 75388 0.00% 75686 75706 -0.03%
stdb_raw u32_u64_str no_index 64 128 2 string 117799 117799 0.00% 118253 118281 -0.02%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 24061 24061 0.00% 24423 24423 0.00%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 23029 23029 0.00% 23323 23311 0.05%
sqlite u32_u64_str no_index 64 128 2 string 144677 144677 0.00% 146163 146163 0.00%
sqlite u32_u64_str no_index 64 128 1 u64 124027 124027 0.00% 125345 125345 0.00%
sqlite u32_u64_str btree_each_column 64 128 1 u64 131344 131344 0.00% 132730 132738 -0.01%
sqlite u32_u64_str btree_each_column 64 128 2 string 134482 134476 0.00% 136148 136146 0.00%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 895990 896470 -0.05% 954384 954916 -0.06%
stdb_raw u32_u64_str btree_each_column 64 128 1051061 1046825 0.40% 1125773 1090583 3.23%
sqlite u32_u64_str unique_0 64 128 398158 398158 0.00% 416100 416104 -0.00%
sqlite u32_u64_str btree_each_column 64 128 983481 983475 0.00% 1022611 1022605 0.00%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 152681 152681 0.00% 152731 152731 0.00%
stdb_raw u32_u64_str unique_0 64 15706 15706 0.00% 15772 15752 0.13%
sqlite u32_u64_str unique_0 1024 1046653 1046653 0.00% 1050109 1050109 0.00%
sqlite u32_u64_str unique_0 64 74799 74799 0.00% 75885 75885 0.00%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47374 47374 0.00% 49894 49894 0.00%
64 bsatn 25716 25716 0.00% 27960 27960 0.00%
16 bsatn 8117 8117 0.00% 9443 9443 0.00%
16 json 12126 12126 0.00% 13894 13894 0.00%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 20627797 20410627 1.06% 21475209 21248895 1.07%
stdb_raw u32_u64_str unique_0 64 128 1300359 1302697 -0.18% 1388851 1391797 -0.21%
sqlite u32_u64_str unique_0 1024 1024 1802091 1802091 0.00% 1811641 1811641 0.00%
sqlite u32_u64_str unique_0 64 128 128437 128437 0.00% 131281 131281 0.00%
On-disk benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 5405 5405 0.00% 5451 5451 0.00%
sqlite 5551 5551 0.00% 5977 5977 0.00%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 75398 75398 0.00% 75696 75672 0.03%
stdb_raw u32_u64_str no_index 64 128 2 string 117809 117793 0.01% 118239 118323 -0.07%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 24087 24071 0.07% 24453 24425 0.11%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 23039 23039 0.00% 23297 23297 0.00%
sqlite u32_u64_str no_index 64 128 1 u64 125948 125948 0.00% 127546 127550 -0.00%
sqlite u32_u64_str no_index 64 128 2 string 146598 146598 0.00% 148400 148404 -0.00%
sqlite u32_u64_str btree_each_column 64 128 2 string 136598 136598 0.00% 138642 138642 0.00%
sqlite u32_u64_str btree_each_column 64 128 1 u64 133440 133458 -0.01% 135272 135298 -0.02%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 845738 846337 -0.07% 903546 904141 -0.07%
stdb_raw u32_u64_str btree_each_column 64 128 997997 996625 0.14% 1069307 1068707 0.06%
sqlite u32_u64_str unique_0 64 128 415695 415695 0.00% 433003 433007 -0.00%
sqlite u32_u64_str btree_each_column 64 128 1021736 1021736 0.00% 1059550 1059554 -0.00%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 152691 152691 0.00% 152729 152729 0.00%
stdb_raw u32_u64_str unique_0 64 15716 15716 0.00% 15762 15758 0.03%
sqlite u32_u64_str unique_0 1024 1049721 1049721 0.00% 1053443 1053443 0.00%
sqlite u32_u64_str unique_0 64 76571 76571 0.00% 77881 77881 0.00%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47374 47374 0.00% 49894 49894 0.00%
64 bsatn 25716 25716 0.00% 27960 27960 0.00%
16 bsatn 8117 8117 0.00% 9443 9443 0.00%
16 json 12126 12126 0.00% 13894 13894 0.00%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 19331872 19329835 0.01% 20247656 20246059 0.01%
stdb_raw u32_u64_str unique_0 64 128 1254488 1254331 0.01% 1342118 1342039 0.01%
sqlite u32_u64_str unique_0 1024 1024 1809652 1809652 0.00% 1818326 1818322 0.00%
sqlite u32_u64_str unique_0 64 128 132581 132569 0.01% 135625 135609 0.01%

Please sign in to comment.