Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more test cases on vector size #4240

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Add more test cases on vector size #4240

wants to merge 1 commit into from

Conversation

ray6080
Copy link
Contributor

@ray6080 ray6080 commented Sep 11, 2024

Description

Copy link

Benchmark Result

Master commit hash: 7593ec9813792d26679540426302b609adb7efa1
Branch commit hash: c69b58fc8b2e2b26e42c878bb4c93d7f9c118a36

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 679.42 681.49 -2.06 (-0.30%)
aggregation q28 11962.98 11614.74 348.25 (3.00%)
filter q14 162.56 167.34 -4.78 (-2.86%)
filter q15 159.32 162.62 -3.30 (-2.03%)
filter q16 333.73 343.06 -9.33 (-2.72%)
filter q17 481.00 486.55 -5.55 (-1.14%)
filter q18 1930.70 1934.12 -3.42 (-0.18%)
fixed_size_expr_evaluator q07 581.09 569.27 11.83 (2.08%)
fixed_size_expr_evaluator q08 786.26 782.94 3.32 (0.42%)
fixed_size_expr_evaluator q09 796.10 781.78 14.33 (1.83%)
fixed_size_expr_evaluator q10 277.45 271.40 6.05 (2.23%)
fixed_size_expr_evaluator q11 272.18 267.36 4.81 (1.80%)
fixed_size_expr_evaluator q12 271.46 264.72 6.74 (2.55%)
fixed_size_expr_evaluator q13 1509.90 1510.36 -0.46 (-0.03%)
fixed_size_seq_scan q23 148.21 149.13 -0.92 (-0.62%)
join q31 11.22 6.45 4.77 (74.04%)
ldbc_snb_ic q35 766.17 768.23 -2.05 (-0.27%)
ldbc_snb_ic q36 46.30 45.93 0.36 (0.79%)
ldbc_snb_is q32 8.28 8.43 -0.16 (-1.85%)
ldbc_snb_is q33 15.39 17.91 -2.52 (-14.08%)
ldbc_snb_is q34 7.76 8.80 -1.03 (-11.76%)
multi-rel multi-rel-large-scan 2761.37 2801.42 -40.05 (-1.43%)
multi-rel multi-rel-lookup 56.52 69.61 -13.09 (-18.80%)
multi-rel multi-rel-small-scan 50.98 57.64 -6.66 (-11.55%)
order_by q25 165.26 164.81 0.45 (0.27%)
order_by q26 497.47 490.87 6.60 (1.35%)
order_by q27 1470.12 1444.58 25.54 (1.77%)
scan_after_filter q01 204.30 205.03 -0.73 (-0.35%)
scan_after_filter q02 194.61 194.35 0.26 (0.13%)
shortest_path_ldbc100 q37 3958.90 3462.77 496.13 (14.33%)
shortest_path_ldbc100 q39 112.23 109.46 2.78 (2.54%)
shortest_path_ldbc100 q40 82.43 75.18 7.25 (9.64%)
var_size_expr_evaluator q03 2138.48 2102.26 36.22 (1.72%)
var_size_expr_evaluator q04 2307.49 2267.97 39.51 (1.74%)
var_size_expr_evaluator q05 2536.70 2592.47 -55.77 (-2.15%)
var_size_expr_evaluator q06 1377.92 1422.56 -44.64 (-3.14%)
var_size_seq_scan q19 1512.02 1515.91 -3.89 (-0.26%)
var_size_seq_scan q20 3258.36 3218.01 40.35 (1.25%)
var_size_seq_scan q21 2488.38 2521.98 -33.60 (-1.33%)
var_size_seq_scan q22 136.78 136.95 -0.16 (-0.12%)

Copy link

codecov bot commented Sep 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.35%. Comparing base (d12c21f) to head (2b4fccc).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4240      +/-   ##
==========================================
- Coverage   87.36%   87.35%   -0.01%     
==========================================
  Files        1349     1349              
  Lines       56587    56587              
  Branches     7079     7079              
==========================================
- Hits        49436    49434       -2     
- Misses       6979     6981       +2     
  Partials      172      172              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

Copy link

Benchmark Result

Master commit hash: 7593ec9813792d26679540426302b609adb7efa1
Branch commit hash: aac4ea1d15f32aef8e9d27598e12d680e3820a6f

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 679.07 681.49 -2.42 (-0.35%)
aggregation q28 11950.08 11614.74 335.34 (2.89%)
filter q14 158.49 167.34 -8.84 (-5.29%)
filter q15 157.80 162.62 -4.82 (-2.96%)
filter q16 334.51 343.06 -8.55 (-2.49%)
filter q17 479.58 486.55 -6.97 (-1.43%)
filter q18 1936.69 1934.12 2.56 (0.13%)
fixed_size_expr_evaluator q07 569.30 569.27 0.03 (0.01%)
fixed_size_expr_evaluator q08 779.75 782.94 -3.19 (-0.41%)
fixed_size_expr_evaluator q09 781.84 781.78 0.06 (0.01%)
fixed_size_expr_evaluator q10 271.09 271.40 -0.31 (-0.11%)
fixed_size_expr_evaluator q11 266.43 267.36 -0.93 (-0.35%)
fixed_size_expr_evaluator q12 264.75 264.72 0.03 (0.01%)
fixed_size_expr_evaluator q13 1505.49 1510.36 -4.87 (-0.32%)
fixed_size_seq_scan q23 150.67 149.13 1.54 (1.03%)
join q31 12.38 6.45 5.93 (92.03%)
ldbc_snb_ic q35 719.04 768.23 -49.19 (-6.40%)
ldbc_snb_ic q36 46.55 45.93 0.62 (1.34%)
ldbc_snb_is q32 8.40 8.43 -0.03 (-0.36%)
ldbc_snb_is q33 16.46 17.91 -1.46 (-8.12%)
ldbc_snb_is q34 8.15 8.80 -0.64 (-7.30%)
multi-rel multi-rel-large-scan 2761.02 2801.42 -40.40 (-1.44%)
multi-rel multi-rel-lookup 46.78 69.61 -22.83 (-32.80%)
multi-rel multi-rel-small-scan 54.16 57.64 -3.48 (-6.04%)
order_by q25 163.16 164.81 -1.65 (-1.00%)
order_by q26 493.12 490.87 2.25 (0.46%)
order_by q27 1437.14 1444.58 -7.44 (-0.52%)
scan_after_filter q01 204.47 205.03 -0.56 (-0.27%)
scan_after_filter q02 193.05 194.35 -1.31 (-0.67%)
shortest_path_ldbc100 q37 3410.23 3462.77 -52.54 (-1.52%)
shortest_path_ldbc100 q39 132.04 109.46 22.58 (20.63%)
shortest_path_ldbc100 q40 73.80 75.18 -1.38 (-1.84%)
var_size_expr_evaluator q03 2104.45 2102.26 2.19 (0.10%)
var_size_expr_evaluator q04 2258.92 2267.97 -9.06 (-0.40%)
var_size_expr_evaluator q05 2578.63 2592.47 -13.85 (-0.53%)
var_size_expr_evaluator q06 1420.36 1422.56 -2.19 (-0.15%)
var_size_seq_scan q19 1519.48 1515.91 3.57 (0.24%)
var_size_seq_scan q20 3232.32 3218.01 14.31 (0.44%)
var_size_seq_scan q21 2484.10 2521.98 -37.88 (-1.50%)
var_size_seq_scan q22 137.77 136.95 0.82 (0.60%)

Copy link

Benchmark Result

Master commit hash: 5fa8fe12f36a7cbdd94f91e060768b41452582e8
Branch commit hash: 6c7c56fb61f507952f879300dfe715abc01181ee

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 658.89 654.84 4.05 (0.62%)
aggregation q28 11464.72 11446.44 18.28 (0.16%)
filter q14 125.16 129.93 -4.77 (-3.67%)
filter q15 128.05 124.42 3.63 (2.92%)
filter q16 302.11 309.76 -7.66 (-2.47%)
filter q17 456.01 451.60 4.41 (0.98%)
filter q18 1927.33 1977.03 -49.70 (-2.51%)
filter zonemap-node 86.67 88.09 -1.42 (-1.61%)
filter zonemap-node-lhs-cast 86.68 86.80 -0.12 (-0.14%)
filter zonemap-rel 5632.74 5807.31 -174.56 (-3.01%)
fixed_size_expr_evaluator q07 574.17 547.18 26.99 (4.93%)
fixed_size_expr_evaluator q08 806.18 769.69 36.49 (4.74%)
fixed_size_expr_evaluator q09 808.24 776.92 31.31 (4.03%)
fixed_size_expr_evaluator q10 240.51 243.15 -2.64 (-1.09%)
fixed_size_expr_evaluator q11 238.54 238.74 -0.20 (-0.08%)
fixed_size_expr_evaluator q12 236.66 232.04 4.61 (1.99%)
fixed_size_expr_evaluator q13 1491.42 1490.81 0.61 (0.04%)
fixed_size_seq_scan q23 114.12 116.93 -2.81 (-2.40%)
join q29 608.79 629.56 -20.77 (-3.30%)
join q30 1336.11 1445.21 -109.10 (-7.55%)
join q31 6.25 4.44 1.80 (40.60%)
ldbc_snb_ic q35 421.11 484.06 -62.95 (-13.00%)
ldbc_snb_ic q36 124.84 124.75 0.09 (0.07%)
ldbc_snb_is q32 3.89 6.28 -2.39 (-38.01%)
ldbc_snb_is q33 13.81 14.59 -0.78 (-5.36%)
ldbc_snb_is q34 1.57 1.56 0.01 (0.88%)
multi-rel multi-rel-large-scan 1721.03 1665.27 55.75 (3.35%)
multi-rel multi-rel-lookup 26.62 33.26 -6.64 (-19.97%)
multi-rel multi-rel-small-scan 96.94 104.57 -7.63 (-7.30%)
order_by q25 135.51 128.87 6.64 (5.15%)
order_by q26 453.15 451.94 1.21 (0.27%)
order_by q27 1457.13 1411.33 45.79 (3.24%)
scan_after_filter q01 170.75 175.24 -4.49 (-2.56%)
scan_after_filter q02 156.96 157.17 -0.22 (-0.14%)
shortest_path_ldbc100 q37 83.26 85.00 -1.74 (-2.04%)
shortest_path_ldbc100 q38 472.91 447.27 25.64 (5.73%)
shortest_path_ldbc100 q39 66.58 60.79 5.79 (9.53%)
shortest_path_ldbc100 q40 521.68 536.76 -15.08 (-2.81%)
var_size_expr_evaluator q03 2104.77 2076.08 28.69 (1.38%)
var_size_expr_evaluator q04 2293.48 2250.63 42.84 (1.90%)
var_size_expr_evaluator q05 2640.34 2687.92 -47.59 (-1.77%)
var_size_expr_evaluator q06 1321.77 1345.49 -23.72 (-1.76%)
var_size_seq_scan q19 1510.89 1475.77 35.13 (2.38%)
var_size_seq_scan q20 2700.72 2784.43 -83.70 (-3.01%)
var_size_seq_scan q21 2366.78 2298.10 68.69 (2.99%)
var_size_seq_scan q22 127.17 131.31 -4.14 (-3.15%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant