Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize catalog access #4058

Merged
merged 2 commits into from
Aug 12, 2024
Merged

Optimize catalog access #4058

merged 2 commits into from
Aug 12, 2024

Conversation

andyfengHKU
Copy link
Contributor

@andyfengHKU andyfengHKU commented Aug 12, 2024

Description

getTableCatalogEntry API in the catalog is introducing noticeable overhead when binding large query graph since we now need to check different versions of entry under MVCC.

This PR directly save TableCatalogEntry* in node/rel expression instead of tableID to avoid repeated access of catalog entries.

Fixes # (issue)

Contributor agreement

Copy link

Benchmark Result

Master commit hash: 6a6517854c1bdf1e359ce64fb387efcb5abd2e33
Branch commit hash: 7d3a6efd301473209359d7ad18e069ae42473d36

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 683.37 687.33 -3.96 (-0.58%)
aggregation q28 11301.98 11739.38 -437.40 (-3.73%)
filter q14 163.84 159.16 4.68 (2.94%)
filter q15 161.76 163.36 -1.60 (-0.98%)
filter q16 342.94 332.47 10.47 (3.15%)
filter q17 479.70 483.27 -3.57 (-0.74%)
filter q18 1940.50 1948.36 -7.86 (-0.40%)
fixed_size_expr_evaluator q07 570.87 575.65 -4.78 (-0.83%)
fixed_size_expr_evaluator q08 783.70 779.31 4.39 (0.56%)
fixed_size_expr_evaluator q09 783.63 786.85 -3.22 (-0.41%)
fixed_size_expr_evaluator q10 272.65 273.91 -1.26 (-0.46%)
fixed_size_expr_evaluator q11 267.88 268.14 -0.27 (-0.10%)
fixed_size_expr_evaluator q12 265.72 266.88 -1.16 (-0.43%)
fixed_size_expr_evaluator q13 1501.93 1492.52 9.41 (0.63%)
fixed_size_seq_scan q23 153.10 149.81 3.29 (2.20%)
join q31 12.29 12.34 -0.05 (-0.39%)
ldbc_snb_ic q35 768.75 782.55 -13.80 (-1.76%)
ldbc_snb_ic q36 50.13 52.31 -2.18 (-4.17%)
ldbc_snb_is q32 9.39 8.76 0.64 (7.28%)
ldbc_snb_is q33 18.33 17.09 1.24 (7.27%)
ldbc_snb_is q34 8.57 9.03 -0.46 (-5.06%)
multi-rel multi-rel-large-scan 2747.23 2952.89 -205.66 (-6.96%)
multi-rel multi-rel-lookup 67.07 64.13 2.94 (4.59%)
multi-rel multi-rel-small-scan 54.80 70.79 -15.98 (-22.58%)
order_by q25 164.57 164.87 -0.30 (-0.18%)
order_by q26 483.86 486.68 -2.83 (-0.58%)
order_by q27 1422.05 1428.61 -6.56 (-0.46%)
scan_after_filter q01 204.33 206.12 -1.79 (-0.87%)
scan_after_filter q02 193.54 194.68 -1.14 (-0.58%)
shortest_path_ldbc100 q39 154.65 159.87 -5.22 (-3.27%)
var_size_expr_evaluator q03 2081.71 2081.89 -0.18 (-0.01%)
var_size_expr_evaluator q04 2276.43 2281.95 -5.52 (-0.24%)
var_size_expr_evaluator q05 2638.82 2696.76 -57.93 (-2.15%)
var_size_expr_evaluator q06 1357.49 1347.50 9.99 (0.74%)
var_size_seq_scan q19 1486.99 1493.05 -6.06 (-0.41%)
var_size_seq_scan q20 3213.79 3162.30 51.49 (1.63%)
var_size_seq_scan q21 2463.39 2426.45 36.95 (1.52%)
var_size_seq_scan q22 135.80 134.45 1.35 (1.01%)

src/binder/expression/node_rel_expression.cpp Outdated Show resolved Hide resolved
Copy link

Benchmark Result

Master commit hash: 6a6517854c1bdf1e359ce64fb387efcb5abd2e33
Branch commit hash: 0265825d202551d621aef46671e1d102181e0471

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 669.03 687.33 -18.30 (-2.66%)
aggregation q28 11276.05 11739.38 -463.33 (-3.95%)
filter q14 143.75 159.16 -15.40 (-9.68%)
filter q15 145.66 163.36 -17.70 (-10.84%)
filter q16 325.29 332.47 -7.18 (-2.16%)
filter q17 465.67 483.27 -17.59 (-3.64%)
filter q18 1921.65 1948.36 -26.71 (-1.37%)
fixed_size_expr_evaluator q07 556.44 575.65 -19.21 (-3.34%)
fixed_size_expr_evaluator q08 771.61 779.31 -7.70 (-0.99%)
fixed_size_expr_evaluator q09 770.53 786.85 -16.32 (-2.07%)
fixed_size_expr_evaluator q10 258.60 273.91 -15.31 (-5.59%)
fixed_size_expr_evaluator q11 254.49 268.14 -13.66 (-5.09%)
fixed_size_expr_evaluator q12 250.52 266.88 -16.37 (-6.13%)
fixed_size_expr_evaluator q13 1495.51 1492.52 2.99 (0.20%)
fixed_size_seq_scan q23 137.54 149.81 -12.27 (-8.19%)
join q31 12.21 12.34 -0.13 (-1.06%)
ldbc_snb_ic q35 818.46 782.55 35.91 (4.59%)
ldbc_snb_ic q36 45.05 52.31 -7.26 (-13.87%)
ldbc_snb_is q32 9.91 8.76 1.15 (13.18%)
ldbc_snb_is q33 16.74 17.09 -0.35 (-2.02%)
ldbc_snb_is q34 7.89 9.03 -1.13 (-12.56%)
multi-rel multi-rel-large-scan 3871.05 2952.89 918.17 (31.09%)
multi-rel multi-rel-lookup 46.98 64.13 -17.14 (-26.73%)
multi-rel multi-rel-small-scan 57.62 70.79 -13.16 (-18.59%)
order_by q25 150.01 164.87 -14.87 (-9.02%)
order_by q26 481.30 486.68 -5.38 (-1.11%)
order_by q27 1417.46 1428.61 -11.15 (-0.78%)
scan_after_filter q01 190.89 206.12 -15.23 (-7.39%)
scan_after_filter q02 179.76 194.68 -14.92 (-7.67%)
shortest_path_ldbc100 q39 155.72 159.87 -4.14 (-2.59%)
var_size_expr_evaluator q03 2079.16 2081.89 -2.73 (-0.13%)
var_size_expr_evaluator q04 2224.69 2281.95 -57.26 (-2.51%)
var_size_expr_evaluator q05 2550.64 2696.76 -146.12 (-5.42%)
var_size_expr_evaluator q06 1346.74 1347.50 -0.76 (-0.06%)
var_size_seq_scan q19 1474.34 1493.05 -18.72 (-1.25%)
var_size_seq_scan q20 3207.93 3162.30 45.63 (1.44%)
var_size_seq_scan q21 2438.01 2426.45 11.56 (0.48%)
var_size_seq_scan q22 129.84 134.45 -4.61 (-3.43%)

@andyfengHKU andyfengHKU merged commit 93a3cd3 into master Aug 12, 2024
@andyfengHKU andyfengHKU deleted the optimize-catalog-access branch August 12, 2024 19:00
ray6080 pushed a commit that referenced this pull request Aug 14, 2024
* Optimzie catalog access when binding query pattern

* Run clang-format

---------

Co-authored-by: CI Bot <andyfengHKU@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants