Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(common): reduce the average recusrion depth in _flatten_collections #8709

Merged
merged 1 commit into from
Mar 21, 2024

Conversation

kszucs
Copy link
Member

@kszucs kszucs commented Mar 20, 2024

_flatten_collections() is called with a sequence node.__args__ and instead of checking for a sequence then recursing into another call, now it directly iterates over the input sparing a constant function call for each node in each traversal.

---------------------------------------------------------------------------- benchmark 'test_bfs': 2 tests -----------------------------------------------------------------------------
Name (time in ms)              Min               Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_bfs (0673_5041894)     2.2382 (1.20)     2.9203 (1.24)     2.3931 (1.23)     0.1005 (1.34)     2.3806 (1.24)     0.1482 (2.74)         48;3  417.8750 (0.81)        178           1
test_bfs (NOW)              1.8689 (1.0)      2.3535 (1.0)      1.9434 (1.0)      0.0752 (1.0)      1.9226 (1.0)      0.0541 (1.0)         23;20  514.5500 (1.0)         208           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_dfs': 2 tests -----------------------------------------------------------------------------
Name (time in ms)              Min               Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dfs (0673_5041894)     2.2766 (1.19)     2.8229 (1.25)     2.4995 (1.23)     0.1268 (1.68)     2.4992 (1.22)     0.1543 (1.37)         12;1  400.0872 (0.82)         39           1
test_dfs (NOW)              1.9076 (1.0)      2.2635 (1.0)      2.0371 (1.0)      0.0754 (1.0)      2.0467 (1.0)      0.1129 (1.0)          56;2  490.9033 (1.0)         189           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------- benchmark 'test_replace_mapping': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                          Min                Max               Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_replace_mapping (0673_5041894)     9.0726 (1.06)     30.7215 (1.13)     10.2926 (1.09)     3.2098 (1.0)      9.7669 (1.12)     0.5696 (3.24)          2;3   97.1575 (0.92)         77           1
test_replace_mapping (NOW)              8.5212 (1.0)      27.3008 (1.0)       9.4436 (1.0)      3.2393 (1.01)     8.7318 (1.0)      0.1755 (1.0)          3;13  105.8916 (1.0)          83           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------ benchmark 'test_replace_pattern': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                           Min                Max               Mean            StdDev             Median               IQR            Outliers      OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_replace_pattern (0673_5041894)     12.6172 (1.03)     32.6150 (1.02)     14.1622 (1.05)     3.2321 (1.0)      13.5553 (1.05)     0.7407 (1.0)           3;3  70.6105 (0.95)         61           1
test_replace_pattern (NOW)              12.2255 (1.0)      32.0460 (1.0)      13.4309 (1.0)      3.4930 (1.08)     12.9361 (1.0)      0.7598 (1.03)          2;2  74.4554 (1.0)          59           1

@kszucs kszucs force-pushed the flatten-collections branch from 5b3d269 to 61457a7 Compare March 20, 2024 14:01
@kszucs kszucs force-pushed the flatten-collections branch from 61457a7 to 68e1d1a Compare March 20, 2024 15:54
@cpcloud cpcloud added this to the 9.0 milestone Mar 21, 2024
@cpcloud cpcloud added the performance Issues related to ibis's performance label Mar 21, 2024
@cpcloud cpcloud merged commit 3d52904 into ibis-project:main Mar 21, 2024
83 checks passed
@cpcloud cpcloud deleted the flatten-collections branch March 21, 2024 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Issues related to ibis's performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants