Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tests: update for dvc 3.x #43

Merged
merged 3 commits into from
Jun 12, 2023
Merged

tests: update for dvc 3.x #43

merged 3 commits into from
Jun 12, 2023

Conversation

pmrowla
Copy link
Contributor

@pmrowla pmrowla commented Jun 9, 2023

No description provided.

@pmrowla pmrowla self-assigned this Jun 9, 2023
@pmrowla pmrowla mentioned this pull request Jun 9, 2023
Merged
2 tasks
@github-actions
Copy link
Contributor

github-actions bot commented Jun 9, 2023 


------------------------------------------------------------------------ benchmark 'test_sharing_s3-add': 7 tests ------------------------------------------------------------------------
Name (time in s)                   Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-add[2.11.0]     9.5704 (1.59)     9.5704 (1.59)     9.5704 (1.59)     0.0000 (1.0)      9.5704 (1.59)     0.0000 (1.0)           0;0  0.1045 (0.63)          1           1
test_sharing_s3-add[2.18.1]     8.4002 (1.40)     8.4002 (1.40)     8.4002 (1.40)     0.0000 (1.0)      8.4002 (1.40)     0.0000 (1.0)           0;0  0.1190 (0.72)          1           1
test_sharing_s3-add[2.39.0]     8.1471 (1.35)     8.1471 (1.35)     8.1471 (1.35)     0.0000 (1.0)      8.1471 (1.35)     0.0000 (1.0)           0;0  0.1227 (0.74)          1           1
test_sharing_s3-add[2.40.0]     8.4279 (1.40)     8.4279 (1.40)     8.4279 (1.40)     0.0000 (1.0)      8.4279 (1.40)     0.0000 (1.0)           0;0  0.1187 (0.71)          1           1
test_sharing_s3-add[2.41.1]     7.6982 (1.28)     7.6982 (1.28)     7.6982 (1.28)     0.0000 (1.0)      7.6982 (1.28)     0.0000 (1.0)           0;0  0.1299 (0.78)          1           1
test_sharing_s3-add[2.45.0]     6.0139 (1.0)      6.0139 (1.0)      6.0139 (1.0)      0.0000 (1.0)      6.0139 (1.0)      0.0000 (1.0)           0;0  0.1663 (1.0)           1           1
test_sharing_s3-add[main]       6.3944 (1.06)     6.3944 (1.06)     6.3944 (1.06)     0.0000 (1.0)      6.3944 (1.06)     0.0000 (1.0)           0;0  0.1564 (0.94)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing_s3-add-noop': 7 tests ------------------------------------------------------------------------
Name (time in s)                        Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-add-noop[2.11.0]     3.6827 (1.00)     3.6827 (1.00)     3.6827 (1.00)     0.0000 (1.0)      3.6827 (1.00)     0.0000 (1.0)           0;0  0.2715 (1.00)          1           1
test_sharing_s3-add-noop[2.18.1]     4.0436 (1.10)     4.0436 (1.10)     4.0436 (1.10)     0.0000 (1.0)      4.0436 (1.10)     0.0000 (1.0)           0;0  0.2473 (0.91)          1           1
test_sharing_s3-add-noop[2.39.0]     3.7550 (1.02)     3.7550 (1.02)     3.7550 (1.02)     0.0000 (1.0)      3.7550 (1.02)     0.0000 (1.0)           0;0  0.2663 (0.98)          1           1
test_sharing_s3-add-noop[2.40.0]     3.6816 (1.0)      3.6816 (1.0)      3.6816 (1.0)      0.0000 (1.0)      3.6816 (1.0)      0.0000 (1.0)           0;0  0.2716 (1.0)           1           1
test_sharing_s3-add-noop[2.41.1]     3.8057 (1.03)     3.8057 (1.03)     3.8057 (1.03)     0.0000 (1.0)      3.8057 (1.03)     0.0000 (1.0)           0;0  0.2628 (0.97)          1           1
test_sharing_s3-add-noop[2.45.0]     4.1343 (1.12)     4.1343 (1.12)     4.1343 (1.12)     0.0000 (1.0)      4.1343 (1.12)     0.0000 (1.0)           0;0  0.2419 (0.89)          1           1
test_sharing_s3-add-noop[main]       4.2137 (1.14)     4.2137 (1.14)     4.2137 (1.14)     0.0000 (1.0)      4.2137 (1.14)     0.0000 (1.0)           0;0  0.2373 (0.87)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing_s3-checkout-noop': 7 tests --------------------------------------------------------------------------------
Name (time in ms)                                Min                   Max                  Mean            StdDev                Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-checkout-noop[2.11.0]       708.0486 (1.0)        708.0486 (1.0)        708.0486 (1.0)      0.0000 (1.0)        708.0486 (1.0)      0.0000 (1.0)           0;0  1.4123 (1.0)           1           1
test_sharing_s3-checkout-noop[2.18.1]       805.7072 (1.14)       805.7072 (1.14)       805.7072 (1.14)     0.0000 (1.0)        805.7072 (1.14)     0.0000 (1.0)           0;0  1.2411 (0.88)          1           1
test_sharing_s3-checkout-noop[2.39.0]       927.1358 (1.31)       927.1358 (1.31)       927.1358 (1.31)     0.0000 (1.0)        927.1358 (1.31)     0.0000 (1.0)           0;0  1.0786 (0.76)          1           1
test_sharing_s3-checkout-noop[2.40.0]       923.8379 (1.30)       923.8379 (1.30)       923.8379 (1.30)     0.0000 (1.0)        923.8379 (1.30)     0.0000 (1.0)           0;0  1.0824 (0.77)          1           1
test_sharing_s3-checkout-noop[2.41.1]       879.5780 (1.24)       879.5780 (1.24)       879.5780 (1.24)     0.0000 (1.0)        879.5780 (1.24)     0.0000 (1.0)           0;0  1.1369 (0.80)          1           1
test_sharing_s3-checkout-noop[2.45.0]       934.3147 (1.32)       934.3147 (1.32)       934.3147 (1.32)     0.0000 (1.0)        934.3147 (1.32)     0.0000 (1.0)           0;0  1.0703 (0.76)          1           1
test_sharing_s3-checkout-noop[main]       1,402.9912 (1.98)     1,402.9912 (1.98)     1,402.9912 (1.98)     0.0000 (1.0)      1,402.9912 (1.98)     0.0000 (1.0)           0;0  0.7128 (0.50)          1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------- benchmark 'test_sharing_s3-pull': 7 tests --------------------------------------------------------------------------
Name (time in s)                     Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-pull[2.11.0]     17.4778 (1.15)     17.4778 (1.15)     17.4778 (1.15)     0.0000 (1.0)      17.4778 (1.15)     0.0000 (1.0)           0;0  0.0572 (0.87)          1           1
test_sharing_s3-pull[2.18.1]     18.5485 (1.22)     18.5485 (1.22)     18.5485 (1.22)     0.0000 (1.0)      18.5485 (1.22)     0.0000 (1.0)           0;0  0.0539 (0.82)          1           1
test_sharing_s3-pull[2.39.0]     18.0167 (1.19)     18.0167 (1.19)     18.0167 (1.19)     0.0000 (1.0)      18.0167 (1.19)     0.0000 (1.0)           0;0  0.0555 (0.84)          1           1
test_sharing_s3-pull[2.40.0]     17.2850 (1.14)     17.2850 (1.14)     17.2850 (1.14)     0.0000 (1.0)      17.2850 (1.14)     0.0000 (1.0)           0;0  0.0579 (0.88)          1           1
test_sharing_s3-pull[2.41.1]     17.2136 (1.13)     17.2136 (1.13)     17.2136 (1.13)     0.0000 (1.0)      17.2136 (1.13)     0.0000 (1.0)           0;0  0.0581 (0.88)          1           1
test_sharing_s3-pull[2.45.0]     15.1848 (1.0)      15.1848 (1.0)      15.1848 (1.0)      0.0000 (1.0)      15.1848 (1.0)      0.0000 (1.0)           0;0  0.0659 (1.0)           1           1
test_sharing_s3-pull[main]       16.0774 (1.06)     16.0774 (1.06)     16.0774 (1.06)     0.0000 (1.0)      16.0774 (1.06)     0.0000 (1.0)           0;0  0.0622 (0.94)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing_s3-pull-noop': 7 tests ------------------------------------------------------------------------
Name (time in s)                         Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-pull-noop[2.11.0]     1.0403 (1.0)      1.0403 (1.0)      1.0403 (1.0)      0.0000 (1.0)      1.0403 (1.0)      0.0000 (1.0)           0;0  0.9613 (1.0)           1           1
test_sharing_s3-pull-noop[2.18.1]     1.2299 (1.18)     1.2299 (1.18)     1.2299 (1.18)     0.0000 (1.0)      1.2299 (1.18)     0.0000 (1.0)           0;0  0.8131 (0.85)          1           1
test_sharing_s3-pull-noop[2.39.0]     1.2957 (1.25)     1.2957 (1.25)     1.2957 (1.25)     0.0000 (1.0)      1.2957 (1.25)     0.0000 (1.0)           0;0  0.7718 (0.80)          1           1
test_sharing_s3-pull-noop[2.40.0]     1.1898 (1.14)     1.1898 (1.14)     1.1898 (1.14)     0.0000 (1.0)      1.1898 (1.14)     0.0000 (1.0)           0;0  0.8404 (0.87)          1           1
test_sharing_s3-pull-noop[2.41.1]     1.2550 (1.21)     1.2550 (1.21)     1.2550 (1.21)     0.0000 (1.0)      1.2550 (1.21)     0.0000 (1.0)           0;0  0.7968 (0.83)          1           1
test_sharing_s3-pull-noop[2.45.0]     1.2541 (1.21)     1.2541 (1.21)     1.2541 (1.21)     0.0000 (1.0)      1.2541 (1.21)     0.0000 (1.0)           0;0  0.7974 (0.83)          1           1
test_sharing_s3-pull-noop[main]       1.4037 (1.35)     1.4037 (1.35)     1.4037 (1.35)     0.0000 (1.0)      1.4037 (1.35)     0.0000 (1.0)           0;0  0.7124 (0.74)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------- benchmark 'test_sharing_s3-push': 7 tests --------------------------------------------------------------------------
Name (time in s)                     Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-push[2.11.0]     30.9018 (2.58)     30.9018 (2.58)     30.9018 (2.58)     0.0000 (1.0)      30.9018 (2.58)     0.0000 (1.0)           0;0  0.0324 (0.39)          1           1
test_sharing_s3-push[2.18.1]     30.2618 (2.53)     30.2618 (2.53)     30.2618 (2.53)     0.0000 (1.0)      30.2618 (2.53)     0.0000 (1.0)           0;0  0.0330 (0.40)          1           1
test_sharing_s3-push[2.39.0]     29.9543 (2.50)     29.9543 (2.50)     29.9543 (2.50)     0.0000 (1.0)      29.9543 (2.50)     0.0000 (1.0)           0;0  0.0334 (0.40)          1           1
test_sharing_s3-push[2.40.0]     29.7085 (2.48)     29.7085 (2.48)     29.7085 (2.48)     0.0000 (1.0)      29.7085 (2.48)     0.0000 (1.0)           0;0  0.0337 (0.40)          1           1
test_sharing_s3-push[2.41.1]     14.2673 (1.19)     14.2673 (1.19)     14.2673 (1.19)     0.0000 (1.0)      14.2673 (1.19)     0.0000 (1.0)           0;0  0.0701 (0.84)          1           1
test_sharing_s3-push[2.45.0]     12.0784 (1.01)     12.0784 (1.01)     12.0784 (1.01)     0.0000 (1.0)      12.0784 (1.01)     0.0000 (1.0)           0;0  0.0828 (0.99)          1           1
test_sharing_s3-push[main]       11.9765 (1.0)      11.9765 (1.0)      11.9765 (1.0)      0.0000 (1.0)      11.9765 (1.0)      0.0000 (1.0)           0;0  0.0835 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing_s3-push-noop': 7 tests ------------------------------------------------------------------------
Name (time in s)                         Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-push-noop[2.11.0]     1.3913 (1.05)     1.3913 (1.05)     1.3913 (1.05)     0.0000 (1.0)      1.3913 (1.05)     0.0000 (1.0)           0;0  0.7188 (0.95)          1           1
test_sharing_s3-push-noop[2.18.1]     1.3265 (1.0)      1.3265 (1.0)      1.3265 (1.0)      0.0000 (1.0)      1.3265 (1.0)      0.0000 (1.0)           0;0  0.7539 (1.0)           1           1
test_sharing_s3-push-noop[2.39.0]     1.4523 (1.09)     1.4523 (1.09)     1.4523 (1.09)     0.0000 (1.0)      1.4523 (1.09)     0.0000 (1.0)           0;0  0.6886 (0.91)          1           1
test_sharing_s3-push-noop[2.40.0]     1.3663 (1.03)     1.3663 (1.03)     1.3663 (1.03)     0.0000 (1.0)      1.3663 (1.03)     0.0000 (1.0)           0;0  0.7319 (0.97)          1           1
test_sharing_s3-push-noop[2.41.1]     1.5280 (1.15)     1.5280 (1.15)     1.5280 (1.15)     0.0000 (1.0)      1.5280 (1.15)     0.0000 (1.0)           0;0  0.6544 (0.87)          1           1
test_sharing_s3-push-noop[2.45.0]     1.3620 (1.03)     1.3620 (1.03)     1.3620 (1.03)     0.0000 (1.0)      1.3620 (1.03)     0.0000 (1.0)           0;0  0.7342 (0.97)          1           1
test_sharing_s3-push-noop[main]       1.6237 (1.22)     1.6237 (1.22)     1.6237 (1.22)     0.0000 (1.0)      1.6237 (1.22)     0.0000 (1.0)           0;0  0.6159 (0.82)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

@pmrowla pmrowla marked this pull request as ready for review June 12, 2023 07:09
@github-actions
Copy link
Contributor 


------------------------------------------------------------------------ benchmark 'test_sharing_s3-add': 7 tests ------------------------------------------------------------------------
Name (time in s)                   Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-add[2.11.0]     5.2931 (1.54)     5.2931 (1.54)     5.2931 (1.54)     0.0000 (1.0)      5.2931 (1.54)     0.0000 (1.0)           0;0  0.1889 (0.65)          1           1
test_sharing_s3-add[2.18.1]     4.4566 (1.30)     4.4566 (1.30)     4.4566 (1.30)     0.0000 (1.0)      4.4566 (1.30)     0.0000 (1.0)           0;0  0.2244 (0.77)          1           1
test_sharing_s3-add[2.39.0]     4.5006 (1.31)     4.5006 (1.31)     4.5006 (1.31)     0.0000 (1.0)      4.5006 (1.31)     0.0000 (1.0)           0;0  0.2222 (0.76)          1           1
test_sharing_s3-add[2.40.0]     4.5207 (1.32)     4.5207 (1.32)     4.5207 (1.32)     0.0000 (1.0)      4.5207 (1.32)     0.0000 (1.0)           0;0  0.2212 (0.76)          1           1
test_sharing_s3-add[2.41.1]     4.4169 (1.28)     4.4169 (1.28)     4.4169 (1.28)     0.0000 (1.0)      4.4169 (1.28)     0.0000 (1.0)           0;0  0.2264 (0.78)          1           1
test_sharing_s3-add[2.45.0]     3.4377 (1.0)      3.4377 (1.0)      3.4377 (1.0)      0.0000 (1.0)      3.4377 (1.0)      0.0000 (1.0)           0;0  0.2909 (1.0)           1           1
test_sharing_s3-add[main]       3.6117 (1.05)     3.6117 (1.05)     3.6117 (1.05)     0.0000 (1.0)      3.6117 (1.05)     0.0000 (1.0)           0;0  0.2769 (0.95)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing_s3-add-noop': 7 tests ------------------------------------------------------------------------
Name (time in s)                        Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-add-noop[2.11.0]     2.1438 (1.0)      2.1438 (1.0)      2.1438 (1.0)      0.0000 (1.0)      2.1438 (1.0)      0.0000 (1.0)           0;0  0.4665 (1.0)           1           1
test_sharing_s3-add-noop[2.18.1]     2.3131 (1.08)     2.3131 (1.08)     2.3131 (1.08)     0.0000 (1.0)      2.3131 (1.08)     0.0000 (1.0)           0;0  0.4323 (0.93)          1           1
test_sharing_s3-add-noop[2.39.0]     2.1912 (1.02)     2.1912 (1.02)     2.1912 (1.02)     0.0000 (1.0)      2.1912 (1.02)     0.0000 (1.0)           0;0  0.4564 (0.98)          1           1
test_sharing_s3-add-noop[2.40.0]     2.2054 (1.03)     2.2054 (1.03)     2.2054 (1.03)     0.0000 (1.0)      2.2054 (1.03)     0.0000 (1.0)           0;0  0.4534 (0.97)          1           1
test_sharing_s3-add-noop[2.41.1]     2.1798 (1.02)     2.1798 (1.02)     2.1798 (1.02)     0.0000 (1.0)      2.1798 (1.02)     0.0000 (1.0)           0;0  0.4588 (0.98)          1           1
test_sharing_s3-add-noop[2.45.0]     2.2678 (1.06)     2.2678 (1.06)     2.2678 (1.06)     0.0000 (1.0)      2.2678 (1.06)     0.0000 (1.0)           0;0  0.4410 (0.95)          1           1
test_sharing_s3-add-noop[main]       2.4313 (1.13)     2.4313 (1.13)     2.4313 (1.13)     0.0000 (1.0)      2.4313 (1.13)     0.0000 (1.0)           0;0  0.4113 (0.88)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_sharing_s3-checkout-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms)                              Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-checkout-noop[2.11.0]     472.5918 (1.0)      472.5918 (1.0)      472.5918 (1.0)      0.0000 (1.0)      472.5918 (1.0)      0.0000 (1.0)           0;0  2.1160 (1.0)           1           1
test_sharing_s3-checkout-noop[2.18.1]     534.5043 (1.13)     534.5043 (1.13)     534.5043 (1.13)     0.0000 (1.0)      534.5043 (1.13)     0.0000 (1.0)           0;0  1.8709 (0.88)          1           1
test_sharing_s3-checkout-noop[2.39.0]     554.2702 (1.17)     554.2702 (1.17)     554.2702 (1.17)     0.0000 (1.0)      554.2702 (1.17)     0.0000 (1.0)           0;0  1.8042 (0.85)          1           1
test_sharing_s3-checkout-noop[2.40.0]     563.2286 (1.19)     563.2286 (1.19)     563.2286 (1.19)     0.0000 (1.0)      563.2286 (1.19)     0.0000 (1.0)           0;0  1.7755 (0.84)          1           1
test_sharing_s3-checkout-noop[2.41.1]     550.2296 (1.16)     550.2296 (1.16)     550.2296 (1.16)     0.0000 (1.0)      550.2296 (1.16)     0.0000 (1.0)           0;0  1.8174 (0.86)          1           1
test_sharing_s3-checkout-noop[2.45.0]     576.8479 (1.22)     576.8479 (1.22)     576.8479 (1.22)     0.0000 (1.0)      576.8479 (1.22)     0.0000 (1.0)           0;0  1.7336 (0.82)          1           1
test_sharing_s3-checkout-noop[main]       860.7668 (1.82)     860.7668 (1.82)     860.7668 (1.82)     0.0000 (1.0)      860.7668 (1.82)     0.0000 (1.0)           0;0  1.1618 (0.55)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------- benchmark 'test_sharing_s3-pull': 7 tests --------------------------------------------------------------------------
Name (time in s)                     Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-pull[2.11.0]     10.7118 (1.13)     10.7118 (1.13)     10.7118 (1.13)     0.0000 (1.0)      10.7118 (1.13)     0.0000 (1.0)           0;0  0.0934 (0.89)          1           1
test_sharing_s3-pull[2.18.1]     12.2371 (1.29)     12.2371 (1.29)     12.2371 (1.29)     0.0000 (1.0)      12.2371 (1.29)     0.0000 (1.0)           0;0  0.0817 (0.78)          1           1
test_sharing_s3-pull[2.39.0]     11.6207 (1.22)     11.6207 (1.22)     11.6207 (1.22)     0.0000 (1.0)      11.6207 (1.22)     0.0000 (1.0)           0;0  0.0861 (0.82)          1           1
test_sharing_s3-pull[2.40.0]     11.5869 (1.22)     11.5869 (1.22)     11.5869 (1.22)     0.0000 (1.0)      11.5869 (1.22)     0.0000 (1.0)           0;0  0.0863 (0.82)          1           1
test_sharing_s3-pull[2.41.1]     11.4166 (1.20)     11.4166 (1.20)     11.4166 (1.20)     0.0000 (1.0)      11.4166 (1.20)     0.0000 (1.0)           0;0  0.0876 (0.83)          1           1
test_sharing_s3-pull[2.45.0]      9.9353 (1.05)      9.9353 (1.05)      9.9353 (1.05)     0.0000 (1.0)       9.9353 (1.05)     0.0000 (1.0)           0;0  0.1007 (0.96)          1           1
test_sharing_s3-pull[main]        9.4908 (1.0)       9.4908 (1.0)       9.4908 (1.0)      0.0000 (1.0)       9.4908 (1.0)      0.0000 (1.0)           0;0  0.1054 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing_s3-pull-noop': 7 tests --------------------------------------------------------------------------------
Name (time in ms)                            Min                   Max                  Mean            StdDev                Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-pull-noop[2.11.0]       690.9807 (1.0)        690.9807 (1.0)        690.9807 (1.0)      0.0000 (1.0)        690.9807 (1.0)      0.0000 (1.0)           0;0  1.4472 (1.0)           1           1
test_sharing_s3-pull-noop[2.18.1]       766.5321 (1.11)       766.5321 (1.11)       766.5321 (1.11)     0.0000 (1.0)        766.5321 (1.11)     0.0000 (1.0)           0;0  1.3046 (0.90)          1           1
test_sharing_s3-pull-noop[2.39.0]       772.4067 (1.12)       772.4067 (1.12)       772.4067 (1.12)     0.0000 (1.0)        772.4067 (1.12)     0.0000 (1.0)           0;0  1.2947 (0.89)          1           1
test_sharing_s3-pull-noop[2.40.0]       789.1465 (1.14)       789.1465 (1.14)       789.1465 (1.14)     0.0000 (1.0)        789.1465 (1.14)     0.0000 (1.0)           0;0  1.2672 (0.88)          1           1
test_sharing_s3-pull-noop[2.41.1]       758.6685 (1.10)       758.6685 (1.10)       758.6685 (1.10)     0.0000 (1.0)        758.6685 (1.10)     0.0000 (1.0)           0;0  1.3181 (0.91)          1           1
test_sharing_s3-pull-noop[2.45.0]       790.4326 (1.14)       790.4326 (1.14)       790.4326 (1.14)     0.0000 (1.0)        790.4326 (1.14)     0.0000 (1.0)           0;0  1.2651 (0.87)          1           1
test_sharing_s3-pull-noop[main]       1,002.1674 (1.45)     1,002.1674 (1.45)     1,002.1674 (1.45)     0.0000 (1.0)      1,002.1674 (1.45)     0.0000 (1.0)           0;0  0.9978 (0.69)          1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------- benchmark 'test_sharing_s3-push': 7 tests --------------------------------------------------------------------------
Name (time in s)                     Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-push[2.11.0]     20.1338 (2.52)     20.1338 (2.52)     20.1338 (2.52)     0.0000 (1.0)      20.1338 (2.52)     0.0000 (1.0)           0;0  0.0497 (0.40)          1           1
test_sharing_s3-push[2.18.1]     19.8476 (2.48)     19.8476 (2.48)     19.8476 (2.48)     0.0000 (1.0)      19.8476 (2.48)     0.0000 (1.0)           0;0  0.0504 (0.40)          1           1
test_sharing_s3-push[2.39.0]     19.8768 (2.49)     19.8768 (2.49)     19.8768 (2.49)     0.0000 (1.0)      19.8768 (2.49)     0.0000 (1.0)           0;0  0.0503 (0.40)          1           1
test_sharing_s3-push[2.40.0]     19.8641 (2.49)     19.8641 (2.49)     19.8641 (2.49)     0.0000 (1.0)      19.8641 (2.49)     0.0000 (1.0)           0;0  0.0503 (0.40)          1           1
test_sharing_s3-push[2.41.1]      8.9081 (1.12)      8.9081 (1.12)      8.9081 (1.12)     0.0000 (1.0)       8.9081 (1.12)     0.0000 (1.0)           0;0  0.1123 (0.90)          1           1
test_sharing_s3-push[2.45.0]      7.9874 (1.0)       7.9874 (1.0)       7.9874 (1.0)      0.0000 (1.0)       7.9874 (1.0)      0.0000 (1.0)           0;0  0.1252 (1.0)           1           1
test_sharing_s3-push[main]        8.0016 (1.00)      8.0016 (1.00)      8.0016 (1.00)     0.0000 (1.0)       8.0016 (1.00)     0.0000 (1.0)           0;0  0.1250 (1.00)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing_s3-push-noop': 7 tests --------------------------------------------------------------------------------
Name (time in ms)                            Min                   Max                  Mean            StdDev                Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-push-noop[2.11.0]       943.9257 (1.0)        943.9257 (1.0)        943.9257 (1.0)      0.0000 (1.0)        943.9257 (1.0)      0.0000 (1.0)           0;0  1.0594 (1.0)           1           1
test_sharing_s3-push-noop[2.18.1]       947.9801 (1.00)       947.9801 (1.00)       947.9801 (1.00)     0.0000 (1.0)        947.9801 (1.00)     0.0000 (1.0)           0;0  1.0549 (1.00)          1           1
test_sharing_s3-push-noop[2.39.0]       983.4958 (1.04)       983.4958 (1.04)       983.4958 (1.04)     0.0000 (1.0)        983.4958 (1.04)     0.0000 (1.0)           0;0  1.0168 (0.96)          1           1
test_sharing_s3-push-noop[2.40.0]       964.9188 (1.02)       964.9188 (1.02)       964.9188 (1.02)     0.0000 (1.0)        964.9188 (1.02)     0.0000 (1.0)           0;0  1.0364 (0.98)          1           1
test_sharing_s3-push-noop[2.41.1]       956.6632 (1.01)       956.6632 (1.01)       956.6632 (1.01)     0.0000 (1.0)        956.6632 (1.01)     0.0000 (1.0)           0;0  1.0453 (0.99)          1           1
test_sharing_s3-push-noop[2.45.0]       961.1014 (1.02)       961.1014 (1.02)       961.1014 (1.02)     0.0000 (1.0)        961.1014 (1.02)     0.0000 (1.0)           0;0  1.0405 (0.98)          1           1
test_sharing_s3-push-noop[main]       1,056.1891 (1.12)     1,056.1891 (1.12)     1,056.1891 (1.12)     0.0000 (1.0)      1,056.1891 (1.12)     0.0000 (1.0)           0;0  0.9468 (0.89)          1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

@github-actions
Copy link
Contributor 


------------------------------------------------------------------------ benchmark 'test_sharing_s3-add': 7 tests ------------------------------------------------------------------------
Name (time in s)                   Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-add[2.11.0]     6.6494 (1.67)     6.6494 (1.67)     6.6494 (1.67)     0.0000 (1.0)      6.6494 (1.67)     0.0000 (1.0)           0;0  0.1504 (0.60)          1           1
test_sharing_s3-add[2.18.1]     5.6291 (1.42)     5.6291 (1.42)     5.6291 (1.42)     0.0000 (1.0)      5.6291 (1.42)     0.0000 (1.0)           0;0  0.1776 (0.71)          1           1
test_sharing_s3-add[2.39.0]     5.6097 (1.41)     5.6097 (1.41)     5.6097 (1.41)     0.0000 (1.0)      5.6097 (1.41)     0.0000 (1.0)           0;0  0.1783 (0.71)          1           1
test_sharing_s3-add[2.40.0]     5.6577 (1.43)     5.6577 (1.43)     5.6577 (1.43)     0.0000 (1.0)      5.6577 (1.43)     0.0000 (1.0)           0;0  0.1768 (0.70)          1           1
test_sharing_s3-add[2.41.1]     5.1900 (1.31)     5.1900 (1.31)     5.1900 (1.31)     0.0000 (1.0)      5.1900 (1.31)     0.0000 (1.0)           0;0  0.1927 (0.76)          1           1
test_sharing_s3-add[2.45.0]     3.9701 (1.0)      3.9701 (1.0)      3.9701 (1.0)      0.0000 (1.0)      3.9701 (1.0)      0.0000 (1.0)           0;0  0.2519 (1.0)           1           1
test_sharing_s3-add[main]       4.1762 (1.05)     4.1762 (1.05)     4.1762 (1.05)     0.0000 (1.0)      4.1762 (1.05)     0.0000 (1.0)           0;0  0.2395 (0.95)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing_s3-add-noop': 7 tests ------------------------------------------------------------------------
Name (time in s)                        Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-add-noop[2.11.0]     2.4448 (1.0)      2.4448 (1.0)      2.4448 (1.0)      0.0000 (1.0)      2.4448 (1.0)      0.0000 (1.0)           0;0  0.4090 (1.0)           1           1
test_sharing_s3-add-noop[2.18.1]     2.6527 (1.09)     2.6527 (1.09)     2.6527 (1.09)     0.0000 (1.0)      2.6527 (1.09)     0.0000 (1.0)           0;0  0.3770 (0.92)          1           1
test_sharing_s3-add-noop[2.39.0]     2.5037 (1.02)     2.5037 (1.02)     2.5037 (1.02)     0.0000 (1.0)      2.5037 (1.02)     0.0000 (1.0)           0;0  0.3994 (0.98)          1           1
test_sharing_s3-add-noop[2.40.0]     2.4912 (1.02)     2.4912 (1.02)     2.4912 (1.02)     0.0000 (1.0)      2.4912 (1.02)     0.0000 (1.0)           0;0  0.4014 (0.98)          1           1
test_sharing_s3-add-noop[2.41.1]     2.5302 (1.03)     2.5302 (1.03)     2.5302 (1.03)     0.0000 (1.0)      2.5302 (1.03)     0.0000 (1.0)           0;0  0.3952 (0.97)          1           1
test_sharing_s3-add-noop[2.45.0]     2.5638 (1.05)     2.5638 (1.05)     2.5638 (1.05)     0.0000 (1.0)      2.5638 (1.05)     0.0000 (1.0)           0;0  0.3900 (0.95)          1           1
test_sharing_s3-add-noop[main]       2.7013 (1.10)     2.7013 (1.10)     2.7013 (1.10)     0.0000 (1.0)      2.7013 (1.10)     0.0000 (1.0)           0;0  0.3702 (0.91)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_sharing_s3-checkout-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms)                              Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-checkout-noop[2.11.0]     506.2144 (1.0)      506.2144 (1.0)      506.2144 (1.0)      0.0000 (1.0)      506.2144 (1.0)      0.0000 (1.0)           0;0  1.9754 (1.0)           1           1
test_sharing_s3-checkout-noop[2.18.1]     571.5558 (1.13)     571.5558 (1.13)     571.5558 (1.13)     0.0000 (1.0)      571.5558 (1.13)     0.0000 (1.0)           0;0  1.7496 (0.89)          1           1
test_sharing_s3-checkout-noop[2.39.0]     608.2001 (1.20)     608.2001 (1.20)     608.2001 (1.20)     0.0000 (1.0)      608.2001 (1.20)     0.0000 (1.0)           0;0  1.6442 (0.83)          1           1
test_sharing_s3-checkout-noop[2.40.0]     593.0452 (1.17)     593.0452 (1.17)     593.0452 (1.17)     0.0000 (1.0)      593.0452 (1.17)     0.0000 (1.0)           0;0  1.6862 (0.85)          1           1
test_sharing_s3-checkout-noop[2.41.1]     591.0161 (1.17)     591.0161 (1.17)     591.0161 (1.17)     0.0000 (1.0)      591.0161 (1.17)     0.0000 (1.0)           0;0  1.6920 (0.86)          1           1
test_sharing_s3-checkout-noop[2.45.0]     673.7633 (1.33)     673.7633 (1.33)     673.7633 (1.33)     0.0000 (1.0)      673.7633 (1.33)     0.0000 (1.0)           0;0  1.4842 (0.75)          1           1
test_sharing_s3-checkout-noop[main]       939.6756 (1.86)     939.6756 (1.86)     939.6756 (1.86)     0.0000 (1.0)      939.6756 (1.86)     0.0000 (1.0)           0;0  1.0642 (0.54)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------- benchmark 'test_sharing_s3-pull': 7 tests --------------------------------------------------------------------------
Name (time in s)                     Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-pull[2.11.0]     11.7494 (1.15)     11.7494 (1.15)     11.7494 (1.15)     0.0000 (1.0)      11.7494 (1.15)     0.0000 (1.0)           0;0  0.0851 (0.87)          1           1
test_sharing_s3-pull[2.18.1]     13.4252 (1.31)     13.4252 (1.31)     13.4252 (1.31)     0.0000 (1.0)      13.4252 (1.31)     0.0000 (1.0)           0;0  0.0745 (0.76)          1           1
test_sharing_s3-pull[2.39.0]     12.2465 (1.20)     12.2465 (1.20)     12.2465 (1.20)     0.0000 (1.0)      12.2465 (1.20)     0.0000 (1.0)           0;0  0.0817 (0.84)          1           1
test_sharing_s3-pull[2.40.0]     11.9879 (1.17)     11.9879 (1.17)     11.9879 (1.17)     0.0000 (1.0)      11.9879 (1.17)     0.0000 (1.0)           0;0  0.0834 (0.85)          1           1
test_sharing_s3-pull[2.41.1]     12.1163 (1.18)     12.1163 (1.18)     12.1163 (1.18)     0.0000 (1.0)      12.1163 (1.18)     0.0000 (1.0)           0;0  0.0825 (0.84)          1           1
test_sharing_s3-pull[2.45.0]     10.5048 (1.03)     10.5048 (1.03)     10.5048 (1.03)     0.0000 (1.0)      10.5048 (1.03)     0.0000 (1.0)           0;0  0.0952 (0.97)          1           1
test_sharing_s3-pull[main]       10.2317 (1.0)      10.2317 (1.0)      10.2317 (1.0)      0.0000 (1.0)      10.2317 (1.0)      0.0000 (1.0)           0;0  0.0977 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing_s3-pull-noop': 7 tests --------------------------------------------------------------------------------
Name (time in ms)                            Min                   Max                  Mean            StdDev                Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-pull-noop[2.11.0]       751.0379 (1.0)        751.0379 (1.0)        751.0379 (1.0)      0.0000 (1.0)        751.0379 (1.0)      0.0000 (1.0)           0;0  1.3315 (1.0)           1           1
test_sharing_s3-pull-noop[2.18.1]       820.7264 (1.09)       820.7264 (1.09)       820.7264 (1.09)     0.0000 (1.0)        820.7264 (1.09)     0.0000 (1.0)           0;0  1.2184 (0.92)          1           1
test_sharing_s3-pull-noop[2.39.0]       856.3305 (1.14)       856.3305 (1.14)       856.3305 (1.14)     0.0000 (1.0)        856.3305 (1.14)     0.0000 (1.0)           0;0  1.1678 (0.88)          1           1
test_sharing_s3-pull-noop[2.40.0]       815.4187 (1.09)       815.4187 (1.09)       815.4187 (1.09)     0.0000 (1.0)        815.4187 (1.09)     0.0000 (1.0)           0;0  1.2264 (0.92)          1           1
test_sharing_s3-pull-noop[2.41.1]       829.8412 (1.10)       829.8412 (1.10)       829.8412 (1.10)     0.0000 (1.0)        829.8412 (1.10)     0.0000 (1.0)           0;0  1.2050 (0.91)          1           1
test_sharing_s3-pull-noop[2.45.0]       863.8349 (1.15)       863.8349 (1.15)       863.8349 (1.15)     0.0000 (1.0)        863.8349 (1.15)     0.0000 (1.0)           0;0  1.1576 (0.87)          1           1
test_sharing_s3-pull-noop[main]       1,082.4444 (1.44)     1,082.4444 (1.44)     1,082.4444 (1.44)     0.0000 (1.0)      1,082.4444 (1.44)     0.0000 (1.0)           0;0  0.9238 (0.69)          1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------- benchmark 'test_sharing_s3-push': 7 tests --------------------------------------------------------------------------
Name (time in s)                     Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-push[2.11.0]     20.8126 (2.57)     20.8126 (2.57)     20.8126 (2.57)     0.0000 (1.0)      20.8126 (2.57)     0.0000 (1.0)           0;0  0.0480 (0.39)          1           1
test_sharing_s3-push[2.18.1]     20.9122 (2.58)     20.9122 (2.58)     20.9122 (2.58)     0.0000 (1.0)      20.9122 (2.58)     0.0000 (1.0)           0;0  0.0478 (0.39)          1           1
test_sharing_s3-push[2.39.0]     20.3480 (2.51)     20.3480 (2.51)     20.3480 (2.51)     0.0000 (1.0)      20.3480 (2.51)     0.0000 (1.0)           0;0  0.0491 (0.40)          1           1
test_sharing_s3-push[2.40.0]     19.9745 (2.47)     19.9745 (2.47)     19.9745 (2.47)     0.0000 (1.0)      19.9745 (2.47)     0.0000 (1.0)           0;0  0.0501 (0.41)          1           1
test_sharing_s3-push[2.41.1]      9.8544 (1.22)      9.8544 (1.22)      9.8544 (1.22)     0.0000 (1.0)       9.8544 (1.22)     0.0000 (1.0)           0;0  0.1015 (0.82)          1           1
test_sharing_s3-push[2.45.0]      8.3890 (1.04)      8.3890 (1.04)      8.3890 (1.04)     0.0000 (1.0)       8.3890 (1.04)     0.0000 (1.0)           0;0  0.1192 (0.96)          1           1
test_sharing_s3-push[main]        8.0908 (1.0)       8.0908 (1.0)       8.0908 (1.0)      0.0000 (1.0)       8.0908 (1.0)      0.0000 (1.0)           0;0  0.1236 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing_s3-push-noop': 7 tests --------------------------------------------------------------------------------
Name (time in ms)                            Min                   Max                  Mean            StdDev                Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-push-noop[2.11.0]       984.0158 (1.01)       984.0158 (1.01)       984.0158 (1.01)     0.0000 (1.0)        984.0158 (1.01)     0.0000 (1.0)           0;0  1.0162 (0.99)          1           1
test_sharing_s3-push-noop[2.18.1]       972.2939 (1.0)        972.2939 (1.0)        972.2939 (1.0)      0.0000 (1.0)        972.2939 (1.0)      0.0000 (1.0)           0;0  1.0285 (1.0)           1           1
test_sharing_s3-push-noop[2.39.0]     1,024.4190 (1.05)     1,024.4190 (1.05)     1,024.4190 (1.05)     0.0000 (1.0)      1,024.4190 (1.05)     0.0000 (1.0)           0;0  0.9762 (0.95)          1           1
test_sharing_s3-push-noop[2.40.0]     1,019.8112 (1.05)     1,019.8112 (1.05)     1,019.8112 (1.05)     0.0000 (1.0)      1,019.8112 (1.05)     0.0000 (1.0)           0;0  0.9806 (0.95)          1           1
test_sharing_s3-push-noop[2.41.1]     1,037.8599 (1.07)     1,037.8599 (1.07)     1,037.8599 (1.07)     0.0000 (1.0)      1,037.8599 (1.07)     0.0000 (1.0)           0;0  0.9635 (0.94)          1           1
test_sharing_s3-push-noop[2.45.0]     1,008.8259 (1.04)     1,008.8259 (1.04)     1,008.8259 (1.04)     0.0000 (1.0)      1,008.8259 (1.04)     0.0000 (1.0)           0;0  0.9913 (0.96)          1           1
test_sharing_s3-push-noop[main]       1,102.6023 (1.13)     1,102.6023 (1.13)     1,102.6023 (1.13)     0.0000 (1.0)      1,102.6023 (1.13)     0.0000 (1.0)           0;0  0.9069 (0.88)          1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

@pmrowla pmrowla changed the title tests: update 3.x stage hash tests: update for dvc 3.x Jun 12, 2023
@efiop
Copy link
Contributor

efiop commented Jun 12, 2023

Looks like the rest of the failures are my fault. I'll merge and fix on top. Thank you.

@efiop efiop merged commit e6f0d74 into main Jun 12, 2023
@pmrowla pmrowla deleted the dvc-3.x branch June 12, 2023 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@pmrowla pmrowla

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pmrowla @efiop