Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor tests #4

Merged
merged 5 commits into from
Oct 20, 2024
Merged

refactor tests #4

merged 5 commits into from
Oct 20, 2024

Conversation

danieleades
Copy link
Contributor

small refactor of the tests.

None of these are integration tests. I've moved the performance tests to 'benches' and the remaining tests to unit tests within the lib.

It might be an idea to further refactor out the randomised test to a specific fuzzing target

@danieleades
Copy link
Contributor Author

for discussion- i've removed the 'inline' attributes. This leads to slightly better performance on most bench tests on my machine:

Gnuplot not found, using plotters backend
shortest_csc_path       time:   [260.61 ns 261.85 ns 263.15 ns]                              
                        change: [-22.085% -17.785% -13.408%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) high mild
  7 (7.00%) high severe

shortest_ccc_path       time:   [187.12 ns 188.96 ns 191.87 ns]                              
                        change: [-2.6969% -1.0143% +0.6483%] (p = 0.25 > 0.05)
                        No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) high mild
  7 (7.00%) high severe

shortest_path           time:   [354.46 ns 356.52 ns 358.73 ns]                          
                        change: [-15.823% -12.699% -9.7411%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe

many_sample             time:   [10.384 µs 10.846 µs 11.342 µs]                         
                        change: [+6.9659% +10.427% +13.981%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe

i suspect this is because the const methods can be more efficiently optimised if they're not inlined.

There is one performance regression here though- might be interesting in future to selectively replace the inline calls, verifying with the bench tests, and see which one is useful and which are counterproductive

Copy link
Owner

@VirxEC VirxEC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I've been meaning to properly separate out the benchmarks.

@VirxEC VirxEC merged commit 7d3aacf into VirxEC:master Oct 20, 2024
@danieleades danieleades deleted the bench branch October 20, 2024 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants