Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add benchmarks to the crate #563

Merged
merged 3 commits into from
Aug 18, 2021
Merged

Add benchmarks to the crate #563

merged 3 commits into from
Aug 18, 2021

Conversation

CPerezz
Copy link
Contributor

@CPerezz CPerezz commented Aug 18, 2021

Results obtained with: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz

Prove 2^5 = 32 constraints                                                                           
                        time:   [18.471 ms 18.564 ms 18.677 ms]

Prove 2^6 = 64 constraints                                                                           
                        time:   [24.155 ms 24.291 ms 24.539 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Prove 2^7 = 128 constraints                                                                           
                        time:   [38.306 ms 38.598 ms 38.874 ms]

Prove 2^8 = 256 constraints                                                                           
                        time:   [60.509 ms 62.260 ms 64.296 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Prove 2^9 = 512 constraints                                                                           
                        time:   [90.462 ms 92.621 ms 95.947 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Benchmarking Prove 2^10 = 1024 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.1s or enable flat sampling.
Prove 2^10 = 1024 constraints                                                                          
                        time:   [150.89 ms 157.24 ms 164.71 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Prove 2^11 = 2048 constraints                                                                          
                        time:   [288.66 ms 289.86 ms 291.59 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Prove 2^12 = 4096 constraints                                                                          
                        time:   [494.76 ms 500.91 ms 511.34 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Benchmarking Prove 2^13 = 8192 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.6s.
Prove 2^13 = 8192 constraints                                                                          
                        time:   [947.94 ms 957.07 ms 966.46 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Benchmarking Prove 2^14 = 16384 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 18.5s.
Prove 2^14 = 16384 constraints                                                                          
                        time:   [1.8323 s 1.8603 s 1.8921 s]

Benchmarking Prove 2^15 = 32768 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 36.1s.
Prove 2^15 = 32768 constraints                                                                          
                        time:   [3.5213 s 3.5465 s 3.5755 s]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Benchmarking Prove 2^16 = 65536 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 65.9s.
Prove 2^16 = 65536 constraints                                                                          
                        time:   [6.6283 s 6.6885 s 6.7484 s]

Benchmarking Prove 2^17 = 131072 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 133.5s.
Prove 2^17 = 131072 constraints                                                                          
                        time:   [13.369 s 13.432 s 13.501 s]

Verify 2^5 = 32 constraints                                                                           
                        time:   [9.9774 ms 10.015 ms 10.066 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Verify 2^6 = 64 constProve 2^5 = 32 constraints                                                                           
                        time:   [18.471 ms 18.564 ms 18.677 ms]

Prove 2^6 = 64 constraints                                                                           
                        time:   [24.155 ms 24.291 ms 24.539 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Prove 2^7 = 128 constraints                                                                           
                        time:   [38.306 ms 38.598 ms 38.874 ms]

Prove 2^8 = 256 constraints                                                                           
                        time:   [60.509 ms 62.260 ms 64.296 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Prove 2^9 = 512 constraints                                                                           
                        time:   [90.462 ms 92.621 ms 95.947 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Benchmarking Prove 2^10 = 1024 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.1s or enable flat sampling.
Prove 2^10 = 1024 constraints                                                                          
                        time:   [150.89 ms 157.24 ms 164.71 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Prove 2^11 = 2048 constraints                                                                          
                        time:   [288.66 ms 289.86 ms 291.59 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Prove 2^12 = 4096 constraints                                                                          
                        time:   [494.76 ms 500.91 ms 511.34 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Benchmarking Prove 2^13 = 8192 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.6s.
Prove 2^13 = 8192 constraints                                                                          
                        time:   [947.94 ms 957.07 ms 966.46 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Benchmarking Prove 2^14 = 16384 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 18.5s.
Prove 2^14 = 16384 constraints                                                                          
                        time:   [1.8323 s 1.8603 s 1.8921 s]

Benchmarking Prove 2^15 = 32768 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 36.1s.
Prove 2^15 = 32768 constraints                                                                          
                        time:   [3.5213 s 3.5465 s 3.5755 s]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Benchmarking Prove 2^16 = 65536 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 65.9s.
Prove 2^16 = 65536 constraints                                                                          
                        time:   [6.6283 s 6.6885 s 6.7484 s]

Benchmarking Prove 2^17 = 131072 constraints: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 133.5s.
Prove 2^17 = 131072 constraints                                                                          
                        time:   [13.369 s 13.432 s 13.501 s]

Verify 2^5 = 32 constraints                                                                           
                        time:   [9.9774 ms 10.015 ms 10.066 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Verify 2^6 = 64 constraints                                                                           
                        time:   [10.036 ms 10.177 ms 10.408 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Verify 2^7 = 128 constraints                                                                           
                        time:   [10.139 ms 10.524 ms 10.996 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Verify 2^8 = 256 constraints                                                                           
                        time:   [10.031 ms 10.143 ms 10.233 ms]

Verify 2^9 = 512 constraints                                                                           
                        time:   [10.069 ms 10.121 ms 10.192 ms]

Verify 2^10 = 1024 constraints                                                                           
                        time:   [10.046 ms 10.093 ms 10.126 ms]

Verify 2^11 = 2048 constraints                                                                           
                        time:   [10.083 ms 10.369 ms 10.653 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Verify 2^12 = 4096 constraints                                                                           
                        time:   [10.194 ms 10.267 ms 10.351 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Verify 2^13 = 8192 constraints                                                                           
                        time:   [10.157 ms 10.238 ms 10.342 ms]

Verify 2^14 = 16384 constraints                                                                           
                        time:   [10.114 ms 10.151 ms 10.217 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Verify 2^15 = 32768 constraints                                                                           
                        time:   [10.254 ms 10.301 ms 10.384 ms]

Verify 2^16 = 65536 constraints                                                                           
                        time:   [10.409 ms 10.436 ms 10.480 ms]

Verify 2^17 = 131072 constraints                                                                           
                        time:   [10.639 ms 10.684 ms 10.728 ms]

raints                                                                           
                        time:   [10.036 ms 10.177 ms 10.408 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Verify 2^7 = 128 constraints                                                                           
                        time:   [10.139 ms 10.524 ms 10.996 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Verify 2^8 = 256 constraints                                                                           
                        time:   [10.031 ms 10.143 ms 10.233 ms]

Verify 2^9 = 512 constraints                                                                           
                        time:   [10.069 ms 10.121 ms 10.192 ms]

Verify 2^10 = 1024 constraints                                                                           
                        time:   [10.046 ms 10.093 ms 10.126 ms]

Verify 2^11 = 2048 constraints                                                                           
                        time:   [10.083 ms 10.369 ms 10.653 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

Verify 2^12 = 4096 constraints                                                                           
                        time:   [10.194 ms 10.267 ms 10.351 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Verify 2^13 = 8192 constraints                                                                           
                        time:   [10.157 ms 10.238 ms 10.342 ms]

Verify 2^14 = 16384 constraints                                                                           
                        time:   [10.114 ms 10.151 ms 10.217 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

Verify 2^15 = 32768 constraints                                                                           
                        time:   [10.254 ms 10.301 ms 10.384 ms]

Verify 2^16 = 65536 constraints                                                                           
                        time:   [10.409 ms 10.436 ms 10.480 ms]

Verify 2^17 = 131072 constraints                                                                           
                        time:   [10.639 ms 10.684 ms 10.728 ms]

Resolves: #555

@CPerezz CPerezz added the team:Core Low Level Core Development Team (Rust) label Aug 18, 2021
@CPerezz CPerezz requested review from ZER0 and vlopes11 August 18, 2021 14:06
ZER0
ZER0 approved these changes Aug 18, 2021
View reviewed changes
Copy link
Contributor

@ZER0 ZER0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@CPerezz CPerezz merged commit 9af8ea1 into release-0.9 Aug 18, 2021
@CPerezz CPerezz deleted the criterion_benches branch August 18, 2021 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ZER0 ZER0 ZER0 approved these changes

@vlopes11 vlopes11 Awaiting requested review from vlopes11

Assignees
No one assigned
Labels
team:Core Low Level Core Development Team (Rust)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@CPerezz @ZER0 @vlopes11