[Tests] Improve benchmark precision #2538
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Great backport.
splendid. Love the features that martinus added upstream ❤️ There is an issue with gdb seems to suggest that Initializing a const |
|
Might be the case of a dangling |
|
cmake build fails (when Need to add if(NAT-PMP_FOUND)
target_link_libraries(bench_pivx PRIVATE ${NAT-PMP_LIBRARY})
target_include_directories(bench_pivx PRIVATE ${NAT-PMP_INCLUDE_DIR})
endif()to |
furszy
force-pushed
the
2021_improve_bench_framework
branch
from
b7c3576
to
dbcaf86
Compare
|
done, added NAT-PMP link. Squashed in 70b1485. |
|
Even though the documentation says
with the current hardcoded With the following changes: BENCHMARK(Base58Encode, 58 * 1000);
BENCHMARK(Base58CheckEncode, 43 * 1000);
BENCHMARK(Base58Decode, 136 * 1000);
BENCHMARK(BenchTimeDeprecated, 158000000);
BENCHMARK(BenchTimeMillis, 3460000);
BENCHMARK(BenchTimeMillisSys, 3460000);
BENCHMARK(BenchTimeMock, 59500000);
BENCHMARK(CCheckQueueSpeedPrevectorJob, 122);
BENCHMARK(CHACHA20_64BYTES, 5000000);
BENCHMARK(CHACHA20_256BYTES, 1250000);
BENCHMARK(CHACHA20_1MB, 338);
BENCHMARK(DeserializeBlockTest, 56500);
BENCHMARK(DeserializeAndCheckBlockTest, 4500);
BENCHMARK(FastRandom_1bit, 195 * 1000 * 1000);
BENCHMARK(FastRandom_32bit, 34 * 1000 * 1000);
BENCHMARK(LockedPool, 58);
PREVECTOR_TEST(Clear, 4042, 25300)
PREVECTOR_TEST(Destructor, 4058, 24400)
PREVECTOR_TEST(Resize, 4059, 25200)
BENCHMARK(RIPEMD160, 275);
BENCHMARK(SHA1, 305);
BENCHMARK(SHA256, 156);
BENCHMARK(SHA256_32b, 2300 * 1000);
BENCHMARK(SHA512, 240);I get total times closer to 1 sec: It would be useful to see the results on different systems to compare. |
|
Running a single evaluation for each bench:
|
|
After the changes, same evaluations, i get the following results:
|
furszy
force-pushed
the
2021_improve_bench_framework
branch
from
dbcaf86
to
79f8271
Compare
* inline performance critical code * Average runtime is specified and used to calculate iterations. * Console: show median of multiple runs * plot: show box plot * filter benchmarks * specify scaling factor * ignore src/test and src/bench in command line check script * number of iterations instead of time * Replaced runtime in BENCHMARK makro number of iterations. * Added -? to bench_bitcoin * Benchmark plotly.js URL, width, height can be customized * Fixed incorrect precision warning
This benchmark's runtime was rather unpredictive on different machines, not really a useful benchmark.
furszy
force-pushed
the
2021_improve_bench_framework
branch
from
79f8271
to
a5fcdac
Compare
|
Yeah, we can leave the numbers as they are. We won't find values that sort-of-work with all possible machines/systems anyway. |
|
i rebased it right after sharing my results (3 days ago) and squashed the new benchmarks inside 17c4bcf ;). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Coming mainly from bitcoin#11517, which improves microbenchmarking with multiple features:
Plus added CMake support for the benchmarking framework as well.
-- Needed for a coming PR that i'm cooking that contains a new benchmark --