testeth: add stTimeConsuming #5576
Conversation
test/tools/jsontests/StateTests.cpp
Outdated
| @@ -147,7 +147,8 @@ class GeneralTestFixture | |||
| boost::filesystem::path suiteFillerPath = suite.getFullPathFiller(casename).parent_path(); | |||
|
|
|||
| // Check specific test cases | |||
| static vector<string> timeConsumingTestSuites{string{"stQuadraticComplexityTest"}}; | |||
| static vector<string> timeConsumingTestSuites{ | |||
|
cc @chfast |
|
I don't understand the direction this is going. Is it safe to skip these tests on regular basis? I'd rather check why they take so much time, maybe we can reduce the number of them, etc. If you disable them, 99% of users (including us) will not run them. If you moved e.g. stSstore tests, does it mean we don't test (unless explicitly enabled) one of the features at all? |
|
Only those tests I moved which take a lot of time to execute. To run all tests you anyway have to run with --all flag |
You mean test suites or test cases? |
|
Test cases. One specific test is 50000 sha256 execution. |
|
Ok, that's fine then.
Are you sure such tests are useful? I believe you get the same effect when running sha256 10x instead of 50000. |
don't remember why exactly Christoph added this test case. but I see its potential to crush some clients or make it timeout due to not optimized coding. |
Codecov Report
@@ Coverage Diff @@
## master #5576 +/- ##
==========================================
- Coverage 62.11% 62.11% -0.01%
==========================================
Files 347 347
Lines 29107 29103 -4
Branches 3302 3299 -3
==========================================
- Hits 18080 18077 -3
+ Misses 9847 9843 -4
- Partials 1180 1183 +3 |
|
net tests failing randomly |
| - Changed: [#5568](https://github.com/ethereum/aleth/pull/5568) Improve rlpx handshake log messages and create new rlpx log channel. | ||
| - Changed: [#5576](https://github.com/ethereum/aleth/pull/5576) Moved sstore_combinations and static_Call50000_sha256 tests to stTimeConsuming test suite. (testeth runs them only with `--all` flag) |
There was a problem hiding this comment.
(see https://keepachangelog.com/en/1.0.0/#how for changelog format)
this tests take more then 3 minutes of test execution time.
also PR updates the test submodule head