Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This Pr optimises rules tests in a way that we no longer perform e2e test but do integration tests for them instead.
Trough profiling we've found that biggest chunk cpu time is spent on compiling all the rules.
I've optimised it in a way that we use worker to compile all the rules only once, then reuse that worker.
It isn't without it's downsides. Because we reuse single testing worker, and because we use viper for default config, we can no longer run tests in parallel.
Getting tests to run in parallel is limited to single test case which translates to single rule (it usually has multiple files which run in parallel)
However I've manged to improve the total time it takes to run our rules test by 2x-3x and allow us to run tests with debugger
summary changes
There are cases where we detect 2 different rules in summary.
Eg. of those rules is
ruby_weak_encryption
andruby_weak_encryption_with_data
stats
localmachine (macbook pro)
old_summary - 187.077s
new - 51.512s
github runner
old_summary - 332s
new - 183.678s
related
#588
Checklist