You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should automate the routine mutation testing issue creation (cref #2069#2105 ).
This should be pretty straightforward:
Make github action that runs once per week
It runs make test-mutation
Takes mutation_test_result.txt, and creates a github issue. (with some formatting. Perhaps only parse out failures, and split them up with a number and separate code block)
The mutation test issue should also include next steps. (Going from the workflow noted before):
We then manually "group" these issues and sort into buckets:
{errors that are legit}
{errors that are noise}
Then for legit bucket, further sort errors legitimate errors by testing strategy that will hit it, and make new issues for that testing strategy into follow-up issues. These should be solved within the week, or otherwise be looped into a larger plan.
For the noise bucket, make a new github issue detailing the noise error type. We should aim to get a fork of gomuttest / try disabling noise / finding a strategy to mitigate noise.
The text was updated successfully, but these errors were encountered:
Background
We should automate the routine mutation testing issue creation (cref #2069 #2105 ).
This should be pretty straightforward:
The mutation test issue should also include next steps. (Going from the workflow noted before):
{errors that are legit}
{errors that are noise}
Then for legit bucket, further sort errors legitimate errors by testing strategy that will hit it, and make new issues for that testing strategy into follow-up issues. These should be solved within the week, or otherwise be looped into a larger plan.
For the noise bucket, make a new github issue detailing the noise error type. We should aim to get a fork of gomuttest / try disabling noise / finding a strategy to mitigate noise.
The text was updated successfully, but these errors were encountered: