16

This release of Muter contains bug fixes and new features!

🆕 What's new? 🆕

• We are pleased to announce the introduction of Mutant Schemata. This feature increases the performance a lot, please give it a try and let us know if you found any issues.
• Fetching files coverage now works.
• Now you can pass a list of operators to be applied with the flag --operators <operators>.
• Operators now have a brief documentation that can be checked on the CLI. Use muter operator all print all the available operators.
• Muter checks for the latests version and let you know if there is a new version. This can be disabled by passing the flag --skip-update-check.
• The report format now have its own command: --format <format> with the existing reports plain, json, html and, xcode. Passing --output will save it on the disk.

📣🎉 Shout outs 🎉📣
Special thanks to @ZevEisenberg for the help and the cool ideas.

v15

This release of Muter contains bug fixes and new features!

🆕 What's new? 🆕

Exclusion lists now supports glob expressions, like /Sources/**/*.swift #179
Reports now show before vs after for all mutation operators #102
New HTML report #194

⬆️ What's improved? ⬆️

Ignores line within compiler directives #114
Ignores Swift Package Manager manifest file #186
Uses test process termination code #185 #189
@ZevEisenberg
Ignores files without coverage #197

📣🎉 Shout outs 🎉📣

Special thanks to @ZevEisenberg for his amazing work and @karlprieb for helping our HTML report come to life.

v14

This release of Muter updates it to Swift 5.3 and many suggestions/improvements that came from you all, the community!

🆕 What's new? 🆕

We've added support for users to skip mutations from being applied.
@Uriah-Eisenstein-ATT #153

⬆️ What's improved? ⬆️

We are now appending the mutation summary at the end of Xcode report.
@mrvincenzo #157

📣🎉 Shout outs 🎉📣

Thanks to @ZevEisenberg and @rakaramos, muter went through a big upgrade. swift-syntax has been updated, Commandant was replaced by Swift Argument Parser for parsing commands, and we now have a fancy logo 🧟

v13

This release of Muter contains a bug fix and a new support for projects that are using Buck as build system.

🆕 What's new? 🆕

• We've added support for Buck outputs (thanks @henrique-morbin-ifood)

⬆️ What's improved? ⬆️

• The side-effect operator now will try to avoid removing code that can lead to deadlocks, causing Muter to hang.
• We've changed the language for NegateConditionals to RelationalOperatorReplacement.

An Import Bug Fix for Xcode Users

This release of Muter contains a fix for users who use Muter within Xcode.

⬆️ What's improved? ⬆️

The internals of Muter underwent a substantial rewrite in v11, which introduced a regression where the paths used to report the location of surviving mutants in your code pointed to the temporary directory Muter operated from, rather than your project's directory. This release fixes this issue.

Thanks to @ZevEisenberg for reporting this issue. If you're interested in how we improve our testing strategy to prevent this from happening again, check out this comment.

v11

This release of Muter adds a whole bunch of new functionality, including a facelift for our commandline users. This release is a significant step towards making Muter an effective & easily adopted CI tool.

🆕 What's new? 🆕

• We've added a new command line flag so you can practice incremental mutation testing: --files-to-mutate! You can optionally use this flag, and provide a space separated list of files, paths, or glob expressions which point Muter to the files you would like to mutation test. Using this flag overrides your exclude list from your muter.conf.json.
• We've added a progress bar which estimates how long until mutation testing is finished when you are running Muter from the commandline. It's initial estimate will be very conservative, but it will get more accurate the longer it runs.
• We've rewritten how the commandline output looks, and what information it provides you. Previously, it was difficult to know where the temporary directory Muter was working from. To solve that, Muter now logs where it copies your project, giving you deeper insight into how Muter works in the event you need to debug something.
• ...and speaking of debugging, Muter now copies the logs it generates to the root of your project directory into a directory called muter_logs. You can use these logs to diagnose how a particular mutant impacts your test suite, or to troubleshoot an issue you may be experiencing when setting Muter up.
• Should your test suite or application fail to compile or pass Muter's initial baseline test, Muter will now print out the issue so you don't have to go digging around for a log file. This is so you can have a faster and streamlined set up experience.

⬆️ What's improved? ⬆️

• The internals of Muter underwent a substantial rewrite so as to make new features easier to add and to significantly reduce the memory footprint of Muter. Muter now uses a fraction of the memory it previously had while it's running.

Many thanks to @rakaramos for the amount of work he did for this release on the new commandline flag and making it easier for you to find your logs. This is a release that was truly a team effort. I'm excited for the next few releases that are coming up!

v10

This release of Muter updates it to Swift 5. In addition to this, it contains a handful of new features we're quite excited about!

🆕 What's new? 🆕

• There is a new mutation operator, InvertLogicalConnector! This operator is useful for driving out testing issues that don't catch all the branches that your logical operators can take. Read more about it here.
• Running muter init will now generate a configuration file that is filled out with Muter's best guess at a configuration that will work for your project. This means that it can be as simple as muter init && muter to get your project up and running! Right now, the automatic configuration generation supports projects that are using Xcode projects and workspace, as well as SPM projects. If you don't use any of these for your build and test system, Muter is still configurable manually inside of its muter.conf.json.

⬆️ What's improved? ⬆️

• The error messaging for when Muter detects a missing or invalid configuration has been improved to help you troubleshoot what might be going wrong.

Many thanks to @rakaramos, @ZevEisenberg, and a many others for their suggestions on what to improve, and their help getting this release together. A particular amount of gratitude is due to @mdiasdev for his help getting a CI set up for this project. Thank you all! :D

v8

This release of Muter contains many suggestions and improvements that came from you all, the community!

🆕 What's new? 🆕

• Muter is now formally licensed under the GNU Affero General Public License v3.0 (GNU AGPL v3). You can see the rationale behind the licensing choice here. (Thanks for reminding us of the importance of this, @ZevEisenberg)

⬆️ What's improved? ⬆️

• Muter's reporting of mutation test results now avoids using the language of passing and failing. Instead, it uses the domain language of having killed mutants or surviving mutants, with an accompanying reason for how a mutant was killed. Check out the new test report format here. This was in response to many of you finding the previous language confusing, which hindered your ability to digest the report and make changes based on it.
• Muter's commandline output will now concatenate the filename and line number where a mutation operator was applied, which allows you to more rapidly look it up in your editor of choice. Previously, these were separate columns. Now it's formatted as fileName.swift:1. (Thanks for the suggestion, @fcanas)
• Muter will now ignore operator declarations when discovering where it can apply the negate conditionals operator. This change eliminates a category of build errors that Muter caused by modifying operator declarations. Muter will now run faster on code bases which have custom operator declarations. (Thanks again @fcanas!)
• Muter now gracefully handles an edge case were a division by zero would occur if your test report only contained build errors.

v7

This release of Muter contains improvements to Muter's XCTest integration, a new report file format, and a new command line argument.

🆕 What's new? 🆕

• Muter now has the option of conditionally exporting a JSON report of all of the information contained in its test report. You can enable this by running Muter with the new commandline flag - --output-json. The report will be written to a file named muterReport.json. Stay tuned for some interesting features we have planned which will take advantage of this report format. ;)
• Muter will now abort testing if it notices 5 consecutive build errors during mutation testing. This is a fail-fast feature intended to help callout anomalous test runs to you. This makes it faster to diagnose any potential issues Muter may have running your test suite.

⬆️ What's improved? ⬆️

• Muter is now able to identify more cases of test failures within XCTest's test logs. This improves the accuracy of the mutation scores generated by Muter.

📣🎉 Shout outs 🎉📣
I'm excited to shout out @rakaramos for his work on adding the new commandline argument and fail-fast/fail-early features that are included with this release. He worked on the bulk of the features included in this release, and is a generally awesome person to collaborate with. 👍

I'm also excited to shout out @reedr3, who I paired with on writing Muter's new regression test suite. The work we did together makes sure that future releases of Muter can't adversely affect its test reporting, which is essential for depending on it as a testing tool. ✌️

Thank you both for all of your work, and here's to many more releases!

v6

This release includes several updates that affect Muter's handling of build errors, run-time errors, and mutation scores.

Specifically:

• Muter performs an initial test suite run which will prevent mutation testing if it fails. In previous versions, it would continue performing its tests even if it was unable to compile your application or test suite. This would produce meaningless results and waste time.
• Muter is able to identify when a build error occurs as a result of a mutant it creates, and no longer treats a build error as a passing mutation test. This improves the signal of generated mutation scores.
• Muter now identifies run-time errors that lead to a crash, and differentiates them from test failures.
• Muter's mutation score calculations now include failing tests, run-time errors, and passing tests, but explicitly exclude any build errors.

Additionally, there's been an improvement to the messaging when Muter doesn't discover source files; it's output now includes some suggestions to help you troubleshoot.

And lastly, there are a few minor style and copy changes to the mutation scores to make them a little bit easier to read.