-
Notifications
You must be signed in to change notification settings - Fork 3k
Why and how to write unit tests
We can use all kinds of feedback loops to detect problems in production code. For example, we can keep an eye on crash reports, bug issues from QAs and customer complaints. But that's the longest loop. After making an incorrect change, it takes a long time (weeks or months) to get that feedback. The easiest and fastest feedback is having automated testing that runs on every change we make in our CI pipeline. This is what unit tests are for. In short, unit tests:
By running unit tests frequently during the development process, you can catch bugs and issues early on in the development cycle, when they are easier and cheaper to fix.
Unit tests provide a safety net when refactoring or modifying existing code. If you make a change that causes a test to fail, you know immediately that something went wrong, and you can fix it before it gets further down the line (into QA hands for example, or worst case scenario, in Beta or Release).
It improves the code quality by reducing the code complexity. Writing unit tests forces you to think more carefully about the design and functionality of your code, which can lead to better code quality overall. Additionally, having a comprehensive suite of tests can help to ensure that your code is robust and reliable.
You can develop faster. While it may seem like writing tests takes extra time, in the long run, it can actually speed up development. By catching bugs early and preventing regressions, you can avoid wasting time and resources on fixing problems that could have been prevented.
Unit tests provide a shared understanding of how a piece of code is intended to work, while also documenting it. By having a suite of tests that everyone can run and rely on, it can facilitate collaboration and make it easier for team members to work together.
Unit tests are a subset of automated tests where the feedback is quick, consistent, and unambiguous.
- Quick: A single unit test should complete in milliseconds, enabling us to have thousands of such tests.
- Consistent: Given the same code, a unit test should report the same results. The order of execution shouldn't matter. Global state shouldn't matter.
- Unambiguous: A failing unit test should clearly report the problem detected.
A test suite goal is to test the functionalities of a class, ensuring any modifications in that part of the code will not introduce a regression. This class under test can be called our subject, also called system under test. We prefer to use subject in Firefox iOS. In our project, we use the provided XCTestCase to create our test suites. If you're unfamiliar with unit testing with Swift here's a good article on how to get started, as the rest of this page assumes you know the basics.
This isn’t an exhaustive list, but those are the minimum to follow when writing tests:
- It’s important to test all code paths in your unit tests. This includes testing both the expected behavior and any potential edge cases or error scenarios.
- Inside each test case, follow the
Given,When,Then(orArrange,Act,Assert) principle. Adding blank lines between each of those phase helps clarify the function of each line of code in your tests. - Use mocking to allow you to create controlled environments for your tests and test only the specific behavior you're interested in. More on how to do so in next sections.
- Use descriptive names that clearly describe what is being tested. You can name your functions using the Given/When/Then principle as well. This is to make it easier for others to understand what the test is doing.
- Have one use case per unit test helps to have small and focused tests. This makes it easier to debug and maintain your tests in the long term.
- Aim for maximum code coverage: While we can aim for 100% test coverage, this might not be always desirable or possible. That being said, we should aim for the most possible coverage given our set of constraints.
- Avoid conditional branches in test code to keep test code simple. You can do this by choosing an assertion that expresses the condition you need.
- Use the right assertions to check if the test is passing or failing. Example:
XCTAssertFalseorXCTAssertTrueto check if a Boolean is false or true,XCTAssertNilorXCTAssertNotNilto check the presence of an object,XCTAssertEqualto check for equality between two objects, etc.
There's more to tests than assertions. When does XCTest create and run the tests? To avoid flaky tests, we want to run each test in a virtual clean room. There should be no leftovers from previous tests or from manual runs.
Each test method runs in a separate instance of it's XCTestCase subclass. These instances live inside a collection of all tests, which the test runner is iterating over. So all tests cases exists before test execution and are never deallocated until the test runner terminates. This means it's important to use setUp and tearDown methods to respectively create and nullify any stored properties in our XCTestCase subclasses. The preferred method is to use unwrapped optional:
private var subject: SubjectClass!
override func setUp() {
super.setUp()
subject = SubjectClass()
}
override func tearDown() {
super.tearDown()
subject = nil
}
When testing is difficult, this reveals flaws in the architectural design of the code. By making changes to enable testing, you'll be shaping the code into a cleaner design. Design decisions that were once hidden and implicit will become visible and explicit.
But what are difficult cases to test? It includes cases that are:
- Slow: Code that execute in response to external triggers, i.e. if there's no way to trigger the code execution immediately.
- Non-isolated: Dependencies that break the rule of isolation, such as global variables and persistent storage. We can think of singletons or static properties as well.
- Non-repeatable: Dependencies that yield a different result when called. Like current time or date or random numbers.
- Side effects: Dependencies that cause side effects outside the invoked type, such as playing audio or video. There exists others cases, but those are good to keep in mind when planning how you'll write code and how it can be tested.
Once we've identified dependencies that make testing difficult, what should we do with them? We need to find ways to isolate them between boundaries. In other words, the subject under tests shouldn't care about implementation details of the dependencies it uses. Having them isolated enables us to replace them with substitutes during testing, so we keep the test consistent, quick and unambiguous. We can implement boundaries using protocols instead of relying on the concrete object in our production code, which will then enable us to use a technique called mocking in our test cases.
To isolate the behavior of the object you want to test, you replace the other objects by mocks that simulate the behavior of the real objects. This is useful if the real objects are impractical to incorporate into the unit test (see previous section on difficult cases to test). In other words, mocking is creating objects that simulate the behavior of real objects.
Here's an example. Let's say our subject to test is called SnowMachine, and depend on the Weather to create snow.
class SnowMachine {
// Create snow from a certain water quantity, depends on the weather to cristalize snow
func createSnow(with waterQuantity: Int) -> Int {
guard Weather().getTemperature() <= 0 else { return 0 }
return waterQuantity / 2
}
}
class Weather {
private var temperature: Int = 0 // in Celcius
func getTemperature() -> Int {
// Imagine here a complicated algorithm we don't control to determine the temperature
return temperature
}
}
Now our current SnowMachine depends on the concrete type of Weather. If we want to unit tests this class, we cannot control the temperature input and therefor wouldn't be able to test the different edge cases of snow making. A better approach would be using dependency injection and a protocol such as:
protocol WeatherProtocol {
func getTemperature() -> Int
}
class SnowMachine {
private var weather: WeatherProtocol
init(weather: WeatherProtocol) {
self.weather = weather
}
// Create snow from a certain water quantity, depends on the weather to cristalize snow
func createSnow(with waterQuantity: Int) -> Int {
guard weather.getTemperature() <= 0 else { return 0 }
return waterQuantity / 2
}
}
This way we can create a mock of the Weather class, that we'll be able to inject into our subject to test the different cases and control the temperature. Example:
final class SnowMachineTests: XCTestCase {
private var weather: MockWeather!
override func setUp() {
super.setUp()
weather = MockWeather()
}
override func tearDown() {
super.tearDown()
weather = nil
}
func testColdTemperature_SnowIsCreated() {
weather.mockedTemperature = -10
let subject = SnowMachine(weather: weather)
let result = subject.createSnow(with: 10)
XCTAssertEqual(result, 5, "Quantity of snow created is half of the water input")
}
func testWarmTemperature_SnowIsntCreated() {
weather.mockedTemperature = 10
let subject = SnowMachine(weather: weather)
let result = subject.createSnow(with: 10)
XCTAssertEqual(result, 0, "There was no snow create as the temperature is too warm")
}
func testZeroTemperature_SnowIsCreated() {
weather.mockedTemperature = 0
let subject = SnowMachine(weather: weather)
let result = subject.createSnow(with: 10)
XCTAssertEqual(result, 5, "Quantity of snow created is half of the water input")
}
}
final class MockWeather: WeatherProtocol {
var mockedTemperature = 0
func getTemperature() -> Int {
return mockedTemperature
}
}
Note that with legacy code, it can be necessary to use techniques such as subclassing or overriding to be able to write unit tests. The idea is to create a subclass of production code that lives only in test code, giving us a way to override methods that are problematic for testing. Subclassing and overriding shouldn't be used as testing strategies in new code, and if you feel yourself forced to used such strategy to test code please discuss with the team so we can find a solution together.