This guide is intended to layout the general automated test strategy for OpenLMIS.
OpenLMIS, like many software projects, relies on testing to guide development and prevent regressions. To effect this we've adopted a standard set of tools to write and execute our tests, and categorize them to understand what types of tests we have, who writes them, when they're written, run, and where they live.
The following test categories have been identified for use in OpenLMIS. As illustrated in this great slide deck, we expect the effort/number of tests in each category to reflect the test pyramid:
- Who: written by code-author during implementation
- What: the smallest unit (e.g. one piece of a model's behavior, a function, etc)
- When: at build time, should be /fast/ and targetted - I can run just a portion of the test suite
- Where: Reside inside a service, next to unit under test. Generally able to access package-private scope
- Why: to test fundamental pieces/functionality, helps guide and document design and refactors, protects against regression
-
Every single test should be independent and isolated. Unit test shouldn't depend on another unit test.
DO NOT:
List<Item> list = new ArrayList<>(); @Test public void testListSize1() { Item item = new Item(); list.add(item); assertEquals(1, list.size()); } @Test public void testListSize2() { Item item = new Item(); list.add(item); assertEquals(2, list.size()); }
-
One behavior should be tested in just one unit test.
DO NOT:
@Test public void testIsNotAnAdultIfAgeLessThan18AndIsPersonAbleToRunForPresident() { int age = 17; boolean isAdult = ageService.isAdult(age); assertFalse(isAdult); boolean isAbleToRunForPresident = electionsService.isAbleToRunForPresident(age) assertFalse(isAbleToRunForPresident); }
DO:
@Test public void testIsNotAnAdultIfAgeLessThan18() { int age = 17; boolean isAdult = ageService.isAdult(age); assertFalse(isAdult); } @Test public void testIsPersonAbleToRunForPresident() { int age = 17; boolean isAbleToRunForPresident = electionsService.isAbleToRunForPresident(age) assertFalse(isAbleToRunForPresident); }
-
Every unit test should have at least one assertion.
DO NOT:
@Test public void testIsNotAnAdultIfAgeLessThan18() { int age = 17; boolean isAdult = ageService.isAdult(age); }
DO:
@Test public void testIsNotAnAdultIfAgeLessThan18() { int age = 17; boolean isAdult = ageService.isAdult(age); assertFalse(isAdult); }
-
Don't make unnecessary assertions. Don't assert mocked behavior, avoid assertions that check the exact same thing as another unit test.
DO NOT:
@Test
public void testIsNotAnAdultIfAgeLessThan18() {
int age = 17;
assertEquals(17, age);
boolean isAdult = ageService.isAdult(age);
assertFalse(isAdult);
}
-
Unit test has to be independent from external resources (i.e. don't connect with databases or servers)
DO NOT:
@Test
public void testIsNotAnAdultIfAgeLessThan18() {
String uri = String.format("http://127.0.0.1:8080/age/", HOST, PORT);
HttpPost httpPost = new HttpPost(uri);
HttpResponse response = getHttpClient().execute(httpPost);
assertEquals(HttpStatus.ORDINAL_200_OK, response.getStatusLine().getStatusCode());
}
-
Unit test shouldn't test Spring Contexts. Integration tests are better for this purpose.
DO NOT:
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = {"/services-test-config.xml"})
public class MyServiceTest implements ApplicationContextAware
{
@Autowired
MyService service;
...
@Override
public void setApplicationContext(ApplicationContext context) throws BeansException
{
// something with the context here
}
}
-
Test method name should consistently show what is being tested.
DO:
@Test public void testIsNotAnAdultIfAgeLessThan18() { ... }
DO NOT:
@Test public void firstTests() { ... }
-
Unit test should be repeatable - each run should yield the same result.
DO NOT:
@Test public void testIsNotAnAdultIfAgeLessThan18() { int age = randomGenerator.nextInt(100); boolean isAdult = ageService.isAdult(age); assertFalse(isAdult); }
-
You should remember about intializing and cleaning each global state between test runs.
DO:
@Mock private AgeService ageService; private age; @Before public void init() { age = 18; when(ageService.isAdult(age)).thenReturn(true); } @Test public void testIsNotAnAdultIfAgeLessThan18() { boolean isAdult = ageService.isAdult(age); assertTrue(isAdult); }
-
Test should run fast. When we have hundreds of tests we just don't want to wait several minutes till all tests pass.
DO NOT:
@Test
public void testIsNotAnAdultIfAgeLessThan18() {
int age = 17;
sleep(1000);
boolean isAdult = ageService.isAdult(age);
sleep(1000);
assertFalse(isAdult);
}
- Who: Code author during implementation
- What: Test basic operation of a service to persistent storage or a service to another service. When another service is required, a test-double should be used, not the actual service.
- When: As explicitly asked for, these tests are typically slower and therefore need to be kept seperate from build to not slow development. Will be run in CI on every change.
- Where: Reside inside a service, seperated from other types of tests/code.
- Why: Ensures that the basic pathways to a service's external run-time dependancies work. e.g. that a db schema supports the ORM, or a non-responsive service call is gracefully handled.
- Who: Code author during implementation, with input from BA/QA.
- What: Enforces contracts between and to services.
- When: Ran in CI.
- Where: Reside inside seperate repository: openlmis-contract-tests.
- Why: Tests multiple services working together, testing contracts that a Service both provides as well as the requirements a dependant has.
- Who: QA / developer with input from BA.
- What: Typical/core business scenarios.
- When: Ran in CI.
- Where: Resides in seperate repository.
- Why: Ensures all the pieces are working together to carry-out a business scenario. Helps ensure end-users can achieve their goals.
OpenLMIS is using WireMock for mocking web services. An example integration test can be found here: https://github.com/OpenLMIS/openlmis-example/blob/master/src/test/java/org/openlmis/example/WeatherServiceTest.java
The stub mappings which are served by WireMock's HTTP server are placed under src/test/resources/mappings and _src/test/resources/_files For instructions on how to create them please refer to http://wiremock.org/record-playback.html