Writing effective unit tests

Unit tests are as important as the business code. Hence it is imperative to discuss how to write effective unit tests and make sure they are well maintained. It is because if we take care of our unit tests, they will take care of our code.

I can’t stress enough the importance of unit tests, and if you haven’t already, check out my article on why I consider the unit tests so important. Having said that, in this article, we will not discuss the benefits of unit tests and it is assumed that we all agree that unit tests are important. Let us have detailed guidelines on how to write tests effectively so that they are useful and meaningful.

As I said, these are guidelines that I have devised based on my experience and learning from different sources. Since these are guidelines, it is ok to break them once in a while. But we have a good reason to do so.

Don’t focus on coverage

As counter-intuitive as it might sound, just writing unit tests to increase the coverage is not a good idea. A test suite with good coverage is obviously desirable, however, tests written with just the coverage in mind might not be as ideal. A well-written and meaningful suite of tests, as we will discuss further in this article, should automatically result in greater coverage. This implies that coverage is the byproduct of a well-written test suite.

Focus on a single unit of test

A single unit test must focus on a single unit of code under test. Does that mean one test must have only a single assert statement? No. We can have multiple asserts in a single unit test, however, it should test a single logical unit.

Lets say we have a UserService that returns a list of all users. It also supports filtering of users by name.

@Test
public void findAllTest() {
  List<User> users = userService.findAll(nameFilter);
  assertNotNull(users);
  assert(users.size(), 3);

  List<User> users = userService.findAll(nameDoesNotExistFilter);
  assertNotNull(users);
  assert(users.size(), 0);
}

Apart from other problems that we will discuss later in this article, one problem in this unit test is that it tests two different logical units in a single test. This is not desirable. First, the test checks how does the service behave when a valid name filter is passed. Then it goes on to check if the service returns an empty list instead of null when the filter does not match any record in the database.

Clearly, these two things deserve two different tests. One major reason is that each test should have a micro focus on a single unit. Another thing is that let’s say because of a bug the UserService API is completely broken and all the methods are misbehaving. In such a scenario, if we combine multiple logical units, the test report will show only a single failure saying findAllTest() failed. But we would never know which all units inside the test failed.

Name the tests properly

The test names must clearly state what unit it is responsible for testing. Looking at the test report we must be able to tell which all things are working and which are not. Continuing with the above example, lets break the test and give proper names.

@Test
public void shouldReturnAllUsersMatchedByFilter() {
  // test code
}

@Test
public void shouldReturnEmptyWhenNoUserIsMatchedByFilter() {
  // test code
}

Obviously, this is not the only correct way. Someone might choose given…When…Then…() style of names. Whatever it is, make sure that it conveys the meaning properly. Someone should be able to get the use-case by reading the name of the test.

One point regarding naming I should bring is that generally, we should not have tool lengthy names for methods. However, when it comes to tests, we have a bit of freehand. This is not because tests are lesser important than the business code, I would argue its the reverse 😉

(Java-specific) Repeat after me, do not prefix the tests with the name testXXX(). Prior to Java 4, there was no support for annotations. During those dark ages, the testing frameworks needed some mechanism to identify the test methods vs the helper methods. The solution was to prefix all test methods with test. It is no longer a requirement and we have more power to choose better names. And as uncle Ben said, “With great power, comes great responsibility”. As a side note, explore the @DisplayName annotation if you are on Junit 5.

Tests should not have any side effects

I think this one should be pretty clear as it is well understood. If our tests are writing some temporary files, make sure to delete them. Basically, leave the system as it was before executing the tests.

Prepare and destroy the data required for the tests

This should be taken care of mostly in case of integration tests. The data required for tests could be some files, DB entries, some mock behavior or anything that is required for the tests to run successfully.

The cleaning up activity can be done as soon as the test is completed or a group of closely related tests is completed. Consider a scenario where there is a test shouldCreateUser that creates a user with the name user1 (not a very intuitive name, but a realistic possibility). This test makes sure the data is deleted after the test is completed. A good developer has done their piece of hard work. If someone creates another test shouldGetUser that is executed before this test and also creates a user with the same name and does not delete it, somewhere down the line, the shouldCreateUser test will fail as the user already exists.

The scenario gets pretty complicated and hard to debug when there are hundreds of tests with various moving pieces. Trust me, I have been there.

Do not use the service code to prepare the data

Consider the following piece of code.

@Test
public void shouldGetUserByName() {
  // arrange
  userService.createUser(new User("UserOne"));

  // act
  User userOne = userService.findByName("UserOne");

  // assert
  assertUser(userOne);
}

This code utilizes the userService to create the test data. It is very important not to do so. The code to prepare the test data should be decoupled from the business code.

Do this instead.

@Test
public void shouldGetUserByName() {
  // arrange
  userHelper.createUser(new User("UserOne"));

  // act
  User userOne = userService.findByName("UserOne");

  // assert
  assertUser(userOne);
}

The userHelper is a helper class for tests. It belong to the test code and is independent of the business code.

One reason is that it would be easy to maintain tests. Any unrelated changes in business code should not be propagated to the unit tests. Let’s say, if we are to modify the createUser() logic, then it should not impact the tests that are doing delete, or get operations.

Another point is that when a bug is introduced in the createUser() logic, all the tests using the service method for setting up test data might fail, giving a scarier picture than reality.

Avoid magic numbers in test assertions

Consider the following example of code.

@Test
public void shouldGetAllUsers() {
  //arange
  userHelper.createUser(new User("userOne"));
  userHelper.createUser(new User("userTwo"));
  userHelper.createUser(new User("userThree"));

  //act
  List<User> usersFromResponse = userService.findAll();

  //assert
  assertEquals(usersFromResponse.size(), 3);
}

Here, the number 3 really does not tell anything meaningful. Why should there be only 3 users? We should strive to make tests as much clear as possible.

Consider this example instead.

@Test
public void shouldGetAllUsers() {
  //arange
  userHelper.createUser(new User("userOne"));
  userHelper.createUser(new User("userTwo"));
  userHelper.createUser(new User("userThree"));

  //act
  List<User> usersFromResponse = userService.findAll();

  //assert
  List<User> usersFromDb = userHelper.findAllUsers();
  assertEquals(usersFromResponse.size(), usersFromDb.size()));
}

Just looking at the assert part itself we come to know that the list of users obtained from the service must match the one present in the database. It’s so intuitive.

(Java-specific) Reset the shared mocks after each test or a group of closely related tests

Now this one is kind of repetitive as we have already discussed to clear the test data. But this one, the one right here, yes this one, needs special attention. If not done correctly, it can lead to wonderfully hidden bugs in a large test suite of a complex application. I have personally spent several hours under frustration and have helped several developers fix these issues.

The best part of such issues are they might not appear when we run a single test or a test file. And they show up only the entire suite is run.

Do not duplicate test code

Maintenance of tests should not become a nightmare. A good start to prevent this is to make sure that the code is not duplicated. Test suites must be organized properly, common pieces must be extracted and reused. Tests should not be treated as second class citizens.

If we find ourselves modifying small things here and there every time because of some single change in business code then it should serve as a red signal.

Avoid hard timeouts in tests

Try to avoid hard timeouts. This might unnecessary delay the test execution or even unnecessary test failures.

@Test
public void shouldMaintainOrderWhenMultipleRequestsArrive() throws InterruptedException {
  List<Thread> workerThreads = createWorkers(5);
  startAllThreads(workerThreads);
  Thread.sleep(5000); // all workers finish within 5
}

The above code is error prone. What if the workers are optimized and now only takes a couple of seconds? What if new processing is introduced and it takes way above 5 seconds now? How do we decide how much time to wait in any case?

A better approach might be to wait() for threads to finish or a similar alternate mechanism.

The test suite should be able to run anywhere

In some cases, this might be true for integration tests and even more so for unit tests. The test should not need any kind of external setup to start running. For example, Java-based unit tests must be able to run wherever a JVM can be installed. All the setup must be done by tests themselves.

Test execution should generate meaningful reports

All of the modern test suite runners have this ability. Many of those generate an HTML report that can be viewed from any browser. Many of the test platforms allow ability to configure plugins to generate reports.

Leave a Reply

Your email address will not be published. Required fields are marked *