Just like production code, test code needs to be rigorously examined to ensure it's clean and bug free. In this first half of a two-part article, Klaus Berg makes the case for why good unit tests are as important as high-quality production code, then provides a comprehensive listing of agile tools and best practices used to improve the internal quality of test code. Level: Intermediate
Code quality is an important topic, but the definition of quality varies. According to the ISO 9126 standard, you can distinguish between internal quality, external quality, and quality in use. In the past, most companies looked only at production code quality, but that shouldn't be the end of your quest for better code. Writing good unit tests (GUTs) is just as important as writing high-quality code.
In this two-part article you'll get an overview of software quality -- how we define it and improve it -- followed by an introduction to methods, tools, and best practices for writing good unit test code. Some of the advice can even be applied to integration and system tests that are written as Java test programs.
Software quality: How to define it, how to improve it
When you talk about software quality, you need to consider both functional and non-functional aspects as defined, for example, in the ISO 9126 standard. These quality attributes (maintainability, portability, and so on) can be measured in two ways:
* Using internal metrics, as covered by ISO 9126 Part 3, Internal Quality. This is typically done by static testing and analysis.
* Using external metrics, as covered by ISO 9126 Part 2, External Quality. This is typically done by dynamic testing.
An internal metric (the focus of this article) is a quantitative method that can be used for measuring an attribute or characteristic of a software product, derived either directly or indirectly from the product itself. (It is not derived from measures of the behavior of the system -- that is, from a test execution.) Internal metrics are applicable to nonexecutable software products during design and coding in early stages of the development process. As Martijn de Vrieze states in his QA-themed blog:
External quality is that which can be seen by customers and which is traditionally tested. Bad external quality is what can be seen: system crashes, unexpected behavior, data corruption, slow performance. Internal quality is the hidden part of the iceberg, i.e. program structure, coding practices, maintainability, and domain expertise. Bad internal quality will result in lost development time; fixes are likely to introduce new problems and therefore require lengthy retesting. From a business point of view, this will invariably result in loss of competitiveness and reputation. External quality is a symptom whereas the root problem is internal quality.
Having read this, a question should occur to you: What do you know about the internal quality of your unit tests? Probably not as much as you know about the quality of your production code. Of course, agile software development processes like extreme programming or Scrum emphasize code-unit testing (preferably before it's written, thanks to test-driven development, or TDD) and thorough testing of software functionality as well as code refactoring steps. (Yael Dubinsky and Orit Hazzan have an excellent paper on this.) However, internal code quality assurance activities for the most part only address production code. It's time to boost the quality of your test code to the same level as the quality of your production code!
In the next sections I'll provide guidelines and best practices for writing good unit tests, both from a general process-oriented viewpoint and from a more technical angle. I'll demonstrate how to measure test coverage for a system under test, and I'll also put these coverage results in a global test context, because that metric is over-interpreted. In the second half of the article you'll learn how to apply those principles to the art of writing good assertions and exception checking test code.
From 'test infected' to 'test obsessed'
In "JUnit test infected: Programmers love writing tests" (Java Report, 1998), Kent Beck and Erich Gamma introduced a testing style called test infection. The goal of a test infected programmer or team is to write a unit test for every class in your system. But if you are a test developer (TD), strongly committed to the philosophies of extreme programming and TDD, you could end up with a test obsession! What are test developers? As Steve Rowe explains in his blog:
Test developers are the heart of a modern test team. There was a day when you could get away with hiring a few people to just use the product and call that a test team. This is no longer the case. Products are becoming more complex. The lifespan of products is increasing. More products are being created for developers instead of end users. These have no UI to interact with, so simple exploratory testing is insufficient. To test complex products, especially over an extended lifespan, the only viable solution is test automation. When the product is an API [library or framework] instead of a user interface, testing it requires programming. Test developers are programmers who happen to work on a test team. It is their job to write software which executes other software and verifies the results.
Because unit test classes don't go into production, you may be disinclined to put a lot of time, effort, and thought into them. But the unfortunate reality is that unit tests are rarely write-only code. Badly written or interdependent unit tests make it more difficult to refactor or add new code to the system. Therefore, a TD should not treat unit test code as a second-class citizen -- and a real test-obsessed TD would never do so. But what are the characteristics of good unit tests?
