In an earlier article I suggested that a good way of measuring project progress is to count the number of passing tests and compare these to the estimated number of planned test cases. The advantage of measuring progress in this way was that test cases are easy to count and estimate and that this approach is more accurate than trying to predict the remaining number of hour’s development will take. A test case either passes or it doesn't - you don't get stuck with a development task that stays in the never-ending 90% done state.
In this article I am going to describe a method for estimating the planned number of unit tests and describe the benefits that you get from doing so.
Testers are pretty good at estimating the number of test cases that something will take because they are used to working this way - but how do you get developers to estimate the number of unit tests? It's like asking an artist to estimate the number of colours that it takes to paint a picture isn't it?
The technique we use is quite simple - mind mapping. If you are not familiar with mind-maps, a mind map is simply a web of words extending out from some central idea. Each node on the web represents a new idea that can also have child nodes of its own. The further out from the central idea/topic you travel the more specialised or fine grained the idea/topic becomes. There are some very good free (Java based) tools out there, for example FreeMind and XMind.
The process work like this; before they start coding you get the developers to sit down and draw a mind map of the unit tests that they will write for the next bit of coding that they will be doing. You then read through the mind-map to count the number of tests and increase this number by a few more (say 10%) for the extra ones you’ll probably also discover that you need once coding begins.
Use case or User Story, it doesn't really matter as the technique can be used in lots of different situations. It also doesn't really matter too much how the mind map is laid out, you can be quite flexible with the styles and graphical design of the mind map. Since it depends a lot on the thinking processes of a person and each person will tackle the problem in a slightly different way. The mind-map provides the high-level list of test cases that the developer will write during construction.
For example, here is a mind map about mixing Cocktails that has three test cases.
Figure 1: Example Unit Test Mind Map
Create a mind map and count the number of test cases that pop out. For something that will take a developer about three weeks of coding effort writing a mind-map before they start will typically only take one or two hours of time, assuming that that they've read all of the requirements and have finished any design work.
Mind Maps are great for…
Once you have a mind map is when things really start to get interesting. Once a developer has produced a mind-map you can;
• Test any designs or Service and GUI interfaces
• Talk to the testers
• Detect scope creep
• See the big picture
• Control how long coding will take
• Track development progress
Test the design
Even before any code has been written the mind map tests the design. If you have class diagrams, sequence diagrams or even other things like web service interfaces and GUI wireframes then a mind map can be used to think about whether or not design is good enough. By reviewing the mind-map you often find that a design of service needs a new method or parameters.
Talk to the testers
Once a developer has a mind map of their planned test cases they can then have a conversation with a relevant tester who may have their own mind map or list of planned test cases. The developer and tester can then compare each other's list of test cases and talk about what is missing from each list building up a better understanding of what is actually required. You often get situations where the tester will think that the code should work one way and the developer thinks it should work a different way. If you can catch and resolve these situations before anything has been coded then you are going to save time and effort during construction.
Detect Scope creep
Mind maps also help detect scope creep before it occurs. One source of scope creep is from the project team members themselves. Developers and Testers will sometimes expand or invent features beyond what the client originally intended. Generally this isn’t intentional or malicious, it tends to be more in the case of “Wouldn’t it be good if it did this...”. Specifying an initial set of test cases will help provide visibility about what a developer (or tester) thinks should be done before they actually do it, and it’s too late.
Technical leads can review these mind maps and prevent certain situations of scope creep from occurring.
See the big picture
If a developer has a list of planned test cases then the list can act as a To Do list and so help the developer becoming too bogged down in one area of their coding. People like to do the best work that they can, but sometimes and particularly with unit testing they can focus too much on one area and run out of time to make sure there is enough overall coverage.
The usual approach to test design involves looking at some number of independent variables and then creating tests based on the various combinations of input data. I'm not really suggesting that this is a bad approach, but what I am suggesting is that this sort of approach generates a large number of overlapping test cases that might be technically independent of one another but in practice you probably only needed half of the various combinations to achieve a "good enough" overall result.
Control how long coding takes
Unless a developer has the luxury of unlimited time to complete a task then having the mind map of planned test cases helps to remind them of the total amount of work that they must complete in the budgeted time-frame. So to a certain extent you can control how long coding will take because you can review the mind-map and decide which tests will be coded and which tests will be discarded. You might choose to discard certain tests because sufficient coverage will be provided by other automated, or by manual testing.
One goal I like to set developers is to write the smallest number of most useful test cases. To take a real world example a Bridge Engineer doesn't design the absolute strongest bridge possible capable of withstanding the strongest ever known earthquake at the same time that the bridge is also hit by a ocean going oil tanker. The Bridge Engineer designs the cheapest bridge they can that still passes the minimum set of requirements. With programming you can't write every possible test case anyway so you will only implement a subset of the possible test cases. Mind maps are a cost effective way of picking the most useful test cases to implement.
Track development progress
Mind-maps are also useful in tracking the progress of construction. Given the total number of planned test cases you can measure the progress of construction by counting the test cases that pass without failure and objectively calculate the completion percentage for your project.
We have written a build metric tool that does this automatically called Trace Analyst. It works by storing the total number of planned test cases and then analysing JUnit report files to count the number of working test cases. Trace Analyst works from either Ant or Maven and by integrating it with a continuous integration server like Bamboo or Hudson you get real-time reporting on project completion status. So for example in the following screenshot you can see that this project has a 50% completion status for Unit Testing.
Mind maps for planning unit tests are a cost effective technique for improving your software development methodology. They provide early detection of design flaws or scope creep and can also be used to help manage the coding phase of your project.