Why Evangelising Unit Testing Needs to Stop

After reading Is the popularity of unit tests waning? I thought about my own evangelism of unit testing. The practice of unit testing is well established in most languages and yet many Java programmers I have worked with don't do it.

When Java was first released and C++ challenged it was clear what class of problems that Java made go away. The garbage collector and lack of pointers removed two major problems that every C/C++ programmer had come across, errors with pointer arithmetic and the fail to free memory. Since both of these bugs where painful to track down something that removed them completely was a good idea. Even developers early in their careers suffered these problems and Java as an offering was obviously solving the problem. Indeed the same is true today comparing Ruby on Rails to J2EE development, where the differences in line count and productivity are night and day. Of course just like the rise of Java performance suffers somewhat eith Rails and there are things you can't do in Rails. The same was true when Java came out and some wondered how we would get by without using the Windows API and pointers. The value proposition of languages and many frameworks are obvious.

Part of Java's success was its copying of C like syntax. Many technologies are built to make a better mouse trap, language churn and frameworks are good examples. The subtle implications come much later after using the new tool for a while and new problems appear. Modern unit testing builds on a long history of automated tests going back at least as far as Smalltalk, its not so much that its a new technique or technology. The modern unit testing frameworks for Rails are better than basic JUnit but they are still pretty basic. I think this is a C syntax like wooing where they try to keep the framework as simple as possible so that writing the tests is very familiar.

It seems that only 20% of developers are convinced by the vague benefits of unit testing. Of course those which are convinced are also the guys writing in Ruby, looking at Scheme and generally building the modern web. The other 80% of developers don't seem to be feeling the unit testing vibe. I think that is because the class of problems that it solves is not obvious. Indeed its something that can apparently can be solved with quality assurance teams and every development team has those! Passing the work of testing to the testers is much easier and less work for a busy developer.

Worse still is the cost of unit testing. The obvious cost of adding extra code is easy to see, but also there is the cost of designing and implementing the code in a different decoupled way. Adding mocks and other nastiness only adds to the ick feeling of writing unit tests. This can add complexity and certainly adds to development time. It can be argued that decoupled code is better but then when was anything other than working software the goal important? Most systems descend into untouchable code anyway and management won't allow the developers to change anything for fear it will all unravel or add significant bugs and worse require a complete regression test, which would take months. I have seen countless clients throw away old systems and start rewriting because they could not keep adding functionality any more. The code had rotted beyond what could be maintained and the testing effort had gone up linearly with the amount of functionality and so the cycle starts again.

Unit tests are just like JavaDoc, we all know they should be written and are "important". But just like JavaDoc there is no obvious benefit to an individual developer. Sure I know that someone trying to read the code later would benefit from JavaDoc and unit tests will catch other peoples bugs in the future. Why would I want to help the idiot in the future that touched my code. Certainly doubling the effort to writing the code to add these benefits seems like a really high price to be paying.

Yet evangelism is still strong and popularity with good developers still high. Of all the good developers I have had the fortune of working with all of them but one wrote good unit tests. The one exception almost proved the rule as he wrote great integration tests, he was just rubbish at unit tests. Every developer I've met which sucked (about 85%) didn't write unit tests. I have unfortunately met some useless developers who claim to write tests, although they are very poor at it.

Whether unit testing really helps write better code is hard to say. The costs are obvious but the benefits are softer than most evangelists would like to admit, productivity remains unmeasurable. Any practice that adds lines of code and yet claims to reduce overall effort is going to struggle to gain acceptance. Unit testing has become the required documentation of the future, just as UML died a death unit testing will have to go the same way. That doesn't mean its no use, it simply represents the best solution we've found to a class of problems we've so far failed to find the best solution to. Just as Java (nearly) killed memory leaks so will something in the future kill the need for unit tests. In the mean time the best way to reduce the class of bugs that unit testing fixes is to write unit tests.

The evangelising of unit tests to some extent has to stop. Right now its hit the perfect point of identifying the top 20% of developers and making the other 80% easy to spot. Indeed I think such a practice is unlikely to really break into the mainstream any time soon due to its weak proposition. So if all the evangelism could just go away that would be just perfect.