I've been a zone leader with DZone since 2008, and I'm crazy about community. Every day I get to work with the best that JavaScript, HTML5, Android and iOS has to offer, creating apps that truly make at difference, as principal front-end architect at Avego. James is a DZone Zone Leader and has posted 639 posts at DZone. You can read more from them at their website. View Full User Profile

I Don't Write Unit Tests Because.... : The Excuses

08.12.2010
| 16364 views |
  • submit to reddit

As someone who's seen the benefits of the approach, I'm a huge believer in test driven development. It adds a level of quality and maturity to the field of software development, yet it's still not a widespread practice across development projects. When it comes to a choice between the features, time and quality, it's always the quality that suffers. We don't want to add extra time for testing and we don't want to compromise on the feature set of the delivery. If you haven't set out to do test driven development at the start of the phase, then it's difficult to fit in.

We've all heard excuses for not taking the test driven approach, but nowhere compiles them better than "Pragmatic Unit Testing in Java With JUnit", from the Pragmatic Bookshelf.  I read the book a few years ago, and afterward I thought there was no way that any responsible developer could read the book without truly believing that unit tests were one of the most important aspects of the development activity. 

The most worrying excuse I've heard is that it's too difficult to test my code. This can be for one of two reasons. One is that your code is mainly UI related, and automating the UI tests is too difficult. I'll concede that UI automation is a tricky area (but not impossible, as we'll see in a later article), but every effort should be made to automate it if possible: think about regression tests. If you have a fully automated test suite, including the UI behaviour, you can make a change to your code, and have full confidence that you haven't broken anything if you have tests to re-run.

The second reason that your code is too difficult to test is that you've messed up your design. Maybe your logic and UI code are too tightly coupled, and without that automated UI layer existing, the dependency between the logic and the UI will cause a problem. This is where test driven development helps to ensure a clean design, following best practices. Using JUnit is simple, so if I can just point my test at a clean logic layer, I can test all logic that the UI will ever touch.  What's that? You're missing your data model? Well then just use mock objects - there are many frameworks for that!

For those who haven't read the book yet, I'll give a quick summary of the excuses for not testing as outlined in the introduction chapter:

  • It takes too much time to write tests
    As the number one reason that developers give, and thinking back on my own education, I think this is down to the way computer science is taught. For the most part, we're lead to believe that testing happens at the end of the process, not all the way through.

    Pragmatic Unit Testing encourage the "pay as you go" model, where having tests written as you are developing means that you don't have that crunch time at the end where you try to cram all your unit tests in.

    Anyway, if you're not writing tests as you go along, how are you checking if the code behaves as you expect? You run it manually? Wouldn't it make sense to invest some of that manual testing time into writing a JUnit test for it? I mean you'll have to run that piece of code again sometime - it's not going to remain untouched for the lifetime of the product.

    In fact, there are many more time penalties that happen if you don't write unit tests. You'll have to spend time debugging code trying to work out why something doesn't work. Or you'll refactor the code, and then find that nothing works as it used to after the refactor. If you had the unit tests written, you have a base of confidence.
  • It takes too long to run the tests
    I'll admit, there can be a good amount of time taken running tests that involve external entities, or the user interface. But you're supposed to have more than one level of test. The unit level tests should run quickly, and there may be a level of integration tests that can run without a big time impact. If there are higher level integration tests that are taking time, just run them less frequently - maybe even in a nightly build. That's not to say you should ignore the lower level unit tests. These should always run fast enough that it becomes a natural part of your own development process.
  • It's not my job to test code
    I don't know at what stage a software developer decides that he can just throw code over the wall. If your job title was simple Coder than maybe there's an excuse. But as your job is to develop working software, you need to be able to say that your code is functional.  The book makes a good point that if the QA development finds it difficult to find bugs in your code, it will do wonders for your reputation.
  • I don't really know how the code is supposed to behave so I can't test it
    This is a difficult one to react to without incredulity. If you don't know how the code is supposed to behave, you shouldn't have started writing it. You need to understand the requirements first.
There are a few more excuses listed in the book, but these are the main ones. What are the excuses that you've heard for not writing tests.
Tags:

Comments

Sadsds Dsadsadsa replied on Thu, 2010/08/12 - 7:11am

Good article overall. Can't disagree with the sentiments.

 Should point out that there a lot of people who believe in writing unit tests but NOT doing TDD. They find their unit tests less than optimal (one guy I worked with even disabled the asserts in one of the tests because the code was too tightly coupled!) but still don't see how TDD would be a better approach.

 R

Josh Berry replied on Thu, 2010/08/12 - 7:42am

I would add that the number one hurdle in all test writing endeavers is that most developers do not consider how their code will be tested when writing it.  This usually leads to a fair bit of coupling between all layers of their system, such that adding tests in later is less than pleasant.

 

Of course, I could just be projecting on to everyone else. :)

Ronald Miura replied on Thu, 2010/08/12 - 8:21am

My usual excuse is: "My app is not that complex, it's just a bunch of tables and some screens that show and modify some data, without much so called business logic. What should I test? HQL queries? The generated HTML? Or should I create artificial layers and abstractions, just to test the connections between them?" Is it a valid excuse? :)

Robert Lauer replied on Fri, 2010/08/13 - 1:47am

The worst excuse I've heard, stated by an architect of a competitor, was: "Tests are a mortgage - you keep paying for them". I thought about this, and it makes sense on superficial inspection - you do have to maintain the test over the lifetime of the product. But when you start thinking about it: Why did you write the test in the first place? You wanted to verify that a certain feature works as expected. You do want to ensure this by regularly running this test. The alternative would be to (manually) check the feature on a regular basis. The interesting point though: You are paying a "mortgage" no matter if you verify that feature's correctness manually or automatically. Ultimately, the mortgage is the customer requirement to have that feature in the product. The customer is paying for new features as well as maintained features!

Mario Fusco replied on Thu, 2010/08/12 - 11:54am

Don't understand me wrong: even if I am not a big fan of TDD I believe unit-test are mandatory for ALL projects. BTW, No: "my app is not that complex" is not a valid excuse.

My only concern is that sometimes I feel I have to break the encapsulation or information hiding of my objects (e.g. make public a method that should be private) in order to test them.

Does somebody else have the same problem? Do you have any suggestion to workaroud it?

Ronald Miura replied on Thu, 2010/08/12 - 12:19pm in response to: Mario Fusco

Ok, I agree it's a lame excuse :)

But, "What should I test? HQL queries? The generated HTML? Or should I create artificial layers and abstractions, just to test the connections between them?" It isn't a valid excuse, but I think it's a valid question.

BTW, you could make your methods package-visible instead of public if the tests for that class are in the same package (in a separate '/test' source tree, please). There is that other school of thought that says that only the public interface of an object should be tested, since that's what should matter. If that is the case, but your method is so complex that you want to test it in isolation, you could refactor it to another class, make it public there, and add it as a dependency of the original class.

Bruce Wallace replied on Thu, 2010/08/12 - 12:43pm

My recent blog post "Is Morality Eating Your Own Dogfood?" on ExistentialProgramming.com (soon to be an article here on JavaLobby?) injects some ideas from from Philosophy and Economics into this topic (i.e. should programmers write their own tests?).


Chris Wash replied on Thu, 2010/08/12 - 1:24pm

I agree with the spirit of your argument, but it's very uncompelling.  I would recommend looking at Growing Object Oriented Software, Guided By Tests by Nat Pryce and Steve Freeman.  Their gardening metaphor is the best way to "sell" testing to others.

 When you're trying to promote the use of testing to someone who says "it's too hard" then "well just use mock objects!" is not a valid response.  I like where you're going with this, but continue to dig deeper.

Pooria Mellati replied on Thu, 2010/08/12 - 1:29pm

Mario Fusco,

Just make those "special" methods protected, and put your tests in the same package (but a different folder). This way you'll be able to test them without making them public.

Mario Fusco replied on Thu, 2010/08/12 - 2:17pm in response to: Pooria Mellati

I wrote "public" just to keep the question short. I already know this technique and indeed I use it quite often. But anyway I don't like it. Protected (or package protected) has a different meaning and it should be used only with that meaning. If I am making protected a method that should be private only at the purpose to test it I think I am breaking its encapsulation as well.

Mladen Girazovski replied on Thu, 2010/08/12 - 4:08pm in response to: Mario Fusco

Mario, i had this Problem when is started wirting unit tests. I didn't start with TDD, so my Classes didn't have an easily testable Design, also i wanted to test private Methods, while we should test the public methods.I also find that using explicit Interfaces, the implementing Class could have more Public setter without polluting the Interface. By extracting classes out of bigger classes you'll get you chance to test public methods and using Mocks gets easier & more productive.

The thing is, testing does get easier with TDD :)

However, TDD s hard to learn  if you're already used to the non-TDD way of programming.

 

David Parks replied on Thu, 2010/08/12 - 5:08pm

This is one of my favorite holy wars.  I agree with many of your points but that may be because you are fighting the freshman arguments against TDD.  I like both TDD and testing.  A lot.  But the rules for when are just a bit more complex than "always".  Here's some more professional arguments against both TDD and testing.  I initially discarded all of them, then learned how wrong I was, and finally accepted that there is no simple replacement for thinking about what you are doing instead of blindly doing it.

1. "I have a large, fairly heavily coupled system". 

This is a blend of "code is too hard to test", "I don't know how it works" and "It takes too much time" from your article.  Your answers were, "You screwed it up, suck it up and test anyway, you'll be better in the long run". 

"You screwed up": Monolithic code bases of poor quality never have one creator.  Indeed, its *very* common for the person working on the code to have no relationship to its history.  Yes, it was screwed up.  But punishment doesn't need to be a part of code development.  It rarely fixes the bug.

"Test anyway": Some times (less than half I'd guess) this makes sense.  Most of the time you have just killed your project by agreeing to a large delay that, when over, will have you back where you started (but with a cleaner code base).  

"You'll be better in the long run": Sort of.  Of course, refactoring for testing should be done incrementally.  Of course, this is a colossal risk since refactoring regularly introduces bugs.  Not a huge deal if you have good tests but... umm... clearly you didn't!  I'm now pretty far from the 'better in the long run' feeling.  I'm almost at the "You'll end up homeless and destitute" side.

2. "My language / build-framework / code base is heavily against testing."

You mention something like this with UI testing.  I'd go further than UIs... anything with output that is channeled through an unknown API (like a UI or 3D graphics or audio driver implementation).  A step past that is languages and frameworks that make testing hard.  If you are working on a C project then there are quite a few testing frameworks available.  If you are used to the time saved with simple jUnit usage, you'll be stunned by the time sink that is C unit testing.  It's certainly doable but if you add one more minor issue (like a cranky build system) then you suddenly have a nightmare scenario.  And remember again, if it doesn't save time or improve quality (two things that are hard to achieve with *added* complexity) then you shouldn't do it.  Even if you read on some smart dude's blog that you should.

3. "You have tests.  They are bad."

Bad tests, like bad client code, can *cause* needless coupling.  Often (incorrectly written) unit tests don't test a unit at all -- they cover a huge swath of the code base through statics or whatever.  Of course, you delete these but you could be left with few to no tests... putting you back at #1. But more likely, the guilt over potentially not having tests will lead you to keep them, making life harder.  Worse, you may not realize what the test's real problem is.  Again, its a bad test so for sure *you* didn't write it!

 

So... do what saves time and energy.  Some of the arguments for what saves time via TDD are complex. Know those arguments.  But when you have a simple reason for why TDD is not right for a project, if that reason isn't satisfactorily handled by the arguments for TDD, then don't do it!  Trust your own brain first.  We aren't running a religion here.

- David

Stephane Vaucher replied on Thu, 2010/08/12 - 6:59pm in response to: Mario Fusco

@ Mario - I've got to say that mostly I focus on testing my public interface. If I really need to test private methods, I use package protection. In fact, that is the only case when I use package visibility. Hence, the meaning in my code context is that I want to test something. You can also use reflection (or http://privaccessor.sourceforge.net/).
The issue you have is that you want for elegant code (as you do not want to break encapsulation). However, you need to decide if:
  1. The code is well designed if you cannot test it: you might need to redesign
  2. if it is well designed, but you cannot test, then you have a problem... you need to choose between correctness and elegance. I would choose correctness and change visibility (package protected).
  3. If you still do not want to change visibility, use reflection. It's not elegant, but at least your code will look the way you want.

Kris Karczmarczyk replied on Fri, 2010/08/13 - 2:32am

In response to Mario's question about testing private.

You need to ask a question why you want to test private? If this is very complex and important code why it's not a separate class then? Also there should be some public method using your private right? Why not testing this public method.

Over time I learned that common mistake people do that results in further creating opinions that 'tests are an expensive mortgage you have to pay' is that many times developers focus on testing the implementation details instead of the external behavior of the class. That may be extremely expensive as very little change of the tested code may result in broken tests even though the external behavior stays intact.

Cheers,

Kris

 

 

Ronald Miura replied on Fri, 2010/08/13 - 5:45am in response to: Stephane Vaucher

if it is well designed, but you cannot test, then you have a problem... you need to choose between correctness and elegance.

So if it is well designed and elegant, but hard to test, it implies that it is not correct?

John Wheeler replied on Fri, 2010/08/13 - 12:25pm

The process doesn't prohibit anyone from writing shit code; it just gets in the way. How am I supposed to work with some hipster that only wants to write tests because he's on Martin Fowler's jock? Doesn't he realize that, for EVERY FUCKING CHANGE we make to the code base, we'll have to update tests that really don't catch bugs anyway? I mean really, how many bugs have your unit tests caught in the last year? If it's less than 10, you're not getting value out of the unit tests. If it is more than 10, you probably write shitty code. And, what's with these frameworks that automatically generate tests for code you're not really supposed to fuck with anyway? For example, automatic tests that Spring ROO generates that are going to pass. Do you really need that cruft littered around your code base? I work on a team where the guys just know what they are doing. They don't need so called 'confidence' that unit testing provides. We don't chime in with the rest of the audience and laugh at the conference speaker lamenting about buggy code and garnering sympathy. We don't write buggy code. Really. Because we are good at what we do as it is our craft. We don't need your stinking unit tests.

Josh Berry replied on Fri, 2010/08/13 - 1:23pm in response to: Mario Fusco

Missed this the other day.  Since I usually place the test for a class in the same package, I have elevated some to package protected instead of private, but never all the way to public.

 

In general, I have seen it argued recently that if you have some behavior you want to test in your object that can not be triggered by external calls, you should abstract out that piece into something your current code uses, such that you can test it.  So, you wind up with less private stuff, and more public classes that anybody can use.  (And you can test.)

Loren Kratzke replied on Fri, 2010/08/13 - 6:13pm in response to: Kris Karczmarczyk

This can not be overstated: Test function, not implementation. Only a bonehead would want to directly test private methods. Said bonehead would then have to fix their tests every time code changes even if the function of the code executes identically.

Loren Kratzke replied on Fri, 2010/08/13 - 6:31pm

Regarding this article - I am quite frankly offended by TDD bubble heads when they spout pretentious BS like "We've all heard excuses for not taking the test driven approach...". Excuses? Really? I need an EXCUSE not to follow TDD? Really? Wow. Really?

Then they follow this with pathetic whiney speech fragments usually heard from 5 year olds when asked to clean their room - contrived solely to belittle any reader who does not fully agree with them (e.g. those readers not already in the choir).

I have never seen one spec of scientific evidence that TDD is anything more than a self gratifying massively spastic fit of mental masturbation.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.