DevOps Zone is brought to you in partnership with:

Jens Schauder is software developer since 1997. He loves software development for the constant challenges and constantly changing environment. A great chance to learn and teach. He is also blogger, author of various articles and speaker at conferences. Jens is a DZone MVB and is not an employee of DZone and has posted 86 posts at DZone. You can read more from them at their website. View Full User Profile

The Cost of Testing or You don’t Know what You are Missing

  • submit to reddit

Twice this week people challenged my belief in automated testing and Test Driven Development. The argument that was brought forward was simply: Unit tests are too expensive.

I actually believe the opposite is true. Let me tell you about a little dialog that happened in my development team after two developer where working on a feature for about an hour or two.

First Developer: “I think we got everything. Lets start the application and see if it works.” (starts the application)

Second Developer: “Hmm, doesn’t work.”

First: “Oh, I know, we forget to actually invoke the new feature” (fixes that and restarts the application)

Second: “Looks great. It works. Lets commit it to SVN”

What’s so special about this? They didn’t start the application a single time during the development process for over an hour. Can you do that without writing tests? Of course you can and you will spend restarting the application during the rest of the day until you fixed all the bugs. How long does your application need to restart? Including the time you need to navigate to the state where you can test your application. How long does that take?

Not having to do that. Or at least not very often is a huge time saver. And it’s not only the direct cost of restarting the application. Waiting for the application is often an interruption of your concentration. Do you really think about the next aspect of the feature to implement? Or do you check mail? Get a coffee? In my experience going Test Driven in Pairs is a way more concentrated type of working.

And that is not even considering that your tests will run over and over again, making sure later changes don’t break anything. This alone saved my butt a couple of times.

If you reached that level of TDD it will save money for you and your team. I promise.

But I’ll admit, I didn’t believed that a couple of years ago. I was using tests only when implementing rather abstract stuff. And I thought it wouldn’t be possible to efficiently automate tests for most of the code we wrote. I had to learn a lot. Testing isn’t easy. Most of the code we produce isn’t fizz buzz, but is concerned with GUIs and databases, both make testing hard and expensive. But these problems can be solved, so you can gain the benefit of TDD even for these cases.

And as a side effect you will have to come up with a really nice and clean design of your code in order to test it. This in itself will make future changes more easy and cheap.

Therefore I urge you: Start learning how to test today. If you don’t you will never know what you are missing.

And if promises don’t motivate you: I’m convinced that in a couple of years from now the places where you can just puke code into your IDE will get rare. So if you still have a couple of years to cover until retirement: Beef up your testing skills. It’s one of the best investments you can make in your own future as a software developer.



Published at DZone with permission of Jens Schauder, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)



darryl west replied on Mon, 2011/07/25 - 11:54pm

I agree: "...Start learning how to test today. If you don’t you will never know what you are missing." One step further--start testing today, or get out of the business. It's never faster to do things the wrong way--it just seems that way at the time.

Ryan Lea replied on Tue, 2011/07/26 - 1:25am

I still remember the first time I went through the process those developers went through. Was both cool and scary to think that you can actually deliver working features with such a small number of application restarts.

Jilles Van Gurp replied on Tue, 2011/07/26 - 1:48am

TDD is a bit of a religion where you just got to have faith (as opposed to empirical evidence). Great software gets written using TDD. Great software has been written without it as well (actually the vast majority of software you depend on in your daily life). And I've seen some mediocre teams waste shitloads of time writing poor tests and poor code. I've also seen situations where the code was fine but the tests were a major obstacle to refactoring/adapting it. Or worse, where the tests were hopelessly tied to the poor design of the code making both needlessly hard to fix.

I tend to be a bit more pragmatic. Writing new code is a process where things are highly fluid and change from minute to minute. I find TDD slows me down unacceptably in that kind of situation.I basically doubles the amount of code I need to worry about. So I often code first; write a few blackbox style integration tests to verify I'm on the right track and once I have that confidence expand the unit test coverage to cover at least the non trivial parts of code.



Ronald Miura replied on Tue, 2011/07/26 - 5:58am in response to: Jilles Van Gurp

@Jilles +1

Nicolas Bousquet replied on Tue, 2011/07/26 - 7:20am

If there one thing we learned from the past 50 years in software developpement is that there is no silver bullet.

XML, OOP, TDD, Waterfall, Agile, Scrum... All were suposed to give huge performance gains to develop software.

This is the hype. But the conclusions from studies tends to differs. Agile techniques seems to cause similar final cost than Waterfall. OOP is wel suited to some problems but not so good for others and with it as a monoteist religion we add lot of complexity to many problems (ORMs being a good polemical example).

TDD? I will say it highly depend of the context. This include how easy is it to run your code anyway, if you already have or no a testing infrastructure or the size/complexity of your software.

From my experience on huge software with lot of legacy code, when you are on maintenance (correcting bugs) and don't master the code (and you never master the 14 millions lines of code), TDD is useless. In fact your best tool is a good debugger that allow you to see how the thousand of configurations parameters and external inputs combine to create this special case. This is because most bugs are integration problem, or more a combinations of lot things in an unsuspected way. Something that is very difficult to test automatically. The problem finally is that unit test test what you think you should do and most bug exist because you shouldn't do that in the first place anyway.

I agree that when you add new functionnality, if you manage to put them in a centralised place (like a few new classes), it is fairly easy to add unit tests.Theses test still need to be written and updated. How many time do you change a test just because of a refactoring or something is still valid but retrieved in a different order?

Unit test roughly double the amount of code to write and maintain Counting that code complexity is exponential with code size and that you find only the most obvious bug with unit tests, this maybe not the best tradoff.

For huge system like where I work, I believe far more in integration tests (manual or not) than unit tests. And TDD is mostly demonstrated on stupid example like testing a pure immutable mathematical function.

Jose Maria Arranz replied on Tue, 2011/07/26 - 2:41pm

Nice article Jens

Nice comments Jilles and Nicolas, plenty of common sense.


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.