As an Agile Coach, Miško is responsible for teaching his co-workers to maintain the highest level of automated testing culture, allowing frequent releases of applications with high quality. He is very involved in Open Source community and an author of several open source projects. Recently his interest in Test Driven Developement turned into http://TestabilityExplorer.org with which he hopes will change the testing culture of the open source community. Misko is a DZone MVB and is not an employee of DZone and has posted 38 posts at DZone. You can read more from them at their website. View Full User Profile

Why are We Embarrassed to Admit That We Don’t Know How to Write Tests?

07.08.2009
| 5105 views |
  • submit to reddit

Take your average developer and ask “do you know language/technology X?” None of us will feel any shame in admitting that we do not know X. After all there are so many languages, frameworks and technologies, how could you know them all? But what if X is writing testable code? Somehow we have trouble answering the question “do you know how to write tests?” Everyone says yes, whether or not  we actually know it. It is as if there is some shame in admitting that you don’t know how to write tests.

Now I am not suggesting that people knowingly lie here, it is just that they think there is nothing to it. We think: I know how to write code, I think my code is pretty good, therefore my code is testable!

I personally think that we would do a lot better if we would recognize testability as a skill in its own right. And as such skills are not innate and take years of practice to develop. We could than treat it as any other skill and freely admit that we don’t know it. We could than do something about it. We could offer classes, or other materials to grow our developers, but instead we treat it like breathing. We think that any developer can write testable code.

It took me two years of writing tests first, where I had as much tests as production code, before I started to understand what is the difference between testable and hard to test code. Ask yourself, how long have you been writing tests? What percentage of the code you write is tests?

Here is a question which you can ask to prove my point: “How do you write hard to test code?” I like to ask this question in interviews and most of the time I get silence. Sometimes I get people to say, make things private. Well if visibility is your only problem, I have a RegExp for you which will solve all of your problems. The truth is a lot more complicated, the code is hard to test doe to its structure, not doe to its naming conventions  or visibility. Do you know the answer?

We all start at the same place. When I first heard about testing I immediately thought about writing a framework which will pretend to be a user so that I can put the app through its paces. It is only natural to thing this way. This kind of tests are called end-to-end-tests (or scenario or large tests), and they should be the last kind of tests which you write not the first thing you think of. End-to-end-tests are great for locating wiring bugs but are pretty bad at locating logical bugs. And most of your mistakes are in logical bugs, those are the hard ones to find. I find it a bit amusing that to fight buggy code we write even more complex framework which will pretends to be the user, so now we have even more code to test.

Everyone is in search of some magic test framework, technology, the know-how, which will solve the testing woes. Well I have news for you: there is no such thing. The secret in tests is in writing testable code, not in knowing some magic on testing side. And it certainly is not in some company which will sell you some test automation framework. Let me make this super clear: The secret in testing is in writing testable-code! You need to go after your developers not your test-organization.

Now lets think about this. Most organizations have developers which write code and than a test organization to test it. So let me make sure I understand. There is a group of people which write untestable code and a group which desperately tries to put tests around the untestable code. (Oh and test-group is not allowed to change the production code.) The developers are where the mistakes are made, and testers are the ones who feel the pain. Do you think that the developers have any incentive to change their behavior if they don’t feel the pain of their mistakes? Can the test-organization be effective if they can’t change the production code?

It is so easy to hide behind a “framework” which needs to be built/bought and things will be better. But the root cause is the untestable code, and until we learn to admit that we don’t know how to write testable code, nothing is going to change…

From http://misko.hevery.com

Published at DZone with permission of Misko Hevery, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Tags:

Comments

Artur Biesiadowski replied on Wed, 2009/07/08 - 2:32am

Most of my bugs are multithreading (mostly race conditions, with occasionall deadlock). I have yet to see a way to do the tests which would catch those bugs (or for way to write the code which will allow testing it). So far my only solution was code review by another programmer and running entire application in prelife-like environment with full load and pray bugs will occur sooner rather than later.

One of the examples is simple application which processes around 1 billion events a day, doing basic conversions. For 2-3 events a day it was giving errors with double format. After looking for few days at few hundred lines of java code around this specific place, trying to find errors, we realized later it was rare gc bug in jvm, which was corrupting one byte of memory here and there (sometimes inside double, sometimes inside produced string).

Another (different app, lot more complicated) was deadlock which happened if two events happened withing few hundred cpu cycles of each other, while another rare condition was happening. It has bitten us after 4 months of having the code already in production.

Those are real production problems. If somebody will show me some magic ABCUnitJFramework which can save me from such kind of errors at reasonable cost/effort, I'll be very happy.

 

Mladen Girazovski replied on Wed, 2009/07/08 - 6:41am in response to: Artur Biesiadowski

The magical ABCUnitJFramework you are looking for is... none.

 

If you want to learn about Test-Patterns that heklp you to write Tests for Multithreaded Apps/Components, have a look at "XUnit test Patterns".

IMHO is the problem that you are facing due to the complex problems that you are dealing with, if you have no expirience in Unittests  chances are that you'll get confused and dissapointed.

Andrew Arrigoni replied on Wed, 2009/07/08 - 10:09am

I have an even deeper problem. I write decent code, IMO. My documentation is a little sparse some times but I avoid most of the more heinious WTFs out there. My problem is, however, I don't know how to write good tests, much less how to write testable code.

 Moreover, I have only the internet to teach me and it hasn't shown me much. My company has no testing department. I know I can't test well and I end up catching more bugs that any of the other developers when they do "QA." So, I'm not embarassed to say I don't know how to write tests. I'm embarassed to say I don't know how to find out how to write good tests. :(

Varun Nischal replied on Wed, 2009/07/08 - 1:09pm

Amazing post, I loved every bit of it. Although, I am a person who is just entering the industry. These insights might help in the near future.

Thanks.

Rogerio Liesenfeld replied on Wed, 2009/07/08 - 1:59pm

The secret in tests is in writing testable code.

I agree. But what exactly IS untestable code? All the Java code I can write is fully testable, no matter what combination of Java language keywords and idioms I choose to use in realizing a design. And I mean fully unit-testable, with 100% code and branch coverage.

Obviously, the tools you use for writing the tests do matter. I am happy with the test code I have been writing, but surely it can always be improved... just like the production code.

I think this kind of discussion would be more productive with some real code, in particular the so-called "untestable code".

Franz Allan See replied on Thu, 2009/07/09 - 2:58am

When I first heard about testing I immediately thought about writing a framework which will pretend to be a user so that I can put the app through its paces. It is only natural to thing this way. This kind of tests are called end-to-end-tests (or scenario or large tests), and they should be the last kind of tests which you write not the first thing you think of.

 

I beg to disagree. These end-to-end tests is what you right to make sure that you implemented the feature/behavior that you're trying to implement. These should be written firsts because it is your end goal (more or less).

 After which, you then try to make that end-to-end test pass by implementing the code needed for that. And to do that, you then write your unit tests, and slowly make them pass until your end-to-end test passes.

 

In summary, the sequence of events is something like:

  1. Write end-to-end test for a feature
  2. Write unit test for a chunk of logic that you need to implement as a part of the total implementation of that feature.
  3. Make your unit test pass
  4. Repeat #2 until you've fully implemented the feature which means
  5. Your end-to-end test now passes.

 

Also take note: Although you write your end-to-end test first before your unit tests, your unit tests would pass first before your end-to-end test.

Maxim Gubin replied on Thu, 2009/07/09 - 7:46am

I find really long methods with lots of static calls very hard to test. You gotta mock out or stub out quite a bit to test just the logic you're interested in. And even then, there are so many variables there that you will need to write a whole sleuth of tests just for this method.
Keeping the methods small definitely helps to unit test them better. And creating fixtures, which is basically test data, is the key to having 100% reproducable tests. Integration tests also help but they will not be as reliable as unit tests.
What you referred to as the end-to-end tests are functional tests. For webapps you can run those in a browser etc.
A really good place to learn about testing is books. Check out pragmatic programmers publishers. Www.pragprog.com
they have a few books on the matter.
Also testdriven.com has lots of resources on testing. Good luck! Unit testing should be a priority for anybody. Test driven d

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.