Andrew trained as a biologist before transmuting into a software engineer. He's been coding in Java since 1997, and has done a bit of scientific software, a bit of mobile gaming and lots of enterprise software. He's particularly interested in humane and sustainable development. Professionally, that means code maintainability and agile methods. Outside the professional sphere: permaculture, gardening, cooking, green transport and raising children (not by order of importance). He also loves ensemble singing, but that’s on hold for now. Andrew is a DZone MVB and is not an employee of DZone and has posted 9 posts at DZone. You can read more from them at their website. View Full User Profile

Unit Testing Fundamentals

02.08.2011
| 15890 views |
  • submit to reddit

This article describes the principles of unit testing, rather than the technical details. It's aimed at beginners. Experienced unit-testers will get little from it, except perhaps the pleasure of pointing out mistakes.

(Edit: a few sentences redrafted as a result of the mistakes that were pointed out. ;-)

What is a unit test?

A unit test is a test that tests a single unit of functionality, in isolation from all others, for a single set of conditions, and has a binary pass/fail result. For object-oriented languages, that means testing a single method on a single class with a single set of parameters. It is written by the developer, at approximately the same time as the code, for its purpose is satisfy the developer that said code behaves as she believes it should. (I believe there are huge advantages to writing the unit test immediately before the code it tests, but that isn't the subject of this article.)

In reality, it is difficult or impractical to test the code of a class in total isolation from all other classes. Even the simplest method uses classes from the Java standard library, not to mention classes from other libraries and classes from the application under development. With each of these dependencies, we have at best two options. We can use it as is, or where possible we can isolate the dependency by replacing it with a placeholder object, known as a stub, or mock, or fake, or double, or spy (the differences between these terms are subtle, and often ignored. I'll elaborate shortly).

Placeholders

If a placeholder is used, it needs to be created for the purpose, and often told how to behave. This needs to be done within the test code, before calling the method under test. After calling the method under test, you may also need to interrogate your placeholder to find out whether it was called as you expected.

Is it a mock or a stub?

A short terminological digression: Whether your placeholder is strictly speaking a mock, stub, or spy depends on whether you tell the placeholder how to behave, or ask it how it was treated. The following taxonomy comes from a short post on Martin Fowler's bliki, but note that most people abuse most of these terms liberally.

  • If you simply tell it how to behave in certain circumstances (e.g. "always return 3 from countFoos()"), it's a stub.
  • If you don't much mind how it behaves, but interrogate it afterwards to find out whether certain methods were called on it, and with what parameters, it's a spy. (But Mockito uses the term "spy" with another meaning - see the comments below.)
  • If you do both, and especially if you construct the object such that it will explode if it is not called according to your expectations, then it's a mock.
  • "Double" is a catch-all term for placeholder objects.
  • As for "fake" and "dummy", I'm not convinced anyone really uses those terms as precisely as defined in the link.
 

Off-the-shelf packages

Occasionally you find off-the-shelf packages of mock objects or stubs provided to assist testing of commonly-used classes (such as network or UI packages). Be wary of them: such a static mock framework is rather a heavyweight dependency to add to your project, and nothing guarantees that it will be maintained as assiduously as the product it mocks. You might easily find yourself trapped in an obsolete version of the product, because the mock framework is no longer maintained and you can't afford to rewrite all the test code that uses it. Furthermore, modern on-the-fly mock generators have reduced the amount of work that static mock frameworks can save you.

Mock object generators

I recommend, then, that you generate mocks using a mock-object generator, and for Java the generator I recommend is Mockito. That is the only recommendation for a specific technology you'll find in this article, and I'm making it because (as described in this presentation), Mockito's approach where you specify only those behaviours you need, and afterwards verify only those interactions you need, encourages less brittle and easier-to-read tests, as compared to an older mock generator like EasyMock. EasyMock requires you to verify interactions up-front, i.e. before calling the method under test, and by default performs strict verification, i.e. fails if the mocks are not called exactly as expected.

Masochism

Of course, you can also write your placeholders entirely by hand. But the tedium of it will just drive you to write your own mock-object framework. Don't do this (unless, that is, the framework you write is so good that you release it and it supplants all the existing ones).

Dependency isolation requires dependency injection

Since we want to keep test and production code separate, we can only(*) replace a dependency by a mock or stub during testing if that dependency is injected into the object under test. We cannot replace those that the tested object instantiates or retrieves explicitly by itself.

Thus, isolating dependencies involves extra effort: design effort in the tested class, to ensure that its dependencies are injectable, and coding effort in the test code, to prepare the mocked dependencies. When, then, should we do it, and when should we content ourselves with the real object? We can find the answer by looking at what we aim to achieve with a unit test.

The utility of unit tests

Each unit test tells us whether a particular method call with particular parameters behaved as expected, or not. One use of this is to detect bugs. But integration and acceptance tests will also detect bugs. The particular value of a unit test is that:

  1. It helps us find the cause of a bug, since whenever a test fails, the incorrect code must necessarily be in the tested method - insofar as we have taken the trouble to isolate the tested method.
  2. It finds bugs as soon as they are written, insofar as we launch them very frequently during coding (to encourage which, they need to be quick of execution).
 

Which dependencies to isolate?

To optimise our effort, we should therefore aim to eliminate those dependencies that might:

  • make the test fail when the tested class is correct (or make it pass when the tested class is faulty), thus reducing the value of the information that a test failure gives us
    • Uncertain behaviour: classes that we don't trust to be bug-free (typically the ones we've written ourselves), or whose behaviour we don't fully understand (this sometimes happens when getting to grips with complex UI or persistence frameworks)
    • Indeterminate behaviour: most often, classes whose behaviour is time-sensitive, such that a particular behaviour cannot predictably be reproduced with the original dependency.
    • States that are difficult to trigger: for example, we might want to test how our class behaves when a write to the filesystem throws an IOException.
  • slow down the test, thus reducing the incentive to launch tests often
    • Accesses to underlying resources: database, remote webservices, the filesystem...

Armed with these criteria, we can decide, for each dependency in a class, whether to mock. In practice, we don't even need to think about it in many cases:

  • Should not be mocked: objects from most library classes, especially those in java.lang and java.util, apart from the exceptions below; any objects with no real behaviour, such as parameter objects, data transfer objects and other bean-type objects.
  • Should be mocked: persistence classes; objects that provide user input; objects backed by network resources; timers; anything not trusted to be totally bug free (i.e. outside of a well-used library class); anything whose behaviour may change, such as a service returning real-world data whose content varies from one day to another
  • Grey area: anything that is merely more difficult to obtain or set up than the equivalent mock/stub, but not less reliable or significantly slower to run. The maintenance burden of creating mocks for a test needs to be balanced against the difficulty of using the real object. Further, using the real object makes it more likely that the dependency will behave under test as it does in production.
 

Integration tests: when it's impossible to isolate dependencies

There is one further limitation to isolating dependencies. In some cases it is technically impossible(*) either to dependency-inject or to mock:

  • you can only use DI with an object whose instantiation you control. Any framework that creates objects itself, even if the class of the created object is one you've written, does not give you an opportunity to inject dependencies.
  • you can only create mocks of non-final classes (which is another reason no-one mocks java.lang.String, aside from the pointlessness of such an act).

In these cases, your only option is to do an integration test. Integration tests are any tests written by developers, as unit tests are, that produce an pass/fail result, as unit tests do, but that cover more than one significant class in the system or its libraries. If you test a data access object with a real underlying database, that is an integration test, since you are simultaneously testing your DAO code (sometimes minimal or auto-generated), your ORM mapping and your database schema. And - not the least important - you're testing their coherence with respect to each other.

Good integration testing practice

A word or two more about integration tests, even though they aren't the subject of this article. Iintegration tests aren't only useful as a fallback when unit tests are inappropriate or impossible: you should be doing them anyway. Good practice for integration tests is to have a the minimal set necessary to make sure that different elements like ORM mapping and database schema are coherent.

You need at least this much, because integration tests will catch things that unit tests never can: incompatibility between elements (a property added in a persistence entity bean and the ORM mapping, but missing from the database...) and configuration errors. You don't want any more than that, because integration tests tend to require more work to set up and maintain -- and take more time to run.

* A commenter corrected this assertion, pointing out that there is one mocking framework, JMockit, that allows you to mock even objects instantiated via the new operator by the method you are testing. It does this via instrumentation, rather than dynamic proxies and CGLib as used by the other frameworks I mentioned. I haven't tried it out, so I'd be interested in any feedback from readers.

(Originally posted at www.andrewspencer.net.)

Published at DZone with permission of Andrew Spencer, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Davide Gall replied on Tue, 2011/02/08 - 6:13am

good !

Rogerio Liesenfeld replied on Tue, 2011/02/08 - 6:36am

Actually, there are several mistakes in the article:

1. The third paragraph starts out saying that it's "impossible to test the code of a class in total isolation from all other classes", but later says that "we can isolate the dependency by replacing it". This is contradictory. (And in actual fact it *is* possible to test a class in total isolation.)

2. The definition given for "spy" is correct, but the article fails to mention that a Mockito spy is something else (a stub where the real methods get executed through the spy instance, unless the method was stubbed in the test).

3. EasyMock does *not* "require you to specify all behaviours and expected interactions up-front". This is simply not the case, for *any* mocking tool (EasyMock, jMock, Mockito, etc.)

4. The entire section entitled "Dependency isolation requires dependency injection" is wrong. What actually happens is that *some* specific mocking tools require it. Others do not: JMockit (my own tool) and PowerMock (which is a complement to the EasyMock and Mockito APIs). With these latter tools, we *can* replace dependencies that "the tested object instantiates or retrieves explicitly by itself" (and quite easily, by the way). No extra effort required, at all.

5. The section on integration tests repeats previous mistakes. Strictly speaking, there are no cases in which it is "technically impossible to mock"; there are only limitations in certain mocking tools. One such limitation is the inability of some mocking tools to mock final classes and methods. The java.lang.String class, for example, can be easily mocked with JMockit (and there *are* cases where doing so makes sense).

Nicolas Frankel replied on Tue, 2011/02/08 - 6:59am

Hi Andrew,

Thanks for this interesting article. I'd like to raise two points, though:

  • IMHO, off-the-shelf Mock frameworks are a good thing when applied to "standard" API. I don't want to code for the umpfteenth time the behaviour I want from a mock HttpServletRequest. In this case, I'm much more productive running to Spring's mocks API or MockRunner
  • Also, the title of your last paragraph may be misleading "Integration tests: when it's impossible to isolate dependencies". Integration tests are much more than unit tests that cannot run in isolation due to bad (or legacy) code design. So, I must disagree with your definition :-)

Rogerio Liesenfeld replied on Tue, 2011/02/08 - 7:19am in response to: Nicolas Frankel

Nicolas,

All mocking tools support the reuse of mocking code (recorded expectations, typically), so it's debatable whether canned fake implementations such as those provided by Spring are really more convenient to use. For a complete example test suite where several mocking APIs plus Spring Mock were used to test the same code, see http://code.google.com/p/jmockit/source/browse/trunk#trunk%2Fsamples%2Ftourdemock

Héctor Hugo Bar... replied on Tue, 2011/02/08 - 8:07am

Good article. You should consider reading Mock Roles, Not Objects paper to know when you should use a mock object.

Andrew Spencer replied on Tue, 2011/02/08 - 9:38am in response to: Rogerio Liesenfeld

Thanks for your corrections. I've redrafted several sentences to correct and/or clarify points 1-3. I can't really correct points 4 and 5 without major surgery, so I've made do with a footnote.

Andrew Spencer replied on Tue, 2011/02/08 - 9:41am in response to: Nicolas Frankel

Nicolas, I take your point but I'm still more inclined to agree with what Rogerio said. However, I do very much like the conciseness of the code in his "tourdemock" Spring mock example. (It isn't actually all that concise because he uses hand-built mocks for the non-Spring-provided mock objects, but it would have been had he used a combination of Spring Mocks and Mockito.)

Andrew Spencer replied on Tue, 2011/02/08 - 9:58am in response to: Héctor Hugo Barriuso Mata

Héctor, thanks for the link. That article chiefly describes a style of TDD where mock objects are used to drive the design of the system's classes. The authors state that mocks are useful for isolating code from external dependencies (3rd party libraries), as I described, but they see that as only a secondary use.

What they describe is a way of working, that goes beyond what I've written. I like their approach and it is complementary to my suggestions above. However, not everyone will choose to work that way, whereas I've tried to outline principles that will be valid for anyone doing unit testing.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.