Hamlet D'Arcy has been writing software for over a decade, and has spent considerable time coding in C++, Java, and Groovy. He's passionate about learning new languages and different ways to think about problems, and recently he's been discovering the joys of both F# and Scheme. He's an active member of the Groovy Users of Minnesota and the Object Technology User Group, is a committer on the Groovy project, and is a contributor on a few open source projects (including JConch and the IDEA Groovy Plugin). He blogs regularly at http://hamletdarcy.blogspot.com and can be found on Twitter as HamletDRC (http://twitter.com/hamletdrc). Hamlet is a DZone MVB and is not an employee of DZone and has posted 28 posts at DZone. You can read more from them at their website. View Full User Profile

Mockito - Pros, Cons, and Best Practices

10.14.2010
| 28517 views |
  • submit to reddit

It's been almost 4 years since I wrote a blog post called "EasyMock - Pros, Cons, and Best Practices, and a lot has happened since. You don't hear about EasyMock much any more, and Mockito seems to have replaced it in mindshare. And for good reason: it is better.

A Good Humane Interface for Stubbing
Just like EasyMock, Mockito allows you to chain method calls together to produce less imperative looking code. Here's how you can make a Stub for the canonical Warehouse object:

Warehouse mock = Mockito.mock(Warehouse.class);
Mockito.when(mock.hasInventory(TALISKER, 50)).
        thenReturn(true);

I know, I like a crazy formatting. Regardless, giving your System Under Test (SUT) indirect input couldn't be easier. There is no big advantage over EasyMock for stubbing behavior and passing a stub off to the SUT. Giving indirect input with mocks and then using standard JUnit asserts afterwards is simple with both tools, and both support the standard Hamcrest matchers.

Class (not just Interface) Mocks
Mockito allows you to mock out classes as well as interfaces. I know the EasyMock ClassExtensions allowed you to do this as well, but it is a little nicer to have it all in one package with Mockito.

Supports Test Spies, not just Mocks
There is a difference between spies and mocks. Stubs allow you to give indirect input to a test (the values are read but never written), Spies allow you to gather indirect output from a test (the mock is written to and verified, but does not give the test input), and Mocks are both (your object gives indirect input to your test through Stubbing and gathers indirect output through spying). The difference is illustrated between two code examples. In EasyMock, you only have mocks. You must set all input and output expectations before running the test, then verify afterwards. 

// arrange
Warehouse mock = EasyMock.createMock(Warehouse.class); 
EasyMock.expect(
    mock.hasInventory(TALISKER, 50)).
    andReturn(true).once();
EasyMock.expect(
    mock.remove(TALISKER, 50)).
    andReturn(true).once();
EasyMock.replay(mock); 

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
EasyMock.verify(mock);


That's a lot of code, and not all of it is needed. The arrange section is setting up a stub (the warehouse has inventory) and setting up a mock expectation (the remove method will be called later). The assertion in all this is actually the little verify() method at the end. The main point of this test is that remove() was called, but that information is buried in a nest of expectations. Mockito improves on this by throwing out both the record/playback mode and a generic verify() method. It is shorter and clearer this way:

// arrange
Warehouse mock = Mockito.mock(Warehouse.class);
Mockito.when(mock.hasInventory(TALISKER, 50)).
        thenReturn(true);

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
Mockito.verify(warehouse).remove(TALISKER, 50);

The verify step with Mockito is spying on the results of the test, not recording and verifying. Less code and a clearer picture of what really is expected. Update: There is a separate Spy API you can use in Mockito as well: http://mockito.googlecode.com/svn/branches/1.8.3/javadoc/org/mockito/Mockito.html#13

Better Void Method Handling

Mockito handles void methods better than EasyMock. The fluent API works fine with a void method, but in EasyMock there were some special methods you had to write. First, the Mockito code is fairly simple to read:

// arrange
Warehouse mock = Mockito.mock(Warehouse.class);

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
Mockito.verify(warehouse).remove(TALISKER, 50);
Here is the same in EasyMock. Not as good:
// arrange
Warehouse mock = EasyMock.createMock(Warehouse.class); 
mock.remove(TALISKER, 50); 
EasyMock.expectLastMethodCall().once(); 
EasyMock.replay(mock); 

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
EasyMock.verify(mock); 

Mock Object Organization Patterns

Both Mockito and EasyMock suffer from difficult maintenance. What I said in my original EasyMock post holds true for Mockito:

The method chaining style interface is easy to write, but I find it difficult to read. When a test other than the one I'm working on fails, it's often very difficult to determine what exactly is going on. I end up having to examine the production code and the test expectation code to diagnose the issue. Hand-rolled mock objects are much easier to diagnose when something breaks... This problem is especially nasty after refactoring expectation code to reduce duplication. For the life of me, I cannot follow expectation code that has been refactored into shared methods.

Now, four years later, I have a solution that works well for me. With a little care you can make your mocks reusable, maintainable, and readable. This approach was battle tested over many months in an Enterprise Environment(tm). Create a private static method the first time you need a mock. Any important data needs to be passed in as a parameter. Using constants or "magic" fields hides important information and obfuscates tests. For example:

User user = createMockUser("userID", "name"); 
...
assertEquals("userID", result.id()); 
assertEquals("name", result.name(); 

Everything important is visible and in the test, nothing important is hidden. You need to completely hide the replay state behind this factory method if you're still on EasyMock. The Mock framework in use is an implementation detail and try not to let it leak.

Next, as your dependencies grow, be sure to always pass them in as factory method parameters. If you need a User and a Role object, then don't create one method that creates both mocks. One method instantiates one object, otherwise it is a parameter and compose your mock objects in the test method:

User user = createMockUser(
    "userID", 
    "name", 
    createMockRole("role1"), 
    createMockRole("role2")
); 

When each object type has a factory method, then it makes it much easier to compose the different types of objects together. Reuse. But you can only reuse the methods when they are simple and with few dependencies, otherwise they become too specific and difficult to understand. The first time you need to reuse one of these methods, then move the method to a utility class called "*Mocker", like UserMocker or RoleMocker. Follow a naming convention so that they are always easy to find. If you remembered to make the private factory methods static then moving them should be very simple. Your client code ends up looking like this, but you can use static imports to fix that:

User user = UserMocker.createMockUser(
    "userID", 
    "name", 
    RoleMocker.createMockRole("role1"), 
    RoleMocker.createMockRole("role2")
); 

User overloaded methods liberally. Don't create one giant method with every possible parameter in the parameter list. There are good reasons to avoid overloading in production, but this is test. Use overloading so that the test methods only display data relevant to that test and nothing more. Using Varargs can also help keep a clean test. Lastly, don't use constants. Constants hide the important information out of sight, at the top of the file where you can't see it or in a Mocker class. It's OK to use constants within the test case, but don't define constants in the Mockers, it just hides relevant information and makes the test harder to read later.

Avoid Abstract Test Cases

Managing mock objects within abstract test cases has been very difficult for me, especially when managing replay and record states. I've given up mixing mock objects and abstract TestCase objects. When something breaks it simply takes too long to diagnose. An alternative is to create custom assertion methods that can be reused. Beyond that, I've given up on Abstract TestCase objects anyway, on the grounds of preferring composition of inheritance.

Don't Replace Asserts with Verify

My original comments about EasyMock are still relevant for Mockito: The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked. In contrast, mock objects make it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method's purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug. In fact, some languages don't even allow it! If you're test code contains assert methods then you have a good test. If you're code doesn't contain asserts, and instead contains a long list of verify() calls, then you're relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, "My test method needs to do several things: x, y, and z." The charter and responsibility of the method is no longer clear. This is a candidate for refactoring.

No More All or Nothing Testing

Mockito's verify() methods are much more flexible than EasyMock's. You can verify that only one or two methods on the mock were called, while EasyMock had just one coarse verify() method. With EasyMock I ended up littering the code with meaningless expectations, but not so in Mockito. This alone is reason enough to switch.

Failure: Expected X received X

For the most part, Mockito error messages are better than EasyMock's. However, you still sometimes see a failure that reads "Failure. Got X Expected X." Basically, this means that your toString() methods produce the same results but equals() does not. Every user who starts out gets confused by this message at some point. Be Warned.

Don't Stop Handrolling Mocks

Don't throw out hand-rolled mock objects. They have their place. Subclass and Override is a very useful technique for creating a testing seam, use it.

Learn to Write an ArgumentMatcher

Learn to write an ArgumentMatcher. There is a learning curve but it's over quickly. This post is long enough, so I won't give an example.

That's it. See you again in 4 years when the next framework comes out!

 From http://hamletdarcy.blogspot.com/2010/09/mockito-pros-cons-and-best-practices.html

Published at DZone with permission of Hamlet D'Arcy, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Frode Stokke replied on Thu, 2010/10/14 - 3:26am

Good article! I enjoyed reading it. You have some insightful observations.

Slava Lo replied on Thu, 2010/10/14 - 6:30am

Easymock has a concept of Capture, which I think can be used to 'spy'.

Using captures you can analize what actual values were passed to the method during the execution of the test and later assert them against expected values.

 

Rogerio Liesenfeld replied on Thu, 2010/10/14 - 8:20am

This article has several innacuracies:

1) Maybe Mockito is "more in the news" now, but EasyMock still seems to be more used in the real world: a job search on indeed.com produces 150 hits for "EasyMock" and 46 for "Mockito".
2) In EasyMock 3.0 (the current version), the "Class Extension" API (now deprecated) has been merged with the original EasyMock API. So, now EasyMock also provides class mocking out of the box.
3) Mockito also requires the use of different API when recording (non-strict) expectations on mocks. You can't write when(mock.doSomething()).thenThrow(new XyzException()); with Mockito if doSomething() is a void method; it would have to be doThrow(new XyzException()).when(mock).doSomething();. Other Java mocking APIs don't suffer from this inconsistency.
4) It's simply not true that "Mockito's verify() methods are much more flexible than EasyMock's". The only real difference is that EasyMock supports implicit verification but no explicit verification, while Mockito does the opposite. If you "ended up littering the code with meaningless expectations" when using EasyMock, then you misused the API. Instead, make use of methods like EasyMock.createNiceMock, andStubReturn, etc. You can "verify that only one or two methods on the mock were called", just as easily.

Tomasz Nurkiewicz replied on Thu, 2010/10/14 - 1:34pm

Recently I discovered that ArgumentCaptor is much easier to use than custom ArgumentMatcher. In most cases they can be used interchangeably, but using captor seems more obvious and straightforward.

Jonathan Fisher replied on Thu, 2010/10/14 - 5:36pm

I think the Mockito examples and documentation are far superior to EasyMock. I also think that having the documentation in the JavaDoc is brilliant.

Marek Dec replied on Sun, 2010/10/17 - 6:05am

Cool article, however I have some doubts regarding the "Don't Replace Asserts with Verify" paragraph. I wouldn't say that asserts and verifies are interchangeable. Also, I like your practical advice on the asserts vs verifies usage, though my opinion differs a bit. A unit test verification section should define methods's (or SUT's) responisibilites.
Let's consider an int countBrownEyedUsersAndAlertIfMoreBlueEyed() method which counts the brown eyed users of our system and calls the Alert service if the number of blue eyed ones if bigger. Imagine also, that the the list of users in the system is stored in a DB and our SUT will have to call some DAO service to obtain them.
And here comes the question what we should verify. My answer is: verify both the correct number of brown eyed users is returned and that the alert service has been invoked, do NOT verify the call to the dao as this is not what we really expect to be done (because maybe in the future the users list will be acquired from some cache service, not a DB). In other words, the test should verify that a SUT fulfills its tasks regardless it is an interaction, a state change or a return value. At the same time, it should not verify any internal calls which are a result of chosen implementation, even though sometimes they have to be mocked (in the example a call to the DB will probably have to be mocked, but not verified). And this is what makes the great difference between EasyMock and Mockito i.e. EasyMock verified every call on a Mock object declared in the expect section; with Mockito you can explicitly specify what has to be verified.

 

Joao Nelas replied on Sun, 2010/10/17 - 12:57pm

I use the same principle in organizing mocks. I tend to use a factory method for simple mocks, and convert them to builder classes if they start to get complicated. It makes reading the tests a lot simpler and favors reuse and consistency among different tests.

Graham Lea replied on Fri, 2010/11/12 - 6:59am

Hi Hamlet,

 Thanks a lot for writing this up. Our team use EasyMock a lot and we trialled Mockito a couple of months ago but couldn't really get our heads around why it was any different, so it's great to have someone spell it out.

 I noticed there is a subtle difference between the operation of EasyMock and Mockito looking at your 2nd and 3rd code examples. I can make your Mockito test pass without ever calling hasInventory() in my implementation. If I implement Order.fill() to just call Warehouse.remove(), your test will pass. The EasyMock test, on the other hand, will fail on the verify(), because the call to the collaborator is expected, even though it causes no mutation (I assume).

 Of course, the answer is to implement the corresponding test, where hasInventory() returns false. I'm not saying this is better or worse. I suppose the EasyMock approach could give a bit more of a hand to devs who aren't adept at writing all the necessary tests. I think it is an interesting dichotomy to draw between stubs and spies. It seems to me as if the Mockito designers are trying to force people into this pattern where a method either returns something or changes state, but not both. Again, I'm not saying this is a bad thing, but this isn't a pattern that everyone follows, or probably that anyone follows all the time.

Again - thanks for the write-up. Some food for thought here.

Binh Nguyen Thanh replied on Tue, 2013/07/02 - 11:13am

Thanks, This writing is useful to me

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.