DevOps Zone is brought to you in partnership with:

Gil Zilberfeld has been in software since childhood, writing BASIC programs on his trusty Sinclair ZX81. With more than twenty years of developing commercial software, he has vast experience in software methodology and practices. Gil is an agile consultant, applying agile principles over the last decade. From automated testing to exploratory testing, design practices to team collaboration, scrum to kanban, and lean startup methods – he’s done it all. He is still learning from his successes and failures. Gil speaks frequently in international conferences about unit testing, TDD, agile practices and communication. He is the author of "Everyday Unit Testing", blogs at http://www.gilzilberfeld.com and in his spare time he shoots zombies, for fun. Gil is a DZone MVB and is not an employee of DZone and has posted 71 posts at DZone. You can read more from them at their website. View Full User Profile

Test Attribute #8 – Truthiness

08.07.2014
| 2278 views |
  • submit to reddit
This is the 8th post, soon to be the 8th wonder of the world, in the Test Attribute series that started here. To learn more about testing, contact me.

I want to thank Steven Colbert for coining a word I can use in my title. Without him, all this would still be possible, had I not given up looking for a better word after a few minutes.

Tests are about trust. We expect them to be reliable. Reliable tests tell us everything is ok when they pass, and that something is wrong when they fail.

The problem is that life is not black and white, and tests are not just green and red. Tests can give false positive (fail when they shouldn’t) or false negative (pass when they shouldn’t) results. We’ve encountered the false positive ones before – these are the fragile, dependent tests.

The ones that pass, instead of failing, are the problematic ones. They hide the real picture from us, and erode our trust, not just in those tests, but also in others. After all, when we find out a problematic tests, who can say the others we wrote are not problematic as well?

Truthiness (how much we feel the tests are reliable) comes into play.

Dependency Injection Example

Or rather, injecting an example of a dependency.

Let’s say we have a service (or 3rd party library) our tested code uses. It’s slow and communication is unreliable. All the things that give services a bad name. Our natural tendency is to mock the service in the test. By mocking the service, we can test our code in isolation.

So, in our case, our tested Hotel class uses a Service:
public class Hotel
{ 
    public string GetServiceName(Service service)
    {
        var result = service.GetName();
        return "Name: " + result;
    }
}
To know if the method works correctly, we’ll write this test:
[TestMethod]public void GetServiceName_RoomService_NameIsRoom()
{
    var fakeService = A.Fake<Service>();
    A.CallTo(() => fakeService.GetName()).Returns("Room");

    var hotel = new Hotel();
    Assert.AreEqual("Name: Room", hotel.GetServiceName(fakeService));
}
And everything is groovy.
Until, in production, the service gets disconnected and throws an exception. And our test says “B-b-b-but, I’m still passing!”.

The Truthiness Is Out There

Mocking is an example of obstructing the real behavior by prescriptive tests, but it’s just an example. It can happen when we test a few cases, but don’t cover others.

Here’s one of my favorite examples. What’s the hidden test case here?
public int Increment()
{
    return counter++;
}
Tests are code examples. They work to the extent of our imagination of “what can go wrong?” Like overflow, in the last case.

Much like differentiation, truthiness can not be examined by itself. The example works, but it hides a case we need another test for. We need to look at the collection of test cases, and see if we covered everything.

The solution doesn’t have to be a test of the same type. We can have a unit test for the service happy path, and an end-to-end test to cover the disconnection case. Of course, if you can think of other cases in the first place, why not unit test them?

So to level up your truthiness:
  • Ideate. Before writing the tests, and if you’re doing TDD – the code, write a list of test cases. On a notebook, a whiteboard, or my favorite: empty tests.

  • Reflect. Often, when we write a few test, new test cases come to mind. Having a visual image of the code can help think of other cases.

  • Beware the mock. We use mocks to prescribe dependency behavior in specific cases. Every mock you make can be a potential failure point, so think about other cases to mock.

  • Review. Do it in pairs. Four eyes are better than two.
Aim for higher truthiness. Higher trust in your tests will help you sleep better.

Next time: Determinism.
For training and coaching on topics like this, contact me.
Published at DZone with permission of Gil Zilberfeld, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)