Hi all, my name is Hubert A. Klein Ikkink. Not a very common name, right? To make things easier I just picked the first letters of my firstname and surname and came up with haki. So there you have it, now I am also known as Mr. Haki or mrhaki for short. You can read more blog postings at www.mrhaki.com. I am a passionate Groovy and Java developer based in Tilburg, The Netherlands. My goal is to write clean, elegant, user-centered and high quality software. You can find me on Google+ and Twitter. Hubert is a DZone MVB and is not an employee of DZone and has posted 151 posts at DZone. You can read more from them at their website. View Full User Profile

Gradle Goodness: Show More Information About Failed Tests

05.14.2013
| 8930 views |
  • submit to reddit

Running tests in Gradle is easy. Normally if one of the tests fails the build fails as well. But we don't see immediately in the command-line output why a test fails. We must first open the generated HTML test report. But there are other ways as well.

First we create the following sample Gradle build file:// File: build.gradle

apply plugin: 'java'

repositories {
    mavenCentral()
}

dependencies {
    testCompile 'junit:junit:[4,)'
}

And we use the following sample JUnit test class. Notice this test will always fail, which is what we want in this case.// File: src/test/java/com/mrhaki/gradle/SampleTest.java

package com.mrhaki.gradle;

import org.junit.*;

public class SampleTest {

    @Test public void sample() {
        Assert.assertEquals("Gradle is gr8", "Gradle is great");
    }
    
}

To run our test we execute the test task. If we run the task we see in the output on which line the test fails, but we don't see the assertion why it went wrong:

$ gradle test
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava
:processTestResources UP-TO-DATE
:testClasses
:test

com.mrhaki.gradle.SampleTest > sample FAILED
    org.junit.ComparisonFailure at SampleTest.java:8

1 test completed, 1 failed
:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/mrhaki/Projects/mrhaki.com/blog/posts/samples/gradle/testlogging/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 4.904 secs

We can run the test task again, but now set the Gradle logging level to info with the command-line option -i or --info. Now we get the assertion about what went wrong in the output:

$ gradle test -i
Starting Build
Settings evaluated using empty settings script.
Projects loaded. Root project using build file 
...
Successfully started process 'Gradle Worker 1'
Gradle Worker 1 executing tests.
Gradle Worker 1 finished executing tests.

com.mrhaki.gradle.SampleTest > sample FAILED
    org.junit.ComparisonFailure: expected:<gradle is gr[8]> but was:<gradle is gr[eat]>
        at org.junit.Assert.assertEquals(Assert.java:115)
        at org.junit.Assert.assertEquals(Assert.java:144)
        at com.mrhaki.gradle.SampleTest.sample(SampleTest.java:8)
Process 'Gradle Worker 1' finished with exit value 0 (state: SUCCEEDED)

1 test completed, 1 failed
Finished generating test XML results (0.025 secs)
Generating HTML test report...
Finished generating test html results (0.027 secs)
:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/mrhaki/Projects/mrhaki.com/blog/posts/samples/gradle/testlogging/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.

BUILD FAILED

Total time: 5.117 secs

But this still generates a lot of noise. It is better to customize the test logging by configuring the test task. We can configure the logging on different levels. To get the information about the failure we want we only have to change the exceptionFormat property and set the value tofull. Our Gradle build file now looks like this:

// File: build.gradle
apply plugin: 'java'

repositories {
    mavenCentral()
}

dependencies {
    testCompile 'junit:junit:[4,)'
}

test {
    testLogging {
        exceptionFormat = 'full'
    }
}

We can re-run the test task and use the normal logging level, but this time we also get the reason why our test fails, without the extra noise:

$ gradle test
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test

com.mrhaki.gradle.SampleTest > sample FAILED
    org.junit.ComparisonFailure: expected:<gradle is gr[8]> but was:<gradle is gr[eat]>
        at org.junit.Assert.assertEquals(Assert.java:115)
        at org.junit.Assert.assertEquals(Assert.java:144)
        at com.mrhaki.gradle.SampleTest.sample(SampleTest.java:8)

1 test completed, 1 failed
:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/mrhaki/Projects/mrhaki.com/blog/posts/samples/gradle/testlogging/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 5.906 secs

Sample written with Gradle 1.6

Published at DZone with permission of Hubert Klein Ikkink, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Don Heninger replied on Wed, 2013/05/22 - 8:45am

Excellent Tip!!

Just one question, I have never seen the expression style that you used in your declaration for the junit dependency:  testCompile 'junit:junit:[4,)'.  I assume this means grab the latest version of junit 4.  Correct?

Marco Ehrentreich replied on Sun, 2013/05/26 - 2:53am in response to: Don Heninger

It means a version greater or equal to 4.


Gradle uses the same syntax as Ivy to specify dependencies. See here for a detailed description ;-)


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.