DevOps Zone is brought to you in partnership with:

I've been fascinated with software development since I got my first C64 thirty years ago. And I still like the I emerging possibilities the technology offers us every day. I work as developer and software architect in internal and customer projects focusing on Java and Oracle. Thomas has posted 10 posts at DZone. You can read more from them at their website. View Full User Profile

The Magic Testing Challenge: Part 2

  • submit to reddit

My last article raised an interesting discussion whether you should see tests more as documentation or more as specification. I agree that they can contribute to both of them, but I still think tests are just - tests...

There were also complaints about my statement that testing often becomes tedious work which nobody likes. Also here I agree, that techniques like TDD can help you to structure your code and make sure you code exactly what is needed by writing the tests, but the result of the process will still be a class which needs to be tested somehow.

So I have set up another small challenge to show how the visual approach featured by MagicTest helps to make testing a breeze. As you know, traditional assertion-based test frameworks like TestNG or JUnit force us to include the expected results in the test code. Where this may be more or less suitable for simple tests (like in the previous article), it quickly becomes cumbersome if the test handles complex objects or voluminous data.

The Task

We must test the method createEvenOddTable() (see appended listing) with the following functionality:

  • Create HTML table (elements table, tr, td) with specified number of data rows and columns.

  • An additional row will be added to store header information (element th).

  • An additional column will be added which contains the row number (element th)

  • The rows will have attribute class set to "head", "even", or "odd" for easy styling.

Both the specification (the 4 lines above) and the source code itself (25 lines) are short and simple to understand, so any experienced developer will write this method in a few minutes. So what's the problem with testing this method? We will see if we look at how MagicTest handles this case.

The Magic Test

The MagicTest for this method looks like this:

public class HtmlTableTest {

	public void testCreateEvenOddTable() {
	    HtmlTable.createEvenOddTable(4, 3);

	public static String formatElement(Element elem) {
	    XMLOutputter serializer = new XMLOutputter();
	    return serializer.outputString(elem);

Some details:

  • We use the @Trace annotation to automatically capture information about calls to the method under test.

  • We rely on naming conventions, so the method HtmlTable.createEvenOddTable() is tested by HtmlTableTest.testCreateEvenOddTable().

  • Per default, MagicTest uses the toString() method to report the parameter and return values. As the Element's toString() method returns only its name, we have to define a custom @Formatter to get the full XML tree.

If we run the test, we get the following report:

If we look at the XML element tree in the report, we can see all the details which a complete test should cover: correct nesting of elements (table, tr, td), correct header line, correct line numbers, correct number of rows, correct number of cells for each row, correct class attribute for each row, etc.

But even if you end up with a bunch of lengthy assert statements like

	    assert("head".equals(((Element) elem.getChildren("tr").get(0)).getAttributeValue("class")));

which tests for the correct class attribute, this will not be enough: you should also test the absence of the class attribute for all cells except the first ones in each row. So yes, for a sound test you must actually verify the whole XML tree - and this is exactly the information which MagicTest shows you for confirmation.

Let the Challenge Begin

To run the test yourself, you will need to download the MagicTest Eclipse plug-in. Copy it into the Eclipse dropins folder and restart Eclipse. Then download the attached Eclipse project and import it into your workspace. Run the test class TagsTest by executing Run As / MagicTest.

After the first run, the test report will show up and all test steps will be red. This is the MagicTest way of telling you that a step has failed. In our case, the steps just fail because MagicTest simply does not know anything about the expected result. So we carefully check the output and confirm its correctness by clicking on the save button. Now all steps are green - and the test is successful.

You have now seen how efficiently this test can be realized using MagicTest - it even looked like fun. Does your test tool accept the challenge? How many minutes and lines does it take you to write the test? I'm looking forward to your contributions!

Appendix: Listing HtmlTable

	 * Create HTML table (elements table, tr, td) with specified number of data rows and columns.
	 * An additional row will be added to store header information (element th).
	 * An additional column will be added which contains the row number (element th)
	 * The rows will have attribute class set to "head", "even", or "odd" for easy styling.
	 * @param rows	number of rows
	 * @param cols	number of column
	 * @return		XML element containing the HTML table
	public static Element createEvenOddTable(int rows, int cols) {
		Element table = new Element("table");
		for (int r=0; r<rows+1; r++) {
			Element tr = new Element("tr");
			String trClass;
			if (r == 0) {
				trClass = "head";
			} else {
				trClass = (r%2==1) ? "even" : "odd";
			tr.setAttribute("class", trClass);
			for (int c=0; c<cols+1; c++) {
				String tdElem;
				if (r == 0) {
					tdElem = "th";
				} else {
					tdElem = (c==0) ? "th" : "tr";
				Element td = new Element(tdElem);
				if (c == 0 && r > 0) {
		return table;

MagicTest-Challenge-2.zip189.54 KB
Published at DZone with permission of its author, Thomas Mauch.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Erik Westra replied on Thu, 2014/03/06 - 2:24am

There is a bug in your code... 

Your tests say: OK

I think that proofs some points I made in a comment on your previous article.

Thomas Mauch replied on Thu, 2014/03/06 - 5:26am in response to: Erik Westra

 It's not really a bug, but the text in the article is not correct....

If you download the attached Ecilpse project, it contains the "magictest" directory where MagicTest stores the reference output. So even if you run your test the first time, MagicTest already knows the correct answer and recognizes the test as successful.

Sorry for the minor inaccuracy.

Erik Westra replied on Thu, 2014/03/06 - 3:01pm

I am sorry, my comment was a bit short and might have come across a bit harsh.

What I meant was that it's a bit weird that the output prints "1" and "even" on the same row. On top of that it has "tr" elements in the wrong locations.

What I mean to say is that the "x%2 == 1" and other stuff could be typeo's. These are easily missed using this form of testing.

This technique can only be used for regression testing.

I think regression testing, especially for code without specification, it not very valuable. You might be testing if an unintended bug remains in the code, or that the code keeps behaving in a way that is not good.

I personally think it's more important to focus on writing valuable code than it is to test regression of code for which the intent is unspecified. We are human and make mistakes, if we have our intentions written down, someone else can later on help us fix those mistakes. Writing down intentions can be in a lot of forms, but I think it should be kept close to the code, for example with specifications (or tests) that can be executed.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.