I'm an Agile and Lean Strategist specialised in coaching and managing the transformation of IT departments, from startups to enterprise-scale organisations, to highly efficient, productive and energised environments. I also have experience as senior development manager and architecture governance in large enterprises, especially in the finance sector. Marco is a DZone MVB and is not an employee of DZone and has posted 26 posts at DZone. You can read more from them at their website. View Full User Profile

XML unmarshalling benchmark in Java: JAXB vs STax vs Woodstox

06.27.2011
| 24396 views |
  • submit to reddit

Towards the end of last week I started thinking how to deal with large amounts of XML data in a resource-friendly way.The main problem that I wanted to solve was how to process large XML files in chunks while at the same time providing upstream/downstream systems with some data to process.

Of course I've been using JAXB technology for few years now; the main advantage of using JAXB is the quick time-to-market; if one possesses an XML schema, there are tools out there to auto-generate the corresponding Java domain model classes automatically (Eclipse Indigo, Maven jaxb plugins in various sauces, ant tasks, to name a few). The JAXB API then offers a Marshaller and an Unmarshaller to write/read XML data, mapping the Java domain model.

When thinking of JAXB as solution for my problem I suddendlly realised that JAXB keeps the whole objectification of the XML schema in memory, so the obvious question was: "How would our infrastructure cope with large XML files (e.g. in my case with a number of elements > 100,000) if we were to use JAXB?". I could have simply produced a large XML file, then a client for it and find out about memory consumption.

As one probably knows there are mainly two approaches to processing XML data in Java: DOM and SAX. With DOM, the XML document is represented into memory as a tree; DOM is useful if one needs cherry-pick access to the tree nodes or if one needs to write brief XML documents. On the other side of the spectrum there is SAX, an event-driven technology, where the whole document is parsed one XML element at the time, and for each XML significative event,  callbacks are "pushed" to a Java client which then deals with them (such as START_DOCUMENT, START_ELEMENT, END_ELEMENT, etc). Since SAX does not bring the whole document into memory but it applies a cursor like approach to XML processing it does not consume huge amounts of memory. The drawback with SAX is that it processes the whole document start to finish;  this might not be necessarily what one wants for large XML documents. In my scenario, for instance, I'd like to be able to pass to downstream systems XML elements as they are available, but at the same time maybe I'd like to pass only 100 elements at the time, implementing some sort of pagination solution. DOM seems too demanding from a memory-consumption point of view, whereas SAX seems to coarse-grained for my needs. 

I remembered reading something about STax, a Java technology which offered a middle ground between the capability to pull XML elements (as opposed to pushing XML elements, e.g. SAX) while being RAM-friendly. I then looked into the technology and decided that STax was probably the compromise I was looking for; however I wanted to keep the easy programming model offered by JAXB, so I really needed a combination of the two. While investigating STax, I came across Woodstox;  this open source project promises to be a faster XML parser than many othrers, so I decided to include it in my benchmark as well.  I now had all elements to create a benchmark to give me memory consumption and processing speed metrics when processing large XML documents.

The benchmark plan

In order to create a benchmark I needed to do the following:

  • Create an XML schema which defined my domain model. This would be the input for JAXB to create the Java domain model
  • Create three large XML files representing the model, with 10,000 / 100,000 / 1,000,000 elements respectively
  • Have a pure JAXB client which would unmarshall the large XML files completely in memory
  • Have a STax/JAXB client which would combine the low-memory consumption of SAX technologies with the ease of programming model offered by JAXB
  • Have a Woodstox/JAXB client with the same characteristics of the STax/JAXB client (in few words I just wanted to change the underlying parser and see if I could obtain any performance boost)
  • Record both memory consumption and speed of processing (e.g. how quickly would each solution make XML chunks available in memory as JAXB domain model classes)
  • Make the results available graphically, since, as we know, one picture tells one thousands words.

 The Domain Model XML Schema

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://uk.co.jemos.integration.xml/large-file" xmlns:tns="http://uk.co.jemos.integration.xml/large-file" elementFormDefault="qualified">

    <complexType name="PersonType">
        <sequence>
            <element name="firstName" type="string"></element>
            <element name="lastName" type="string"></element>
            <element name="address1" type="string"></element>
            <element name="address2" type="string"></element>
            <element name="postCode" type="string"></element>
            <element name="city" type="string"></element>
            <element name="country" type="string"></element>
        </sequence>
        <attribute name="active" type="boolean" use="required" />
    </complexType>


    <complexType name="PersonsType">
        <sequence>
            <element name="person" type="tns:PersonType" maxOccurs="unbounded" minOccurs="1"></element>
        </sequence>
    </complexType>

    <element name="persons" type="tns:PersonsType">
    </element>
</schema>

 

I decided for a relatively easy domain model, with XML elements representing people, with their names and addresses. I also wanted to record whether a person was active.

Using JAXB to create the Java model

I am a fan of Maven and use it as my default tool to build systems. This is the POM I defined for this little benchmark:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>uk.co.jemos.tests.xml</groupId>
    <artifactId>large-xml-parser</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>large-xml-parser</name>
    <url>http://www.jemos.co.uk</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.jvnet.jaxb2.maven2</groupId>
                <artifactId>maven-jaxb2-plugin</artifactId>
                <version>0.7.5</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>generate</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <schemaDirectory>${basedir}/src/main/resources</schemaDirectory>
                    <includeSchemas>
                        <includeSchema>**/*.xsd</includeSchema>
                    </includeSchemas>
                    <extension>true</extension>
                    <args>
                        <arg>-enableIntrospection</arg>
                        <arg>-XtoString</arg>
                        <arg>-Xequals</arg>
                        <arg>-XhashCode</arg>
                    </args>
                    <removeOldOutput>true</removeOldOutput>
                    <verbose>true</verbose>
                    <plugins>
                        <plugin>
                            <groupId>org.jvnet.jaxb2_commons</groupId>
                            <artifactId>jaxb2-basics</artifactId>
                            <version>0.6.1</version>
                        </plugin>
                    </plugins>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>2.3.1</version>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <mainClass>uk.co.jemos.tests.xml.XmlPullBenchmarker</mainClass>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.2</version>
                <configuration>
                    <outputDirectory>${project.build.directory}/site/downloads</outputDirectory>
                    <descriptors>
                        <descriptor>src/main/assembly/project.xml</descriptor>
                        <descriptor>src/main/assembly/bin.xml</descriptor>
                    </descriptors>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.5</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>uk.co.jemos.podam</groupId>
            <artifactId>podam</artifactId>
            <version>2.3.11.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.0.1</version>
        </dependency>
        <!-- XML binding stuff -->
        <dependency>
            <groupId>com.sun.xml.bind</groupId>
            <artifactId>jaxb-impl</artifactId>
            <version>2.1.3</version>
        </dependency>
        <dependency>
            <groupId>org.jvnet.jaxb2_commons</groupId>
            <artifactId>jaxb2-basics-runtime</artifactId>
            <version>0.6.0</version>
        </dependency>
        <dependency>
            <groupId>org.codehaus.woodstox</groupId>
            <artifactId>stax2-api</artifactId>
            <version>3.0.3</version>
        </dependency>
    </dependencies>
</project>

 

Just few things to notice about this pom.xml.

  • I use Java 6, since starting from version 6, Java contains all the XML libraries for JAXB, DOM, SAX and STax. 
  • To auto-generate the domain model classes from the XSD schema, I used the excellent maven-jaxb2-plugin, which allows, amongst other things, to obtain POJOs with toString, equals and hashcode support.

I have also declared the jar plugin, to create an executable jar for the benchmark and the assembly plugin to distribute an executable version of the benchmark. The code for the benchmark is attached to this post, so if you want to build it and run it yourself, just unzip the project file, open a command line and run:

$ mvn clean install assembly:assembly

This command will place *-bin.* files into the folder target/site/downloads. Unzip the one of your preference and to run the benchmark use (-Dcreate.xml=true will generate the XML files. Don't pass it if you have these files already, e.g. after the first run):

$ java -jar -Dcreate.xml=true large-xml-parser-1.0.0-SNAPSHOT.jar

Creating the test data

To create the test data, I used PODAM, a Java tool to auto-fill POJOs and JavaBeans with data. The code is as simple as:

JAXBContext context = JAXBContext
                .newInstance("xml.integration.jemos.co.uk.large_file");

        Marshaller marshaller = context.createMarshaller();
        marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
        marshaller.setProperty(Marshaller.JAXB_ENCODING, "UTF-8");

        PersonsType personsType = new ObjectFactory().createPersonsType();
        List<PersonType> persons = personsType.getPerson();
        PodamFactory factory = new PodamFactoryImpl();
        for (int i = 0; i < nbrElements; i++) {
            persons.add(factory.manufacturePojo(PersonType.class));
        }

        JAXBElement<PersonsType> toWrite = new ObjectFactory()
                .createPersons(personsType);

        File file = new File(fileName);
        BufferedOutputStream bos = new BufferedOutputStream(
                new FileOutputStream(file), 4096);

        try {
            marshaller.marshal(toWrite, bos);
            bos.flush();
        } finally {
            IOUtils.closeQuietly(bos);
        }

 

The XmlPullBenchmarker generates three large XML files under ~/xml-benchmark:

  • large-person-10000.xml (Approx 3M)
  • large-person-100000.xml (Approx 30M)
  • large-person-1000000.xml (Approx 300M)

Each file looks like the following:


<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<persons xmlns="http://uk.co.jemos.integration.xml/large-file">
    <person active="false">
        <firstName>Ult6yn0D7L</firstName>
        <lastName>U8DJoUTlK2</lastName>
        <address1>DxwlpOw6X3</address1>
        <address2>O4GGvxIMo7</address2>
        <postCode>Io7Kuz0xmz</postCode>
        <city>lMIY1uqKXs</city>
        <country>ZhTukbtwti</country>
    </person>
    <person active="false">
        <firstName>gBc7KeX9Tn</firstName>
        <lastName>kxmWNLPREp</lastName>
        <address1>9BIBS1m5GR</address1>
        <address2>hmtqpXjcpW</address2>
        <postCode>bHpF1rRldM</postCode>
        <city>YDJJillYrw</city>
        <country>xgsTDJcfjc</country>
    </person>

    [..etc]

</persons>

Each file contains 10,000 / 100,000 / 1,000,000 <person> elements.

The running environments

I tried the benchmarker on three different environments:

  • Ubuntu 10, 64-bit running as Virtual Machine on a Windows 7 ultimate, with CPU i5, 750 @2.67GHz and 2.66GHz, 8GB RAM of which 4GB dedicated to the VM. JVM: 1.6.0_25, Hotspot
  • Windows 7 Ultimate, hosting the above VM, therefore with same processor. JVM, 1.6.0_24, Hotspot
  • Ubuntu 10, 32-bit, 3GB RAM, dual core. JVM, 1.6.0_24, OpenJDK

The XML unmarshalling

To unmarshall the code I used three different strategies:

  • Pure JAXB
  • STax + JAXB
  • Woodstox + JAXB

Pure JAXB unmarshalling

The code which I used to unmarshall the large XML files using JAXB follows:

private void readLargeFileWithJaxb(File file, int nbrRecords)
            throws Exception {

        JAXBContext ucontext = JAXBContext
                .newInstance("xml.integration.jemos.co.uk.large_file");
        Unmarshaller unmarshaller = ucontext.createUnmarshaller();

        BufferedInputStream bis = new BufferedInputStream(new FileInputStream(
                file));

        long start = System.currentTimeMillis();
        long memstart = Runtime.getRuntime().freeMemory();
        long memend = 0L;

        try {
            JAXBElement<PersonsType> root = (JAXBElement<PersonsType>) unmarshaller
                    .unmarshal(bis);

            root.getValue().getPerson().size();

            memend = Runtime.getRuntime().freeMemory();

            long end = System.currentTimeMillis();

            LOG.info("JAXB (" + nbrRecords + "): - Total Memory used: "
                    + (memstart - memend));

            LOG.info("JAXB (" + nbrRecords + "): Time taken in ms: "
                    + (end - start));

        } finally {
            IOUtils.closeQuietly(bis);
        }

    }

 

The code uses a one-liner to unmarshall each XML file:

JAXBElement<PersonsType> root = (JAXBElement<PersonsType>) unmarshaller
                    .unmarshal(bis);

 

I also accessed the size of the underlying PersonType collection to "touch" in memory data. BTW, debugging the application showed that all 10,000 elements were indeed available in memory after this line of code.

JAXB + STax

With STax, I just had to use an XMLStreamReader, iterate through all <person> elements, and pass each in turn to JAXB to unmarshall it into a PersonType domain model object. The code follows:

 

       // set up a StAX reader
        XMLInputFactory xmlif = XMLInputFactory.newInstance();
        XMLStreamReader xmlr = xmlif
                .createXMLStreamReader(new FileReader(file));

        JAXBContext ucontext = JAXBContext.newInstance(PersonType.class);

        Unmarshaller unmarshaller = ucontext.createUnmarshaller();

        long start = System.currentTimeMillis();
        long memstart = Runtime.getRuntime().freeMemory();
        long memend = 0L;

        try {
            xmlr.nextTag();
            xmlr.require(XMLStreamConstants.START_ELEMENT, null, "persons");

            xmlr.nextTag();
            while (xmlr.getEventType() == XMLStreamConstants.START_ELEMENT) {

                JAXBElement<PersonType> pt = unmarshaller.unmarshal(xmlr,
                        PersonType.class);

                if (xmlr.getEventType() == XMLStreamConstants.CHARACTERS) {
                    xmlr.next();
                }
            }

            memend = Runtime.getRuntime().freeMemory();

            long end = System.currentTimeMillis();

            LOG.info("STax - (" + nbrRecords + "): - Total memory used: "
                    + (memstart - memend));

            LOG.info("STax - (" + nbrRecords + "): Time taken in ms: "
                    + (end - start));

        } finally {
            xmlr.close();
        }

    }

 

Note that this time when creating the context, I had to specify that it was for the PersonType object, and when invoking the JAXB unmarshalling I had to pass also the desired returned class type, with:

JAXBElement<PersonType> pt = unmarshaller.unmarshal(xmlr,
                        PersonType.class);

Note that I don't to anything with the object, just create it, to keep the benchmark as truthful and possible by not introducing any unnecessary steps.

JAXB + Woodstox

With Woodstox, the approach is very similar to the one used with STax. In fact Woodstox provides a STax2 compatible API, so all I had to do was to provide the correct factory and...bang! I had Woodstox under the cover working.

    private void readLargeXmlWithFasterStax(File file, int nbrRecords)
            throws FactoryConfigurationError, XMLStreamException,
            FileNotFoundException, JAXBException {

        // set up a Woodstox reader
        XMLInputFactory xmlif = XMLInputFactory2.newInstance();
        XMLStreamReader xmlr = xmlif
                .createXMLStreamReader(new FileReader(file));

        JAXBContext ucontext = JAXBContext.newInstance(PersonType.class);

        Unmarshaller unmarshaller = ucontext.createUnmarshaller();

        long start = System.currentTimeMillis();
        long memstart = Runtime.getRuntime().freeMemory();
        long memend = 0L;

        try {
            xmlr.nextTag();
            xmlr.require(XMLStreamConstants.START_ELEMENT, null, "persons");

            xmlr.nextTag();
            while (xmlr.getEventType() == XMLStreamConstants.START_ELEMENT) {

                JAXBElement<PersonType> pt = unmarshaller.unmarshal(xmlr,
                        PersonType.class);

                if (xmlr.getEventType() == XMLStreamConstants.CHARACTERS) {
                    xmlr.next();
                }
            }

            memend = Runtime.getRuntime().freeMemory();

            long end = System.currentTimeMillis();

            LOG.info("Woodstox - (" + nbrRecords + "): Total memory used: "
                    + (memstart - memend));

            LOG.info("Woodstox - (" + nbrRecords + "): Time taken in ms: "
                    + (end - start));

        } finally {
            xmlr.close();
        }

    }

 

Note the following line:

XMLInputFactory xmlif = XMLInputFactory2.newInstance();

Where I pass in a STax2 XMLInputFactory. This uses the Woodstox implementation.

 

The main loop

Once the files are in place (you obtain this by passing -Dcreate.xml=true), the main performs the following:

System.gc();
            System.gc();

            for (int i = 0; i < 10; i++) {

                main.readLargeFileWithJaxb(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-10000.xml"), 10000);
                main.readLargeFileWithJaxb(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-100000.xml"),
                        100000);
                main.readLargeFileWithJaxb(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-1000000.xml"),
                        1000000);

                main.readLargeXmlWithStax(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-10000.xml"), 10000);
                main.readLargeXmlWithStax(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-100000.xml"),
                        100000);
                main.readLargeXmlWithStax(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-1000000.xml"),
                        1000000);

                main.readLargeXmlWithFasterStax(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-10000.xml"), 10000);
                main.readLargeXmlWithFasterStax(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-100000.xml"),
                        100000);
                main.readLargeXmlWithFasterStax(new File(OUTPUT_FOLDER
                        + File.separatorChar + "large-person-1000000.xml"),
                        1000000);
            }

 

It invites the GC to run, although as we know this is at the GC Thread discretion. It then executes each strategy 10 times, to normalise RAM and CPU consumption. The final data are then collected by running an average on the ten runs.

The benchmark results for memory consumption

Here follow some diagrams which show memory consumption across the different running environments, when unmarshalling 10,000 / 100,000 / 1,000,000 files.

You will probably notice that memory consumption for STax-related strategies often shows a negative value. This means that there was more free memory after unmarshalling all elements than there was at the beginning of the unmarshalling loop; this, in turn, suggests that the GC ran a lot more with STax than with JAXB. This is logical if one thinks about it; since with STax we don't keep all objects into memory there are more objects available for garbage collection. In this particular case I believe the PersonType object created in the while loop gets eligible for GC and enters the young generation area and then it gets reclamed by the GC. This, however, should have a minimum impact on performance, since we know that claiming objects from the young generation space is done very efficiently.

Summary for 10,000 XML elements

Ram-usage-summary-10000
 
 

Summary for 100,000 XML elements

Ram-usage-summary-100000

Summary for 1,000,000 XML elements

Ram-usage-summary-1000000

The benchmark results for processing speed

Results for 10,000 elements

Time-taken-summary-10000

Results for 100,000 elements

Time-taken-summary-100000

Results for 1,000,000 elements

Time-taken-summary-1000000

Conclusions

The results on all three different environments, although with some differences, all tell us the same story:

  • If you are looking for performance (e.g. XML unmarshalling speed), choose JAXB
  • If you are looking for low-memory usage (and are ready to sacrifice some performance speed), then use STax.

My personal opinion is also that I wouldn't go for Woodstox, but I'd choose either JAXB (if I needed processing power and could afford the RAM) or STax (if I didn't need top speed and was low on infrastructure resources). Both these technologies are Java standards and part of the JDK starting from Java 6.

Resources

Benchmarker source code

Benchmarker executables:

Data files:

 

From http://tedone.typepad.com/blog/2011/06/unmarshalling-benchmark-in-java-jaxb-vs-stax-vs-woodstox.html

Published at DZone with permission of Marco Tedone, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Oleg Varaksin replied on Mon, 2011/06/27 - 2:11am

Great post. Thanks for the useful info.

 One off-topic question - What tool did you use for graphics to visualize three results?

Seb Cha replied on Mon, 2011/06/27 - 5:33am

I'm not convince by this benchmark.

You are not comparing JAXB against stax, because in each 3 tests you're using JAXB and Stax : 1) jaxb uses stax under the cover for reading the stream (There is no such thing as "pure JAXB client") 2) use of default JVM stax implementation 3) use of woodstox stax implementation.

This is just a comparison between two approachs: jaxb on the entire stream, or jaxb on chunks. And it is expected that calling 100,000 times the jaxb unmarshaller is slower than calling it 1 time.

 

Marco Tedone replied on Mon, 2011/06/27 - 2:54pm in response to: Oleg Varaksin

Hi Oleg, I used Excel 2010 graphs.Glad you found the article useful

 

 

Marco Tedone replied on Mon, 2011/06/27 - 2:58pm in response to: Seb Cha

Seb, you are quite right when you say I don't compare JAXB and STax as technologies, but rather as approaches. My article is not about benchmarking the Java API (for that the excellent engineers at [now] Oracle have already done the job).

My post is entirely about benchmarking the memory consumption and processing speed if one was to use JAXB to unmarshall the entire XML document versus an iterator oriented approach (e.g. with Stax and Woodstox). 

The post does not want to give absolute numbers about either JAXB, STax or Woodstox, but rather about the infrastructure expenses that one would encounter when choosing amongst the different approaches. 

Apologies if you found the article misleading.

 

 

Panos Grigo replied on Fri, 2011/09/16 - 9:53am

Hi Marco,

I think that Woodstox 4.1 outperforms the others in speed.

Try to run your tests replacing only the Woodstox jars with latest version (4.1)

Paresh Shah replied on Tue, 2011/12/27 - 5:09pm

 

Thanks for the article. 

Can we assume the same conclusion for "Marshalling"? Is there any reason to think that Marshalling a large Java object base into an XML document would behave differently than above?

 

 

Erik Van Ingen replied on Wed, 2012/04/18 - 1:32pm

Excellent article, thank you very much for the work posting this, exactly what I was looking for. 

The question which I still have, is about scaling. Which one scales the best? 

 

Carla Brian replied on Wed, 2012/06/20 - 6:06pm

One benefit of the JAXB technology is that it hides the details and gets rid of the extraneous relationships in SAX and DOM JAXB classes describe only the relationships actually defined in the source schemas. - Instant Tax Solutions Scam

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.