HornetQ Stings the Competition in Peer-Reviewed Benchmarks
Software vendors can say what they want about their products, but most developers want proof, which is why people want to test different products against the competition. Researchers at UT Darmstadt have been testing the performance of enterprise message-oriented middleware servers based on JMS (Java Messaging Service) for several years, and this year the SPECjms2007 benchmark record was broken by JBoss' open source enterprise messaging system, HornetQ.
SPECjms2007 is a peer-reviewed, industry standard benchmark for JMS messaging systems. It measures standard workload and performance metrics for competitive product comparisons. The benchmark provides a framework for in-depth analysis as well. All components that make up the application environment are taken into account, including hardware, JMS server software, JVM software, database software if used for message persistence, and the system network. The benchmark also provides two key scalability metrics: horizontal and vertical topologies.![]()
The benchmark pitted HornetQ 2.0 against Apache ActiveMQ 5.3, which has a reputation for being fast. Each messaging system ran on an IBM x3850 system with the same OS setup. When the dust settled, HornetQ had blown ActiveMQ out of the water. HornetQ not only beat ActiveMQ, but it also broke the record for all SPECjms2007 benchmarks. HornetQ 2.0's scores were up to 307% higher than previously published SPECjms2007 results.
In a recent interview with Tim Fox, the project lead of HornetQ, he told DZone the two main areas that contribute to HornetQ's high-performance. First, there's Netty, which is a low‑level Java NIO library written by the author of Apache MINA. Then there's real "secret weapon" behind HornetQ. Fox explains:
"This system is a very high‑performance journal, which is basically written in 99‑percent Java. But we also have a small layer of native code that's accessed by JNI. And what this does is, when you're running on Linux, it detects that you're running on Linux, and then it automatically enables a bit of native code, which allows us to pass into Linux asynchronous file IO. And basically, what this does is it allows us to get much faster persistence than would be possible in pure Java."
Netty Architecture
The final version of HornetQ 2.0 was released about one month ago. You can learn more about HornetQ in DZone's podcast interview with the project's lead developer, Tim Fox. JBoss says it has already started working on HornetQ 2.1, which will include another round of performance enhancements.






Comments
Rob Davies replied on Sat, 2010/02/13 - 2:47am
Which leaves me wondering, why didn't HornetQ do much better ? ;)
Rob Davies Fuse Source
Tim Fox replied on Sat, 2010/02/13 - 3:34am
Rob,
I very much doubt that's the reason for the big difference in performance.
The real reason HornetQ is faster is due to it's highly tuned journal that takes advantage of AIO when running on Linux. Standard pure Java journals can't touch it.
I wouldn't take it too personally, it's not just ActiveMQ, HornetQ persistence is faster than any other messaging systems we have benchmarked against internally. I wonder if they all will come up with "bugs" that explain their poor performance too?
I think ActiveMQ is a generally sound messaging system, many people use it and are happy with it. Our goal here is not to rubbish ActiveMQ.:ActiveMQ guys have come up with some cool stuff, e.g. STOMP which we also support in HornetQ.
The reality though is HornetQ simply performs better.
Artur Biesiadowski replied on Sat, 2010/02/13 - 6:09am
I wonder how it translated to absolute numbers. How many messages per second can be distributed in persistent/non-persistent cases, point-to-point or point-to-10-subscribers cases ?
I know it depends on many things, especially message size, but in case of small payloads (100-1000 bytes) are we talking about 10k, 100k or 1M messages per second ?
Kai Sachs replied on Sat, 2010/02/13 - 10:15am
in response to:
Artur Biesiadowski
The workload is a mix in terms of msg. size, (non-) transactional, point-to-point, pub/sub etc.
The documentation of the benchmark is available under:
- http://www.spec.org/jms2007/docs/
A detailed description of the vertical & horizontal workload is provided in:
- http://www.dvs.tu-darmstadt.de/publications/pdf/WorkloadCharSPECjms2007.pdf
- http://www.dvs.tu-darmstadt.de/publications/pdf/PerfEvalJ-SPECjms2007.pdf
Salahuddin Chal... replied on Sat, 2010/02/13 - 11:57am
It is exactly what we want! Thanks! :)
Speaking about benchmark, I am wondering how well does it compare to RabbitMQ in term of scaling. Since Erlang, the language behind it, are quite a hype recently.
Cheers,
S. Chalermthai
Rob Davies replied on Sun, 2010/02/14 - 7:52am
in response to:
Tim Fox
http://svn.apache.org/viewvc?view=revision&revision=901300
Btw - we liked the idea of using direct I/O on linux systems so much we added it to ActiveMQ 6 a few months ago.
We like healthy competition - keep up the good work!
cheers,
Rob
Henri Gomez replied on Tue, 2010/02/16 - 1:44pm
Michaël Mathieu replied on Fri, 2011/02/04 - 9:05am
in response to:
Salahuddin Chalermthai
Did you found out any benchmarks with RabbitMQ ?
Thanks in advance
Liezel Jandayan replied on Tue, 2012/05/22 - 9:46pm