Enterprise Integration Zone is brought to you in partnership with:

Mitch Pronschinske is a Senior Content Analyst at DZone. That means he writes and searches for the finest developer content in the land so that you don't have to. He often eats peanut butter and bananas, likes to make his own ringtones, enjoys card and board games, and is married to an underwear model. Mitch is a DZone Zone Leader and has posted 2576 posts at DZone. You can read more from them at their website. View Full User Profile

HornetQ Stings the Competition in Peer-Reviewed Benchmarks

02.12.2010
| 22534 views |
  • submit to reddit

Software vendors can say what they want about their products, but most developers want proof, which is why people want to test different products against the competition.  Researchers at UT Darmstadt have been testing the performance of enterprise message-oriented middleware servers based on JMS (Java Messaging Service) for several years, and this year the SPECjms2007 benchmark record was broken by JBoss' open source enterprise messaging system, HornetQ.

SPECjms2007 is a peer-reviewed, industry standard benchmark for JMS messaging systems.  It measures standard workload and performance metrics for competitive product comparisons.  The benchmark provides a framework for in-depth analysis as well.  All components that make up the application environment are taken into account, including hardware, JMS server software, JVM software, database software if used for message persistence, and the system network.  The benchmark also provides two key scalability metrics: horizontal and vertical topologies.

The benchmark pitted HornetQ 2.0 against Apache ActiveMQ 5.3, which has a reputation for being fast.  Each messaging system ran on an IBM x3850 system with the same OS setup.  When the dust settled, HornetQ had blown ActiveMQ out of the water.  HornetQ not only beat ActiveMQ, but it also broke the record for all SPECjms2007 benchmarks.  HornetQ 2.0's scores were up to 307% higher than previously published SPECjms2007 results.

In a recent interview with Tim Fox, the project lead of HornetQ, he told DZone the two main areas that contribute to HornetQ's high-performance.  First, there's Netty, which is a low‑level Java NIO library written by the author of Apache MINA.  Then there's real "secret weapon" behind HornetQ.  Fox explains:

"This system is a very high‑performance journal, which is basically written in 99‑percent Java.  But we also have a small layer of native code that's accessed by JNI.  And what this does is, when you're running on Linux, it detects that you're running on Linux, and then it automatically enables a bit of native code, which allows us to pass into Linux asynchronous file IO.  And basically, what this does is it allows us to get much faster persistence than would be possible in pure Java."


                                                                                         Netty Architecture


The final version of HornetQ 2.0 was released about one month ago.  You can learn more about HornetQ in DZone's podcast interview with the project's lead developer, Tim Fox.  JBoss says it has already started working on HornetQ 2.1, which will include another round of performance enhancements.

Comments

Rob Davies replied on Sat, 2010/02/13 - 2:47am

There's a good explanation for why HornetQ performed better than ActiveMQ on these benchmarks - ActiveMQ 5.3 had a subtle problem that occurs the Queues cursors caching mechanism. After every message dispatch from a particular Queue - it would check to see if the message store for that Queue was empty by calling a count on the BTreeIndex for Queue. This would result in the entire BTreeIndex being paged in node, by node - to count them all - which isn't the most efficient thing to do, especially for large Queue depths. Needless to say this was fixed a while ago.

Which leaves me wondering, why didn't HornetQ do much better ? ;)

Rob Davies Fuse Source

Tim Fox replied on Sat, 2010/02/13 - 3:34am

Rob,

 I very much doubt that's the reason for the big difference in performance.

The real reason HornetQ is faster is due to it's highly tuned journal that takes advantage of AIO when running on Linux. Standard pure Java journals can't touch it.

 I wouldn't take it too personally, it's not just ActiveMQ, HornetQ persistence is faster than any other messaging systems we have benchmarked against internally. I wonder if they all will come up with "bugs" that explain their poor performance too?

I think ActiveMQ is a generally sound messaging system, many people use it and are happy with it. Our goal here is not to rubbish ActiveMQ.:ActiveMQ guys have come up with some cool stuff, e.g. STOMP which we also support in HornetQ.

The reality though is HornetQ simply performs better.

Artur Biesiadowski replied on Sat, 2010/02/13 - 6:09am

I wonder how it translated to absolute numbers. How many messages per second can be distributed in persistent/non-persistent cases, point-to-point or point-to-10-subscribers cases ?

I know it depends on many things, especially message size, but in case of small payloads (100-1000 bytes) are we talking about 10k, 100k or 1M messages per second ?

Kai Sachs replied on Sat, 2010/02/13 - 10:15am in response to: Artur Biesiadowski

Normal.dotm 0 0 1 56 323 TU Darmstadt 2 1 396 12.0 0 false 21 18 pt 18 pt 0 0 false false false /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Normale Tabelle"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-ascii-font-family:Cambria; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Cambria; mso-hansi-theme-font:minor-latin;}

The workload is a mix in terms of msg. size, (non-) transactional, point-to-point, pub/sub etc.

 

The documentation of the benchmark is available under:

- http://www.spec.org/jms2007/docs/

 

A detailed description of the vertical & horizontal workload is provided in:

- http://www.dvs.tu-darmstadt.de/publications/pdf/WorkloadCharSPECjms2007.pdf

- http://www.dvs.tu-darmstadt.de/publications/pdf/PerfEvalJ-SPECjms2007.pdf

 

Salahuddin Chal... replied on Sat, 2010/02/13 - 11:57am

One feature that is quite unique to HornetQ is "scheduled message".
It is exactly what we want! Thanks! :)
Speaking about benchmark, I am wondering how well does it compare to RabbitMQ in term of scaling. Since Erlang, the language behind it, are quite a hype recently.
Cheers,
S. Chalermthai

Rob Davies replied on Sun, 2010/02/14 - 7:52am in response to: Tim Fox

Tim, HornetQ's journaling system is a lot like ActiveMQ - but is faster when just using nio. Now it might not be the only reason - but the magnitude of the difference can probably be explained by this problem:

http://svn.apache.org/viewvc?view=revision&revision=901300

Btw - we liked the idea of using direct I/O on linux systems so much we added it to ActiveMQ 6 a few months ago.

We like healthy competition - keep up the good work!

cheers,

Rob

Henri Gomez replied on Tue, 2010/02/16 - 1:44pm

I currently do some benchmarks/tests on JMS brokers, IBM Websphere Mq, Active Mq and HornetQ. Pure speed in non persistant/durable mode is only a part of "real life" applications. BTW, I'll be very interested in AMQ 6.0 preview, is it available or "buildable thru SVN/Maven" ? Regards and yes competition is great

Michaël Mathieu replied on Fri, 2011/02/04 - 9:05am in response to: Salahuddin Chalermthai

Did you found out any benchmarks with RabbitMQ ?

 

Thanks in advance

Liezel Jandayan replied on Tue, 2012/05/22 - 9:46pm

So, what if it's true that Java and C are becoming co-leaders with respect to overall usage? Isn't C so old, so un-cool? -Theodore Stroukoff

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.