Enterprise Integration Zone is brought to you in partnership with:

Mitch Pronschinske is the Lead Research Analyst at DZone. Researching and compiling content for DZone's research guides is his primary job. He likes to make his own ringtones, watches cartoons/anime, enjoys card and board games, and plays the accordion. Mitch is a DZone Zone Leader and has posted 2576 posts at DZone. You can read more from them at their website. View Full User Profile

Kafka Possibly Moving to Java CRC, Akka Suggested

12.14.2012
| 6755 views |
  • submit to reddit

If you haven't heard of Kafka, a super-fast distributed message queue from LinkedIn, you probably ought to look into it, and its heavily involved Apache incubator community.

Not to go too far into minutiae, but just today there was some talk about moving to a Java-based CRC32.  Kafka is written in Scala, but the good thing is that if they used a Java CRC it wouldn't need to be modified.  

One developer also suggested using Akka Actors "as they can provide the necessary ordering guarantees, do not require thread pool tuning, and should be easy to reason about. The performance would likely be better than producer/consumer thread chains too, since chained actors often run in the same thread and avoid processor cache thrashing."

Another community member had this to say:
Akka seems a bit overkill for this (although it does have some nice properties). It would be interesting to refactor the threading in Kafka with Akka and see what kind of performance differences there are. -Kafka project page

Kafka is definitely worth checking out and getting involved with if you are a Java or Scala developer.  If not Kafka, maybe you should join another open source project.  It's getting close to the end of the year so it's time to set some new goals!  Or not .