Master Worker Pattern in JMS
A user asked a recent question on SO for their "application which needs to process huge file stored in DB in multiple rows." Here were the steps he wanted to trigger (he didn't know how to get step 2 going):
1. One part of the system receives the file in chunks and keeps it in the DB.
2. The next step logically splits this file into N partitions and sends the metadata of the partitions as JMS Messages.
3. And then the JMS clients process their part of the file, and when they are finished they update the status of the task on another table in the DB.
There were a couple answers to this question. One user suggested the Blackboard Pattern. Javalobby has an article about JGroups that tells how to implement a similar model. One user suggested that.
What would you use?
1. One part of the system receives the file in chunks and keeps it in the DB.
2. The next step logically splits this file into N partitions and sends the metadata of the partitions as JMS Messages.
3. And then the JMS clients process their part of the file, and when they are finished they update the status of the task on another table in the DB.
There were a couple answers to this question. One user suggested the Blackboard Pattern. Javalobby has an article about JGroups that tells how to implement a similar model. One user suggested that.
What would you use?
Tags:
Comments
Mark Unknown replied on Fri, 2012/12/21 - 10:27am
This sounds like a problem for "map/reduce" solution.