Enterprise Integration Zone is brought to you in partnership with:

Mitch Pronschinske is a Senior Content Analyst at DZone. That means he writes and searches for the finest developer content in the land so that you don't have to. He often eats peanut butter and bananas, likes to make his own ringtones, enjoys card and board games, and is married to an underwear model. Mitch is a DZone Zone Leader and has posted 2569 posts at DZone. You can read more from them at their website. View Full User Profile

Retrieving Large Datasets in Mule ESB

08.16.2013
| 3075 views |
  • submit to reddit

Clare Cini recently wrote a useful example for performing a batch select and batch insert transfer of data between two databases with Mule ESB.  The documentation for Mule covers large dataset retrieval, but only briefly.  

The customer wanted to transfer items from one database to another by performing a batch select and then a batch insert. The ‘batch insert’ part is pretty straightforward and is done automatically by Mule when the payload is of type List. However, the batch select is mastered in a different way.
In order to retrieve all the records, we will use the Batch Manager to compute the ID ranges for the next batch of records to be retrieved. This is provided out of the box with Mule EE.

-- Clare Cini

Think of this as a useful addendum to the documentation.