NoSQL Zone is brought to you in partnership with:

Jim Moscater is a DZone Tech Curator. He's always on the hunt for the most interesting developer content in the oceanic web. Jim has posted 50 posts at DZone. View Full User Profile

MongoDB Finds A Major Adopter In Craigslist

05.16.2011
| 22724 views |
  • submit to reddit

 MongoDB recently gained another adopter (possibly its largest yet).

The NoSQL data store is now being used to archive billions of records at Craigslist, the popular classifieds and job posting community that serves 570 cities in 50 countries.

Every post in the history of the site was previously held in a large MySQL cluster. Since Craigslist had a variety of database needs moving forward, ranging from wanting to add new machines without downtime to routing around dead machines without clients failing, the development team decided to initiate a major migration to a NoSQL solution. Mongo DB was the solution they chose.



Here are some basic numbers about the Craigslist MongoDB cluster from Jeremy Zawodny, one of the site's software engineers:

We’re sizing the install for around 5 billion documents. That’s from the initial 2 billion document import we need to do plus room to grow for a few years to come. Average document size is right around 2KB. (Five billion 2KB documents is 10TB of data.) We’re getting our feet wet with MongoDB so this particular task isn’t high throughput or growing in unpredictable ways.

We can put data into MongoDB faster than we can get it out of MySQL during the migration.


Watch a video where Zawodny explains the evolution of data storage at Craigslist and how MongoDB will fit into the future of the site's infrastructure. You'll also find out why Craigslist chose MongoDB over other data stores.

References
Published at DZone with permission of its author, Jim Moscater. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Emma Watson replied on Fri, 2012/03/30 - 12:24pm

there's a lot of interesting information in this post. i especially find the following interesting:

"We can put data into MongoDB faster than we can get it out of MySQL during the migration."

Swing

Carla Brian replied on Tue, 2012/05/29 - 9:11am

This is interesting. Good thing they found some way. Keep up the good work. - Steven P Delarge

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.