Mitch Pronschinske is a Senior Content Analyst at DZone. That means he writes and searches for the finest developer content in the land so that you don't have to. He often eats peanut butter and bananas, likes to make his own ringtones, enjoys card and board games, and is married to an underwear model. Mitch is a DZone Zone Leader and has posted 2576 posts at DZone. You can read more from them at their website. View Full User Profile

The Search Is Over: Integrating Solr and Hadoop in the Same Cluster to Simplify Big Data Analytics

06.02.2012
| 5767 views |
  • submit to reddit


Companies like DataStax and MapR have found a significant demand for having technologies like Hadoop and Solr easily deployable in one cluster.  Jack Norris of MapR talks about this strategy in this video.

This session addresses the biggest issue facing Big Data – Search, Discovery and Analytics need to be integrated. While creating and maintaining separate SOLR and Hadoop clusters is time consuming, error prone and difficult to keep in synch, most Hadoop installations do not integrate with SOLR within the same cluster. Find out how to easily integrate these capabilities into a single cluster. The session will also touch on some of the technical aspects of Big Data Search including how to; protect against silent index corruption that permeates large distributed clusters, overcome the shard distribution problem by leveraging Hadoop to ensure accurate distributed search results, and provide real-time indexing for distributed search including support for streaming data capture. Srivas will also share relevant experiences from his days at Google where he ran one of the major search infrastructure teams where GFS, BigTable and MapReduce were used extensively.

Download Session Video

Comments

Daniel Slazer replied on Tue, 2012/06/12 - 12:11pm

Then you will see exactly what bytes are being sent and what your 'corruption' is really caused by.

If you are going to use character readers you had better make sure that what you are getting is really characters and that the charset is the same as you expect.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.