The Idea of Big Data Doesn't Live Up to the Hype
But an article in Scientific American posits that Big Data isn't yet all it's cracked up to be – and that claims of a revolutionary new way of interacting with the world around us are overblown. The conversation originates in a philosophy and music festival in the UK, where the author sat down with journalists and scientists and picked their brains.
Their most intriguing assertion is that Big Data will allow us to solve problems without necessarily understanding them. Big Data will shift the emphasis of researchers from “causation to correlation,” Cukier and Mayer-Schonberger write. “This represents a move away from always trying to understand the deeper reasons behind how the world works to simply learning about an association among phenomena and using that to get things done.” Former WIRED editor Chris Anderson made similar claims in his 2008 essay “The End of Theory.”
The focus is tending to shift towards drawing actionable conclusions from real-time data as opposed to digging through archival data to discover the "why" behind a problem that has already been pinpointed. What this means for emerging technology, business and science is that more and more, "big data" can be an inspiration for innovation.
...their rhetoric reminds me of the hype generated by the fields of chaos and its successor, complexity, which in my 1996 book The End of Science I lumped together under the term “chaoplexity.” Both fields promised that with faster computers and more sophisticated software, scientists could solve problems that had resisted analysis by stodgy old reductionist methods. Some chaoplexologists hoped to discover profound new principles governing the “self-organization” of a wide range of complex phenomena—and possibly even an “anti-entropy” force.Finally, there is a caution against hype: that even the most advanced and fastest systems have been known to fail, and that big data can't provide a cure-all. Read the original article here and let us know what you think!
These discoveries never happened, and neither have the kinds of practical advances envisioned by Cukier and Schonberger. Take genetics. The Human Genome Project was completed in 2003 in less time and for less money than had been expected because of advances in computers and other technologies. The costs of extracting and analyzing genetic data from humans and other organisms has continued to plummet.
(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)