Big Data/Analytics Zone is brought to you in partnership with:

Reimagining the way work is done through big data, analytics, and event processing, Chris is the cofounder of Successful Workplace. He believes there’s no end to what we can change and improve. Chris is a marketing executive and flew for the US Navy before finding a home in technology 17 years ago. An avid outdoorsman, Chris is also passionate about technology and innovation and speaks frequently about creating great business outcomes at industry events. As well as being a contributor for The TIBCO Blog, Chris contributes to the Harvard Business Review, Venture Beat, Forbes, and the PEX Network. Christopher is a DZone MVB and is not an employee of DZone and has posted 274 posts at DZone. You can read more from them at their website. View Full User Profile

Process is Data and Data is Process

  • submit to reddit

We’re a funny lot. For years we’ve been talking about data and process as though they were two different things. Guess what … they’re not. Process involves the activities of getting work done and data both describes process and is also the beginning and end points of work. We consume data in processes and we produce data in processes. Data kicks off a process and data ends a process. They are inseparable.

Wait a minute … does that mean digging a hole in the ground is somehow data? Does it consume data and produce data? Does data kick it off and end its process? Absolutely. The timing, location, size, depth, rationale and value of having that hole may not have been digitally captured in the past, but every factor that goes into producing and using that hole is absolutely in the realm of data. It always was.

Where Big Data Factors in

What’s happening, and this is where big data enters the story, is that we are now discovering the data that was uncaptured about hole digging and a myriad of other things. The explosion of data that we’re seeing isn’t from ‘new’ data, but from data that was previously unseen … uncaptured, untapped, unused. Invisible, but absolutely in existence.

The way to dig that hole may have been locked in the mind of that worker or given in a hole-digging class as unstructured data (from his supervisor as oral instruction). Even the hard work of digging  can also be considered data. As we move to automate and machines become a bigger part of our lives, that unstructured data quickly becomes very important. It becomes the instruction manual for how machines operate, optimize and help us dig better, smarter, more efficient holes.

Where Process is About to Change

And this is why we’re on the verge of something enormous. We call it big data, but in reality we’re at the front edge of the digitizing of our world. What tipped the scales in data’s favor were a few advances over the past two decades:

  • Fast drop in data storage and retrieval cost
  • Quick rise in computational power and techniques
  • Easy ways to share information, like the Internet
  • Rapid globalization and the need to compete against cheap labor with something smarter

Along the way, Cloud, Mobile and Social happened just to make it interesting, but that’s essentially the landscape we face today. Smart companies are embracing what’s coming next and becoming data and process powerhouses. Watch the progress of Nielsen and others and you’ll see exactly where the global economy is headed.

Published at DZone with permission of Christopher Taylor, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Hendy Irawan replied on Sat, 2013/09/28 - 8:07am

"data and process as though they were two different things. Guess what … they’re not."

Oh no. Not again. "We're funny a lot?" Maybe, but what you said is definitely funny, just not in a good way.

We're engineers, analysts. Part of our job is to recognize patterns, to distinguish things using specific criteria, to classify and breakdown problems, to decompose large unknown into understandable bits of reasonable size.

Even the most honest attempts at defining terms still causes confusion: Big Data, NoSQL, "modern" web browser, Cloud, DevOps.

And here you are: aggregating two terms that were ambiguous in the first place, then make them (in your opinion) interchangeable.

Sir, sorry but you're not helping.


The latter part of your article actually made much sense. Unfortunately the first impression is awful. (probably because you wanted the article to have catchy title or whatever. Next time, please don't ruin a good article by starting it with [unnecessary] bang, this is DZone, not gossip tabloid)

Phil Frost replied on Tue, 2013/10/01 - 4:01am

Two things:

1.) I think Hendy Irawan is being a little harsh in his review, maybe he feels short changed after being drawn in by the title and header lines.

2.) I don't accept, or, more precisely, am not willing to accept, that data is equivalent to process. However one small change to "process means data and data means process" makes me sit a little easier. Yes processes kick out data and can work on data (either directly as part of the actions taking place or as by-product/measurements of the actions/steps) but I think of processes as actions/steps that can choose whether to produce data or not, its not their main purpose.

I do like the point that the explosion of data is regarding a move toward adding measurements to existing events that makes a lot of sense that we are attempting to understand our world more.

Raging Infernoz replied on Sat, 2013/10/05 - 5:34am

Beware of expecting magic from Big Data, often more information data can make useful information harder to find i.e. the needle in the haystack problem.  A lot of process is not linear, single thread, cause and effect, and requires heuristics to do well enough; something our body, brain, and cultures have evolved to do, via time consuming trial and error.

Nicholas Taleb explains a lot of the fallacies in this idea, and other stupidity, in the book "Anti-Fragile".  Big Data is already proving to be a costly White Elephant for some applications, so expect to see more migration to more sophisticated evolutionary approaches e.g. Genetic Algorithms, and 'irrational' conceptual leaps.

Your idea probably has limited application, but the example is naively stupid, because it ignores heuristics, and if you dig too deep into it (pun intended), you will discover a lot of non-linear process, which will bury your analysis!  A lot of science is actually just evolving, good-enough, rules-of-thumb, which is why it works, with some limits (Newton's, and even Einstein's ideas have discovered scope limits), and why arrogant social 'science' is often a big fat fail.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.