recentpopularlog in

Copy this bookmark:

to read

bookmark detail

What Data Can’t Do -
Published: February 18, 2013

there are many things big data does poorly. Let’s note a few in rapid-fire fashion:

* Data struggles with the social. Your brain is pretty bad at math (quick, what’s the square root of 437), but it’s excellent at social cognition. People are really good at mirroring each other’s emotional states, at detecting uncooperative behavior and at assigning value to things through emotion.
* Data struggles with context. Human decisions are embedded in contexts. The human brain has evolved to account for this reality...Data analysis is pretty bad at narrative and emergent thinking.
* Data creates bigger haystacks. This is a point Nassim Taleb, the author of “Antifragile,” has made. As we acquire more data, we have the ability to find many, many more statistically significant correlations. Most of these correlations are spurious and deceive us when we’re trying to understand a situation.
* Big data has trouble with big (e.g. societal) problems.
* Data favors memes over masterpieces. Data analysis can detect when large numbers of people take an instant liking to some cultural product. But many important (and profitable) products are hated initially because they are unfamiliar. [The unfamiliar has to accomplish behavioural change / bridge cultural divides]
* Data obscures hidden/implicit value judgements. I recently saw an academic book with the excellent title, “ ‘Raw Data’ Is an Oxymoron.” One of the points was that data is never raw; it’s always structured according to somebody’s predispositions and values. The end result looks disinterested, but, in reality, there are value choices all the way through, from construction to interpretation.

This is not to argue that big data isn’t a great tool. It’s just that, like any tool, it’s good at some things and not at others. As the Yale professor Edward Tufte has said, “The world is much more interesting than any one discipline.”
massive_data_sets  David_Brooks  data_driven  decision_making  data  Nassim_Taleb  contrarians  skepticism  new_graduates  contextual  risks  social_cognition  self-deception  correlations  value_judgements  haystacks  narratives  memes  unfamiliarity  naivete  hidden  Edward_Tufte  emotions  antifragility  behavioral_change  new_products  cultural_products  masterpieces  EQ  emotional_intelligence 
february 2013 by jerryking
view in context