In Praise of Data and Science
MIT’s Technology Review posted the article: Trusting Data, Not Intuition. The primary idea is to use controlled experiments to test ideas and comes from Ronny Kohavi of Microsoft (and formerly of Amazon). The article can be summarized as follows:
(W)hen ideas people thought would succeed are evaluated through controlled experiments, less than 50 percent actually work out. . . . use data to evaluate an idea rather than relying on . . . intuition. . . . but most businesses aren’t using these principles. . . .What’s important, Kohavi says, is to test ideas quickly, allowing resources to go to the projects that are the most helpful. . . . “The experimentation platform is responsible for telling you your baby is really ugly,” Kohavi jokes. While that can be a difficult truth to confront, he adds, the benefit to business—and also to employees responsible for coming up with and implementing ideas—is enormous.
This articles further supports my thesis that Evidence-based practice, analytics, measurement and practical experimental methodology are closely related, mutually supportive, and a natural synthesis.
In Praise of Aesthetics
I do believe that, while trusting science is an important idea, that trust should also be tempered because it is a tools for decision-making and acting, not a general method for living. A successful life of practice is a balance between the empirical and the aesthetic. You could say that aesthetics, looking at life emotionally and holistically is the real foundation of our experience and how we live life. Within that frame, it is helpful to step back reflexively and consider the use of empirical tools to benefit our experience, but without denying our aesthetic roots. Wittgenstein wrote on this (from the Stanford Encyclopedia of Philosophy article on Wittgenstein’s Aesthetics).
“The existence of the experimental method makes us think we have the means of solving the problems which trouble us; though problem and method pass one another by” (Wittgenstein 1958, II, iv, 232).
For Wittgenstein complexity, and not reduction to unitary essence, is the route to conceptual clarification. Reduction to a simplified model, by contrast, yields only the illusion of clarification in the form of conceptual incarceration (“a picture held us captive”).
What I want is to have access to the tools of science and the wisdom to know when to choose their reflexivity. What I’m against is;
the naturalizing of aesthetics—(which) falsifies the genuine complexities of aesthetic psychology through a methodologically enforced reduction to one narrow and unitary conception of aesthetic engagement.
Hans de Zwart posted a great summation of the critiques of big data and its usages. I will comment his post in 3 sections, the good, the bad, and the ugly.
I like this Dataist’s Venn Diagram on Data Science combining Hacking Skills (innovating with technology), Math and Stat knowledge, with core expertise.
Data Science Venn Diagram
Data science is then a combination of an expert in Machine Learning (to deal with the massive amounts of data being generated), traditional research expertise (to process and analyze that information) and a willingness to engage in creative disciplinary innovation to bring these insights to practice (danger zone). I think this is a list of skill and knowledge needs.
Most of the naughty list is from Drew Conway’s original definition of danger zone and from George Siemens’ 10 concerns. Drew’s reason for calling it a danger zone was to warn of people who hack (innovate) with poor core knowledge and George’s concern list is mostly about data procedures getting away from our intentions. These are valid concerns, but I think they relate to statistical and measurement concerns. My take on the problem is this: due to common pedagogy, most people have a rather formulaic understanding of measurement and statistic. They know how to plug in the numbers, but they aren’t so good understanding what they are doing conceptually and what limitation are being violated. Not only is this a problem because they are operating blindly, but also because they are missing the inherent limitations that exist in their calculations. So people are blind to both the validity problems they are creating and do not have a good conceptual understanding of what their procedures are capable of doing.
Hacking skills are the most likely skill to be ignored in this diagram. This is a new area and it can’t progress without innovation. Even though innovation is widely celebrated, managers do not really like it because the very idea of management is wraped up in the idea of control (with or without command). In a standardized economy people were interchangeable and must conform to existing processes. Today the world, even the data world, changes to quickly for standardized process in most circumstances. To respond, management must be reformed to its core purposes and I don’t think the discipline is ready to tread these waters.
This view is against Chris Andersen’s view of The End of Theory in favor of dimensionally agnostic statistics. Google is just a tool. Popularity does not equal quality or relevance as was pointed out with recent concerns that organizations spamming google results. As the Sloan article Hans quotes states:
Information Must Become Easier to Understand and Act Upon