Combining Evidence and Craft for Successful Practice: No False Dichotomies

Evidence-based (in all its various permutations) is a construct that needs to be carefully worked out.  If evidence-based practice was self-evident, we would have achieved it through the success and extension of operationalism*, but, that wasn’t to be the case.  Participating in a practice requires evidence, craft and experience combined in a way that is fraught with complexity; but improving practices of all kinds is dependent on meeting this complexity.

I recently came across two ideas from Wampold, Goodheart & Levant (Am Psychol. 2007 Sep;62(6):616-8) that go a long way to clarifying this evidence-based construct.  Their first clarification is a definition of evidence and the second is to counter the false dichotomy of evidence vs. experience.

Evidence and Inference

Evidence is not data, nor is it truth.  Evidence can be thought of as inferences that flow from data.  . . . Data becomes evidence when they are considered with regard to the the phenomena being studied, the model used to generate the data, previous knowledge, theory, the methodologies employed, and the human actors.

This is not a simple positivist conception of evidence, but reflects a complex multimodal aggregation.  In addition, I would add that the primary concern of practitioners is really the validity of the practices they are conducting.  The validity of practice is supported by evidence, but it is dependent on the use of practice in context.  We do not validate practice descriptions or practice methodologies, but rather the use of practice in its local contexts, understood by reference to phenomena, models, knowledge, theories, ex-cetera.  I’ll have to look back at validity theory to see if I can get a clearer description of this idea.

Evidence and Experience

A second insight expressed by Wampold, Goodheart & Levant is the integrative nature of evidence and experience as they relate to practice; where any opposition between experience and evidence is considered to be a false dichotomy.  The ability to use evidence is a component of practice expertise including the ability to collect and draw inferences from local data through the lens of theory and empirical evidence or in the ability to adjust practices in response to new evidence.  It’s experience and evidence and evidence as a part of experience.

Evidence and Craft

I find it somewhat serendipitous that I have been drawn into conversations involving design management and evidence-based management.  It is because I believe that the success of each one depends on the other.  The positivist agenda of running the world by science is not tenable.  The world is too complex and there are too many relevant or even distant variables for a positivist program to be sustainable.  Science cannot do it all, but neither can we be successful without science, evidence and data.  We need a bit of craft and a bit of evidence to engage in practice.  That may often include craft in the way that evidence is used and it may entail craft that is beyond evidence. It just should not draw false diochotomies between evidence and craft.

The goal of operationalism “was to eliminate the subjective mentalistic concepts . . .  and to replace them with a more operationally meaningful account of human behavior” (Green 2001, 49). “(T)he initial, quite radical operationalist ideas eventually came to serve as little more than a “reassurance fetish” (Koch 1992, 275) for mainstream methodological practice.” Wikipedia (Incomplete references noted in this article, but it seems trustworthy as I’m familiar with Koch’s work.)