We say that someone is literate if they can read for meaning or if they can calculate with numbers.  There is also a need for measurement literacy which is when someone can say what numbers mean when they are obtained in measures.  Although it is a bit wonkish, it’s still important to remember that measures measure constructs not the thing being measured itself.  Constructs are concepts that are thought (theoretically) to be a property of things, but they are not the things.  To understand the meaning of number obtained from measurement, it is necessary to understand the construct.  Harold Jarche recently posted 2 quotes that expressed negative opinions on measurement processes.  I believe these critiques are ill founded for two reasons:

1. Poorly designed measures should not be used to condemn measurement practice and
2. Eliminating measures often lead to politics, gut instincts and other poorly founded basis for decision-making.

In this post I would like to go deeper on this subject and show how the problem can be explained as a problem with measurement literacy.

First, Jarche quotes from Charles Green at The Trusted Advisor:

If you can measure it, you can manage it; if you can’t measure it, you can’t manage it; if you can’t manage it, it’s because you can’t measure it; and if you managed it, it’s because you measured it.

Every one of those statements is wrong. But business eats it up. And it’s easy to see why. …  The ubiquity of measurement inexorably leads people to mistake the measures themselves for the things they were intended to measure (Emphasis added).

The second quote is from Dave Snowden:

We face the challenge of meeting increasing legitimate demands for social services with decreasing real time resources. That brings with it questions of rationing, control and measurement which, however well intentioned, conspire to make the problem worse rather than better. For me this all comes back to one fundamental error, namely we are treating all the processes of government as if they were tasks for engineers rather than a complex problem of co-evolution at multiple levels (individuals, the community, the environment etc.).

I posted this response on Harold Jarche’s blog:

I agree that there are many instances of problems resulting from measures that are based on little more than common sense or tradition, but it is not helpful to base decisions on gut instincts or politics. I believe the need is to increase people’s understanding of good measurement practices and how to develop a deeper understanding of what their measurements really mean. Everyone should know if their measures are valid. In turn, that means being able to say what your measures mean, how they are relevant to practice, and how they are helping to improve practice. It’s not just for big wigs either. Front line employees need to understand how to use measurement to guide practice.

Going further, Charles Green, also said this in his post;

There’s nothing inherently wrong with measuring. Or transactions. Or markets. They’re fine things.  But undiluted and without moderating influences, they become not just a bad deal; they can be a prime cause of ruining the whole deal.

Green is not clear here, to the extent that he doesn’t explain moderating influence.  For measurement, I believe this moderation influence could be meaning or construct supported meaning.  First, Measurement can easily get out of hand because numbers can do two things.  Through constructs they can have meaning, like words, but they can also be calculated.  Being able to calculate with numbers is not the same as being able to say what they mean.  Though people often conflate the two, they are not the same.  Calculation can result in the potential for meaning, like when we calculate a Pearson’s correlation.  But, understanding meaning requires a deeper understanding of how measures were obtained, what is the theoretical construct basis for the measures as well as consequential and other basis for the validity of the measures.

Many people have a good grasp of statistics and how to calculate, but they have less knowledge about measures, validity, designing measures and measurement meaning.  Mistaking measures for the things they represent is a problem of meaning.  Having measures confound complex evolutionary problems is rooted in miss-understanding measurement meaning.  I believe many people would like to give up measurement, but that would not ultimately result in better consequences.  What is needed is better education directed toward literacy in measurement meaning.