(P)ragmatism is that method of reflexion which is guided by constantly holding in view its purpose and the purpose of the ideas it analyzes . . . a method of reflexion having for its purpose to render ideas clear. Wikipedia the Pragmatic Maxim CS Peirce CS Peirce

Some measurement specialists argue that adding value implications and social consequences to the validity framework unduly burdens the concept. However, it is simply not the case that values are being added to validity in this unified view. Rather, values are intrinsic to the meaning and outcomes of the testing and have always been . . . This makes explicit what has been latent all along, namely, that validity judgements are value judgments. (emphasis in original) (Messick, 1995) Samuel Messick

I’ve found Agile interesting because it puts purpose and values front and center in practice; a natural example of a paradigm change toward pragmatism in action. As Quine argued in Two Dogmas of Empiricism, the empirical world only impinges on our experiences along the edges, but to accommodate this edge experience, we often must re-evaluate our belief systems down to the core. It is easy to ignore these edge experiences and one reason why a paradigm change of core beliefs can be hard. I think Agile comes from this core reevaluation, but I don’t think it would be well served if the measurement paradign didn’t match the nature of the Agile Paradigm.

Assessment can be a method of reflection that constantly holds its purpose and values in view and I believe Pragmatic Assessment can clarify Agile for practice and achieve the purpose of expectations management. One way it does this is by creating an operational definition of Agile that looks not only at the score, but observes the process of how the score and how Agile is achieved. Since Agile can be implemented over time, a Maturity style assessment could help, especially if it spells out how the underlying paradigm changes through the Maturity Process. This is how assessment can clarify Agility, achieve the expectations management that you intend and spell out much of the metaphysics of Agile.

Assessments should not be written in stone, but should change as guided by a scientific inquiry process with a focus on consequences in the use of the assessment. But the purpose of the assessment remains central. François Chollet’s recent piece, The impossibility of intelligence explosion, is an example of what can happen if purpose is ignored. Alfred Benit first developed his IQ test to identify French students for an early version of special education. Later, in the rapid mobilization before WWI, the US Army adapted his ideas to help with their forced choice to identify new recruits for leadership positions. But it is something completely different to use such a test to define a construct of intelligence as we naturally understand it. It leads not only to inapproariate uses by people like Charles Murray, but also confounds our understanding of concepts like Artificial Intelligence. An operational definition of intelligence as scored by an IQ test will expose how different this psychometric definition is from our everyday understanding of being smart. Purpose, sometimes, is everything and appropriately operationalizing a fuzzy concept can really help to clarify our ideas.

Note: The above  post was part of a conversation on medium with Stefan Wolpers about assessment in Agile frameworks, but I believe it represents assessment as a part of a process of pragmatic reflection that applies to most if not all educational measurement. Just as we wrongly think of assessment as measuring chunks of knowledge rather than aspects of performance, educational measurement measures cognitive processes.