A Follow-up on my last post.

Are your measures valid across a range of concerns?  Improving validity will lead to improved actions, better frameworks for acting and ultimately improved performance. In example:

The turn of the century saw an increase in the expectations tied to measurement through such phenomena as “No Child Left Behind” or SAT test prep classes.  This has begun to change as colleges put less emphasis on SAT scores and I believe we’ll soon see similar changes in high stakes graduation tests.  Two observations:

• While high expectations pose difficult challenges for assessment, most of the problems that resulted in less use of assessment are in the expectations placed in specific tests not in the capabilities of assessment in general.  It’s a hermeneutic problem.  The meaning of test scores was much narrower than were the expectations for assessment; a mismatch between the meaning that was required and the meaning that the test could supply.    From a narrow psychometric perspective involving external validity, these test were valid, but from other perspective (structural or consequential validity – see the previous post) they are found wanting.
• People will still act and those actions will still require assessment and those assessments will still be made.  They will just be more casual, less observable, and even less valid that those made by high stakes tests.

Most actionable situations require a range of assessments that are valid across a range of validity concepts.  Just because some are less empirical or more qualitative does not mean they should not be considered in an appropriate mix.