Thinking about new projects. Here are the general contures of a new way of looking at validity.
- There have been criticism of Samuel Messick unified view of construct validity and Kanes Argument based approached. I have yet to accept any logical argument made against either framework, yet I am sympathetic when it is said that these frameworks are not practical administratively.
- Consider an argument made by the philosopher Karl Popper. Popper makes a distinction between justification and criticism on the way to his famous idea of fallisficationism. Just like one cannot claim that one’s theory is true through experimentation (you can only be sure of your results if they are false), so too it is precarious to justify one’s beliefs, but easy to demonstrate if their false. Justification can be seen as a next to impossible task, but criticism is more likely to be seen as true. If we respond to criticism with a desire to improve and adjust our beliefs than our beliefs will approach a closer version of what you might call truth. So, the best way to justify assessment validity is by being open to criticism; always seeking to improve through critical reasoning.
- This does not nullify Messick’s framework (Messick, 1995), but it shifts it from justification to a framework for critique and critical thinking. Messick’s framework moves from a hopelessly difficult attempt at justification and becomes a critical framework for knowledge transparency. Recent developments in philosophy have demonstrated the contingent nature and how the shape of knowledge is shaped by the form of its production. Messick’s transparent critical framework for the production of assessment knowledge is the best way to see the underlying contingencies
- Kane’s framing of validity as an argument is more suited to a critical approach than a justificationist approach. The very nature of argument sets up a 2 sided dialogue. Every argument presupposes a dialogic counter argument. If you enter into an argument you must be willing to entertain and engage with critical position. Kane’s framework is more suited to respond to critical than to depend on justification.