A Caveat to the Use of Theory

I must add a caveat to my last post.  I use theory in a pragmatic instrumentalist way, not in an absolute way.  Theory does have limits, an “everything in moderation” idea.  Alex Kosulin explained the potential problems when theory becomes over extrapolated in his 1992 Introduction to Vygotsky’s Thought and Language*

Tracing the evolution of psychoanalysis, reflexology, Gestaltism, and personalism, (Vygotsky) revealed a uniform pattern to their development, an aggressive expansion in a desperate attempt to attain methodological hegemony.  The first stage in the development of each of these systems is an empirical discovery that proves to be important for the revision of the existing views concerning some specific behavioral or mental phenomena.  In the second stage . . .the initial discovery acquires a conceptual form, which expands so as to come to bear on related problems of psychology.  Even at this stage the ties between the conceptual form and the underlying empirical discovery are eroded.  The third stage is marked by the transformation of the conceptual form into an abstract explanatory principle applicable to any problem within the given discipline.  The discipline is captured by this expanding explanatory principle. . . .At the fourth stage the explanatory principle disengages itself from the subject matter of psychology and becomes a general methodology . . . at which point, Vygotsky observed – it usually collapses under the weight of its enormous explanatory claims.

In other words; theoretical contexts are important and abstraction and extrapolation has its limits.

*Kozulin, A. (1992). Vygotsky in Context, in A. Kozulin (Ed.) Though and Language: Cambridge MA, MIT Press.

Scanning Horizons: The Need for Theory in Practice

I believe that practice requires theory at a greater level than has generally been recognized in the past.  One point of view to substantiate this claim is test validity as I have previously discussed here, here, and here.  Theory is the starting point when discussing substantive and structural aspects of validity.  Also, because theory and measurement constructs are closely related, and because of a unified view of construct validity, theory (and hermeneutics) touches all aspects of validity and measurement.  Consider a practical example from my previous work in disability support services.

I was lucky enough to working for an organization as they were initiating supported employment services for the first time.  I believe the impetus for supported employment was rooted in discrimination.  Many people who were quite capability of holding down a job were forced to work in sheltered workshop settings and they proved to be very successful when given half an opportunity.  However, there was little analysis about what was going on at a deeper theoretical level.  When supported employment began serving cliental with more challenging support needs, the rate of growth and successful decreased.

I now believe that supported employment is about participation in economic activity and providing a level of accommodation that allows full participation by all individuals.  The measures needed are measures of participation, accommodation that are needed, and accommodations delivered.  Instead most descriptions of supported employment are just that, descriptions of what supported employment looks like, not how it functions.  Consider how the Department of Labor defines supported employment:

Supported employment facilitates competitive work in integrated work settings for individuals with the most severe disabilities (i.e. psychiatric, mental retardation, learning disabilities, traumatic brain injury) for whom competitive employment has not traditionally occurred, and who, because of the nature and severity of their disability, need ongoing support services in order to perform their job. Supported employment provides assistance such as job coaches, transportation, assistive technology, specialized job training, and individually tailored supervision.  DOL

This provides a description of what supported employment and supported workers look like, but it does not describe how services function to enable individuals to participate in the economy.

In order to develop a functional understanding of practice, theory is necessary. Once you define things in functional relationships, it’s possible to develop relevant measures and to use experimental methods to get to the bottom of functional relationships and improve the processes involved.

Scanning Horizon: What Institutional and Pedagogical Forms are Needed for Lifelong Learning

I would make it mandatory for executives to keep on learning throughout their careers . . ..  Dan Ariely (2009) in Technology Review

Lifelong learning has been a nebulous concept.  There are many different institutions and pedagogical forms for traditional k-12 and university education, but what exactly are the institutions and pedagogical forms for continuous lifelong learning beyond this tradition.  By what institutional move would Professor Ariely use to make learning mandatory for executives.

I believe that career / personal professional development should mean the integration of many forms of learning into a personal network of learning environments.  College courses, professional publications (oriented to practitioners), communities of practice, social media networks, coaching, mentoring, performance support,  etc. . .. All of these forms can be potentially important!  While most of these forms of learning currently exist, we lack the institutional structures to integrate them into a coherent whole.

Another aspect that is missing from lifelong learning is learning that is strongly contextualized.  I have previously written about the contextual requirements of practice-based learning.  There is very little of this type of learning / instruction available.  What might it look like.  One possibility is to imagine learning goals that are an integrated part of strategic planning.  Another is to imagine learning objectives and resources that are integrated and integral to project planning and project milestones.

Strong robust networks of peers, mentor and every type of learning relationship are also need as a part of lifelong learning.  I have also previously written on this topic.

Scanning Horizons: Data Driven Practice

Summary: Without standards, data becomes more important in guiding practice.  Construct Measurement is also important to generate data that is relevant and of high quality relating to practice.

I proposed that management education and practice should become much more experimental and data-driven in nature — and I can tell you that it is amazing to realize how little business know and understand how to create and run experiments or even how to look at their own data!  We should teach the students, as well as executives, how to conduct experiments, how to examine data, and how to use these tools to make better decisions.  Dan Ariely (2009) in Technology Review

A second horizon exists where measurement is needed, but no standards exist.  Without standards, experimental methodology is another reasonable path.  Important tasks are to design measurement and to develop a clear logic leading from experimental results to improved practice.  Six sigma is an example of this kind of approach.  What can make it perplexing is the difficulties in developing measures when practice is rooted in social variables.  This calls for building measures based on complex educational, social or psychological constructs on which to base experiments.  Some companies that follow a balanced scorecard approach could be improved by the better measurement of relevant constructs.

Scanning Horizons: Standardization and Practice

Summary Where standards exist (including its close cousin, evidence-based practice), evaluation instruments leading to change projects is a good way to reduce complexity and improve practice.

My last 2 jobs involved standardization.  One was designing consultative services designed to assist health care organizations in complying with HIPAA security standards.  The other job was to provide teacher professional development where the primary concern centered on improving student proficiency scores on tests of graduation standards.  The purpose of standardization is to reduce variability, improve quality, facilitate measurement and change practices.

Achieving compliance is generally achieved by standardizing practice.  The HIPAA measures I constructed were designed to use published HIPAA standards to assess an existing level of practice and to provide a logical path in designing projects to bring practices into compliance.  The problem with educational standards is that state performance standards are not directed toward practice (which is easily measured and easily brought in compliance), they are directed toward performance (Which is easily measured, but whose variability is subject to a wide variety of social, environmental, cognitive and developmental variables).  It would make better sense to have two sets of standards: one for student performance and one for teaching practice.  Practice standards would specify the materials and instructional sequences that would be necessary to achieve compliance on performance assessments.

Why hasn’t this been done; there are 4 things that make this difficult:

  • Education is not under the control of the federal government and the undertaking (if it reflects all 12 years of education) may be too large an undertaking for individual states and municipalities.
  • Materials would have to be standardized and there are many textbook providers, all in competition with each other.  Choosing one set of materials would put others out of business.
  • Standardization of practice would reduce teacher’s control, and teachers are not ready to accept this and to think of themselves as standards implementers.
  • Standardized education would work for an industrialized economy, but not for a creative economy.  Just like the Army is always being reorganized to fight the last war, education is often organized to serve the last age.  Still this would be a much more efficient way to organize 40 to 50% of educational needs that could be standardized.

Well, I found teacher professional development to be a somewhat frustrating endeavor.  Standardization is not the only way to go and you could even say this was teaching to the test.  I would fully support moving to something other than a high stakes testing regime, but if your going to use these tests, if the schools and teachers are strongly focussed on them; they imply a standardization process and anything else is a mismatch.  The organization I worked for was a master of standardization in educational practice, but the program they tried to implement was not in this ballpark.

Scanning Horizons of Passion: Finding an Epistemological Identity

After watching Randy Komisar on Academic Earth, it seems a good idea to define the horizons of my passions.  The idea is to understand what values and beliefs are driving you, what makes you want to turn the wheel and take opportunities, what passions count, because life only makes sense in the review mirror, not in the windshield.  I’ll start in the rearview mirror by analyzing beliefs drawn from past jobs, my graduate training and my dissertation studies over the next couple of posts.

An update on my view of Connectivism (CCK08)

During my last post I realized that I needed to update my view on this theory.  Constructivism is a sound theory on human learning except for one thing, it really didn’t change practice all that much.  Its central insight (especially in the Vygotskian version) is that learning and knowledge are social, but that insight changed very little in educational practice where it should have turned practice on its head.  I have come to believe that this is because there was no adequate model of social learning.  Even Vygotskian ideas like the ZPD (zone of proximal development) limits the social model to teacher to student or peer to peer interactions.  Connectivism provides an adequate social model as a wide network.  This is a true social model that shows how learning is expanded by expanding one’s network.

Learning Needs Social Innovation, not just Technical Innovation

Reading about e-learning and social media, I get the feeling that people are trying to solve learning issues with technical applications.  While I believe that technology is a key enabler, learning is social at its core.  That means social innovation should come first.  Social media can be a great enabler, if its application is designed to facilitate interaction where social change has already taken place or at least where the ground is fertile for social change.

Here’s an example:

An individualistic idea of schooling led to a university model where people went to school to get knowledge into their heads and then went out into the world to practice and use that knowledge.  But not only is learning not anywhere near finished when you leave school, to be successful in practice many people need to learn everyday.  In short, the learning is never done!

Yes, we need knowledge from schools, but even more important we need a learning network.  This was my take-away from last falls connectivism course (CCK08).  Providing students with a network of knowers is more important than providing them with knowledge.  While many professors may maintain contact with graduates, what is needed is more.  It’s the expectation that graduates will leave school with a strong learning and practice network that includes strong bond to ties graduates back to their original contexts of learning and to ties schools to rich fields of practice and practitioners.  It’s a two way street.  Now in this type of context, social media can be a real enabler because it is focused on facilitating dynamic social innovation.

A Framework for Action with Reflection: Measurement with Validity

My recent reading reinforce a need for validation thinking.

First, David Jones’ blog in a recent post considers a quote about the problems stemming from theory without action (idealism) and action without “philosophical reflection” (mindlessness).

Second, a reassertion of a limited (but still robust) neo-positivism by Philip Tetlock in Expert Political Judgment.

My Response: Let’s begin by refine Jones quote to read as follows:

Theory without the measurement of empirical correlates to justify action will lead to actions that are based only on biased judgment; and action without broad reflection (even if that action is supported by logical empirical correlates) just as surely leads to action that is based on unrecognized biased judgments.

This is also the basic argument that stands between logical positivism and radical social constructivism (at least in its straw men forms).  I believe that it argues for a dialectic type of response that is `implied by Jones quote.  One of the problems in the positivism / constructivism argument is that both sides spin complex arguments that fail at parsimony, that is, they become needlessly complex in the attempt to justify their radical stances.  This is where validity thinking can serve as Occam’s Razor.

First, consider a unified idea of construct validity: measurements (at least in the real world almost always) measure constructs, not real objects.  (eg. Even if we measure real objects [like pencils] we must define pencils [as not pens or not markers] in a way that indicates that what is indeed being measured is a construct.  This is not quite idealism.  Although it is possible to distinguish between constructs (like IQ) and real objects (like pencils); we cannot operationalize the measurement of real objects without referring to a construct in some way.  So, operationalism, the logical positivist’s banning of constructs by fiat, will not hold.)

Next: The purpose of measurement is to overcome the biases of Jones’ idealism.  We don’t normally fall into idealism because we rely on scientific methodology (based in measurement) to counter bias.  But, we can’t totally escape idealism with measurement because constructs provide a place for ideas and idea bias to creep back in.  Hence the dialectic, we measure with reflection.

In Messick’s validation framework reflection looks like this.

  1. Content validity: does the way we are measuring make sense both logically and through the experience of ourselves and others.
  2. Substantive: is there a theory (empirically supported) that gives meaning to the measures taken.
  3. Structural: do the measures faithfully reproduce the tasks or processes theorized to exist in the contexts or natural settings to which you want to extrapolate.
  4. Generalizability:  Is there evidence that what you are measuring applies to other places and other times.  (Even if you’re not measuring to generalize your findings, evidence against generalizability should cause you to reflect on why the place and time matter)
  5. External: This is mostly (convergent or discriminate) criterion evidence.
  6. Consequential: Show me evidence that measuring is helping. (eg. Are “No Child Left Behind” measures helping students become better prepared for life after school; or are they at least helping to further the right wing agenda to undermine the power of the NEA.  (Opps. . . sorry . . . cynicism slipping in there)

Therefore, a framework for action with reflection is measurement with validity.

Understanding Learning by Understanding Language: It’s Growing, Not Transferring

I believe we cannot understand learning without taking into account the operation of language.  Specifically, understanding the way language operates is a reason why educators should abandon the transportation metaphor of learning (i.e. learning transfer) for a cultivation metaphor (growing the knowledge garden).

Language was at the center of philosophy during the last century.  Beginning with the linguistic turn and ending with the interpretive turn, these 20th Century movements had a profound effect on psychology and education. M.M. Bakhtin was one of the great elucidators of language that I encountered in my studies. I believe he was at heart a teacher, but his work defies easy categorization.  I recently came across this quote which emphasizes languages role as a tool of communication, intellect and thinking. (emphasis added)

Due to stratifying forces resulting from the dialogical contextual use of language; there are no “neutral” words and forms – words and forms that can belong to “no one”; language has been completely taken over, shot through with intentions and accents. For any individual consciousness living in it, language is not an abstract system of normative forms but rather a heteroglot conception of the world.  All words have the taste of a profession, a genre, a tendency, a party, a particular work, a particular person, a generation, an age group, the day and hour. Each word tastes of the context and contexts in which it has lived its socially charged life; all words and forms are populated by intentions.  Contextual overtones (generic, tendentious, individualistic) are inevitable in the word.  [Bakhtin, M.M. (1981). The Dialogical Imagination: Four Essays by M.M. Bakhtin, Austin TX: University of Texas Press. (p.293).]

All language is populated by intentions and overtones that don’t just communicate, but form a conception of the world, by a particular person and in a place and time.  The thought of learning transfer is like thinking of a person that cannot speak except by using direct quotation.  Call it a left-over from a linear behavioral conception of learning.  Imagine if we could only speak to others by using direct quotations.  We don’t do that! We compose our statements according to the context and our purposes.  Similarly, when we learn, we also compose our knowledge (much of which is language) according to the context and purpose.  We may barrow ideas from other people and other times and circumstances, but they are reformulated and grown according to the context and the purpose at hand.  This is the benefit of a cultivation metaphor over a transportation metaphor.