on embodied cognition and mental simulation: a meta-theoretical comment to zwaan's treatise

4
Commentary On embodied cognition and mental simulation: A meta-theoretical comment to Zwaan’s treatise KLAUS FIEDLER * Psychology Department, University of Heidelberg, Hauptstrasse, Heidelberg, Germany Zwaan’s scholarly paper conveys a two-fold message, pointing out many interest and fascinating phenomena in the area of embodiment of cognition, but also raising a number of critical concerns and open questions. On the one hand, Zwaan stresses the important role of mental simulation processes for the explanation of the countless ideomotor and action priming effects that have intrigued students of social cognition for two decades. For example, Bargh, Chen, and Burrow’s (1996) often-cited finding that semantic prime words related to elderly people (e.g., Florida, grey, wise, etc.) elicit such remote behaviors as reduced walking speed, even after several minutes of delay, can be understood by assuming that priming elicits a rich, enduring, and multi-modal simulation process. This mental simulation is conceived as an integral aspect of all language comprehension. As it is supposed to consist of the ‘‘reenactment of perceptual, motor, and introspective states acquired during interaction with word, body, and mind’’ (Barsalou, 2008, p. 618), an active mental- simulation stage can account for priming effects carrying over to perceptual, affective, and manifest motor reactions. Comprehension is particularly dependent on mental simulation in displaced as opposed to embedded language use, when words refer to a remote or unreal situation that can only be imagined or mentally constructed. On the other hand, Zwann points to a number of unresolved issues that become apparent as soon as the embodiment phenomena are spelled out more precisely within a mental-simulation framework. What is the grain size, or degree of specificity, of the sensorimotor responses to specific linguistic stimuli? How does the mental-simulation process deal with contextual or syntactical moderators of substantial word priming (e.g., the word ‘‘stop’’ receiving completely different meanings in different contexts)? How does the process handle negations, counterfactuals, and hypotheticals? And how does mental simulation do justice to the subtleties of language, such as the perspective difference between the two utterances ‘‘He went into the room’’ (outside perspective) and ‘‘He came into the room’’ (inside perspective)? Both parts of Zwaan’s message are presented convincingly and intelligibly, and I find myself in almost perfect agreement with his analysis. I greatly appreciate the enrichment of cognitive theorizing that can be gained from the embodiment approach, the notion that sensorimotor cues, enactment cues, affective cues, and self-generated mental- simulation cues can have strong influences on memory encoding and retrieval, and knowledge resides to a considerable extent not merely in the sensorimotor areas of the central nervous system but may even be distributed in the peripheral (efferent and afferent) synapses of the somatic and autonomic nervous systems. Ignoring these distributive and multi- modal aspects of cognition would result in impoverished theories, and I do share Zwaan’s conviction that a mental- simulation framework is well suited to the theoretical analysis of embodied cognition. I also agree that Zwaan’s challenging questions can motivate fruitful future research. Yet, in spite of all my applauding responses at the semantic level, my embodied reactions, as it were, as a researcher who is himself actively and affectively involved in matters of European Journal of Social Psychology Eur. J. Soc. Psychol. 39, 1156–1159 (2009) Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/ejsp.676 *Correspondence to: Klaus Fiedler, Psychology Department, University of Heidelberg, Hauptstrasse 47-51, 69117 Heidelberg, Germany. E-mail: [email protected] Copyright # 2009 John Wiley & Sons, Ltd.

Upload: klaus-fiedler

Post on 11-Jun-2016

216 views

Category:

Documents


4 download

TRANSCRIPT

European Journal of Social Psychology

Eur. J. Soc. Psychol. 39, 1156–1159 (2009)

Published online in Wiley InterScience

(www.interscience.wiley.com) DOI: 10.1002/ejsp.676

*E

C

Commentary

On embodied cognition and mental simulation: A meta-theoreticalcomment to Zwaan’s treatise

Correspondence to: Klaus Fiedler, Psychology-mail: [email protected]

opyright # 2009 John Wiley & Son

KLAUS FIEDLER*

Psychology Department, University of Heidelberg, Hauptstrasse,Heidelberg, Germany

Zwaan’s scholarly paper conveys a two-fold message, pointing out many interest and fascinating phenomena in the area of

embodiment of cognition, but also raising a number of critical concerns and open questions. On the one hand, Zwaan

stresses the important role of mental simulation processes for the explanation of the countless ideomotor and action

priming effects that have intrigued students of social cognition for two decades. For example, Bargh, Chen, and Burrow’s

(1996) often-cited finding that semantic prime words related to elderly people (e.g., Florida, grey, wise, etc.) elicit such

remote behaviors as reduced walking speed, even after several minutes of delay, can be understood by assuming that

priming elicits a rich, enduring, and multi-modal simulation process. This mental simulation is conceived as an integral

aspect of all language comprehension. As it is supposed to consist of the ‘‘reenactment of perceptual, motor, and

introspective states acquired during interaction with word, body, and mind’’ (Barsalou, 2008, p. 618), an active mental-

simulation stage can account for priming effects carrying over to perceptual, affective, and manifest motor reactions.

Comprehension is particularly dependent on mental simulation in displaced as opposed to embedded language use, when

words refer to a remote or unreal situation that can only be imagined or mentally constructed.

On the other hand, Zwann points to a number of unresolved issues that become apparent as soon as the embodiment

phenomena are spelled out more precisely within a mental-simulation framework. What is the grain size, or degree of

specificity, of the sensorimotor responses to specific linguistic stimuli? How does the mental-simulation process deal with

contextual or syntactical moderators of substantial word priming (e.g., the word ‘‘stop’’ receiving completely different

meanings in different contexts)? How does the process handle negations, counterfactuals, and hypotheticals? And how

does mental simulation do justice to the subtleties of language, such as the perspective difference between the two

utterances ‘‘He went into the room’’ (outside perspective) and ‘‘He came into the room’’ (inside perspective)?

Both parts of Zwaan’s message are presented convincingly and intelligibly, and I find myself in almost perfect

agreement with his analysis. I greatly appreciate the enrichment of cognitive theorizing that can be gained from the

embodiment approach, the notion that sensorimotor cues, enactment cues, affective cues, and self-generated mental-

simulation cues can have strong influences on memory encoding and retrieval, and knowledge resides to a considerable

extent not merely in the sensorimotor areas of the central nervous system but may even be distributed in the peripheral

(efferent and afferent) synapses of the somatic and autonomic nervous systems. Ignoring these distributive and multi-

modal aspects of cognition would result in impoverished theories, and I do share Zwaan’s conviction that a mental-

simulation framework is well suited to the theoretical analysis of embodied cognition. I also agree that Zwaan’s

challenging questions can motivate fruitful future research. Yet, in spite of all my applauding responses at the semantic

level, my embodied reactions, as it were, as a researcher who is himself actively and affectively involved in matters of

Department, University of Heidelberg, Hauptstrasse 47-51, 69117 Heidelberg, Germany.

s, Ltd.

Embodied cognition and mental simulation 1157

research and theorizing, left me with some critical thoughts that were not expressed here as openly as in other writings by

the same author (Pecher & Zwaan, 2005).

His dualistic message—fascinating phenomena but under-developed theories lacking answers to many fundamental

questions—is characteristic of early stages of scientific research programs. It reminded me of George Kelly’s (1955)

metaphor of the creative cycle, to which I am not referring for the first time. The creative cycle involves two stages,

loosening and tightening, which roughly correspond to the random variation and selection stages in evolution theory.

Loosening consists in the generation of new ideas, gathering new findings and the invention of new methods and measures.

This part of the creative cycle must be as open-minded as possible; it must not be hindered or truncated by too restrictive

definitions and paradigmatic norms. In the second stage, tightening, the best exemplars that were generated during the

loosening stage must then be filtered out in a strict and thorough selection process. The function of strict selection is to sort

out those well-established phenomena, valid findings, and viable theories that have a chance to survive the long-term

scientific competition.

With respect to this creative cycle, Zwaan’s double message apparently conveys the message that embodiment research

has hardly overcome an early stage of loosening. The absence of sound answers to a long list of open theoretical questions

reflects a pre-tightening state of affairs. The point I want to make here is that the time ought to be ripe for entering the

tightening stage. This major developmental task of the embodiment approach calls for more precise definitions, distinct

theories, and falsifiable, Popperian hypotheses derived from theoretical models that impose distinct constraints on

embodied cognition.

The current state of the arts is reflected in three characteristics of embodiment research: Its inclusive and

‘‘imperialistic’’ nature, its reluctance to formulate constraints, and the absence of strong and falsifiable (i.e., empirically

non-empty) predictions.

INCLUSIVENESS: IT’S ALL EMBODIMENT

A typical symptom of the enthusiasm and the exploratory goals that characterize the loosening stage of a research program

is that it tries to embrace and to engulf everything. Embodiment is ubiquitous. It is supposed to be a necessary condition of

language and comprehension in general, rather than restricted to a circumscribed subset or subdomain. Many familiar

phenomena, which have been well-known all along, now turn out to be essentially embodied cognition: Priming (Stapel &

Semin, 2007), conditioning, polarity correspondence (Proctor & Cho, 2006) between semantic attributes (strong–weak),

and spatial orientation (up–down). Even the concept of mental simulation (Zwaan, 2004), which sounds like a

mentalization of the comprehension process, is now attributed to its sensorimotor component. The grounding in

sensorimotor functions is assigned an absolute, dominant status, as if it were responsible for each and every cognitive

process. To quote from Zwaan, ‘‘comprehension is not the manipulation of abstract, arbitrary, and amodal symbols, a

language of thoughts. Rather, comprehension is the generation of vicarious experiences making use of the comprehender’s

experiential repertoire.’’

The threshold for labelling phenomena as essentially embodied is very low, as evident from such examples as context

effects on word interpretation (e.g., ‘‘STOP’’), the impact of spatial arrangement of semantically related words, conceptual

combination effects (e.g., ‘‘large man’’ 6¼ ‘‘large’’þ ‘‘man’’), or mental simulation of characters in narrative fiction on

subsequent text comprehension (cf. Mar, Oatley, Hirsh, Dela Paz, & Peterson, 2006). In a different epoch, with a different

Zeitgeist that stresses the mind rather the body, all these examples could well serve to highlight higher order mental and

meta-cognitive processes that must be super-ordinate to specific sensorimotor experience.

WHERE ARE THE CONSTRAINTS?

I have always adhered to a Brunswikian vicarious-functioning approach toward flexible cue utilization in a multi-modal

complex world. Vicarious functioning means that organisms are extremely flexible and inventive in utilizing all kinds of

ecological, symbolic, figurative, sensory, proprioceptive, and interoceptive cues that are useful for comprehension and

prediction. For depth perception (i.e., to recognize that a star in the sky is far away), the organism uses visual cues (e.g., no

Copyright # 2009 John Wiley & Sons, Ltd. Eur. J. Soc. Psychol. 39, 1156–1159 (2009)

DOI: 10.1002/ejsp

1158 Klaus Fiedler

disparity of the two retinal images), auditory cues (e.g., stars to not send out any echo), epistemic cues (e.g., knowledge

about astronomy), or sensorimotor cues (e.g., the experience that the firmament remains invariant when the organisms

moves to a different place). However, the most fascinating aspect about vicarious functioning is that all cues are

substitutable. If we close one eye (i.e., ruling out retinal disparity), depth perception does not break down. The brain,

rather, smoothly turns to other cues which can compensate for the loss of one and several cues. No single cue, or modular

cue system is indispensable. When it’s dark at night, we use different cues for depth perception than during the day, when

the sun is shining. Color-blind people compensate for color cues. Deaf people’s cognitive functioning is largely

unimpaired by the lack of auditory and linguistic cues; they are instead highly sensitive to many visual and tactile cues.

People who have experienced curare paralizing their body, or patients with serious neurological deficits, whose sensory or

motor activities are reduced or mostly eliminated, do not report a breakdown of their cognitive capacities during states of

low embodiment.

Given all the fascinating evidence on vicarious functioning, I wonder if embodiment theory (Glenberg, Satto, Cattaneo,

Riggio, Palumba, & Buccino, 2008; Semin & Smith, 2008) would claim that sensorimotor cues have a special,

indispensable status, which lies outside the adaptive flexibility of vicarious functioning. I wonder what constraints

embodiment theory would impose on other sensorimotor cues, such as vertical orientation, approach-avoidance, or

fluency. If these cues are treated as primitives of embodied cognition, are they firmly entrenched in the ‘‘human hardware,’’

or can they unlearned and relearned easily? Since Kohler’s (1956) mirror glasses, for instance, we know that the vertical

experience of up and down can be reversed within a relatively short time. In a similar vein, research by Unkelbach (2006)

shows that the fluency cue can be reversed as well, signaling non-truth rather than truth. Our own research on evaluative

priming (Fiedler, Bluemke, & Unkelbach, 2009) suggests that approach versus avoidance responses elicited by positive

and negative primes can be reversed when prime valence correlates negatively with the valence of subsequent target

stimuli. If it is so easy to undo and reverse these cues, which are at the heart of embodied functions, the question is whether

such relearning processes can still be explained as immanent aspects of embodiment? Or must the freedom of the mind to

exchange and get rid of affective and bodily cues be attributed to a higher, meta-cognitive level that must be superior to

specific sensorimotor and modality-bound experiences?

For embodiment theory to enter the developmental stage of tightening, it has to develop a theory framework within

which testable constraints can be spelled out transparently. The Brunswikian flexible-cue framework I have depicted might

be profitably combined with the active, generative properties of the mental-simulation framework that Zwaan is

advocating. However, in any case, to obtain real theoretical progress, I believe it is necessary that embodiment researchers

call a spade a spade and spell out the constraints they want to impose on the utilization of embodied cues competing or

interacting with non-embodied cues. A premise for such a transparent theory, to be sure, is that the domain of embodiment

be explicated. Which cues fall outside the domain or, if all cues are embodied anyway, does the theory then not become

empty and non-falsifiable? Ironically, while proponents of embodiment (Niedenthal, Barsalou, Winkielman, Krauth-

Gruber, & Ric, 2005) have called non-falsifiability a main problem of amodal theories, it provides an equally challenging

problem for embodiment theories.

CONSTRAINTS RENDER THEORIES FALSIFIABLE

Falsifiability is indeed not a sign of weakness but a sign of strength and maturity. Once the constraints of the embodiment

approach are visible—and all of Zwaan’s challenging questions call for constraints—the theory acquires the ability to

make errors and to learn. I look forward to seeing strong embodiment hypotheses tested critically, to see researchers

commit themselves to risky predictions about which sensorimotor cues are (allegedly) indispensable, invariant, optimal

for learning and memory, and most likely to generate rich mental simulation processes. Real progress will not only come

from some strong and hardly believable hypotheses being verified but also from precise alternative hypotheses being

falsified and discarded. Translating the notion of embodiment into a set of vulnerable assumptions, rather than insisting in

the rhetorical argument that embodiment is ubiquitous, would indicate a successful transition from a loosing to a

tightening stage.

Let me illustrate this with a final example. Many decades ago, Stevens (1966) began to study cross-modality matching.

People have the capacity to translate a loudness scale into a dozen of other scales representing different sensory modalities.

Copyright # 2009 John Wiley & Sons, Ltd. Eur. J. Soc. Psychol. (2009)

DOI: 10.1002/ejsp

Embodied cognition and mental simulation 1159

We translate colors into melodies, animals into politician faces, temperature into colors, abstract paintings into semantic

labels. This phenomenon of cross-modality matching suggests a plethora of intriguing research questions, concerning the

psychophysical functions relating different modalities, the impact of modalities on learning and memory strength, or

the role of self-generated modality transitions for mental simulation. Ironically, the phenomenon highlights the

substitutability of every single modality or sensory quality. The very ability to translate one cue into the other suggests a

super-ordinate function that can replace the originally experienced cues. At the end of such tightened research, an absolute

role assigned to the body may turn out to be untenable, and a modality-independent regulation process may emphasize the

relativity and dispensability of all cues, whether they are grounded in the body or in abstract thought. However, even when

some aspects of a loose embodiment theory are falsified and discarded, this would not reflect its weakness but rather its

success and fertility.

ACKNOWLEDGEMENTS

Comments on a draft of this paper by Theodore Alexopoulos are gratefully acknowledged.

REFERENCES

Bargh, J. A., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activationon action. Journal of Personality and Social Psychology, 71, 230–244.

Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.Fiedler, K., Bluemke, M., & Unkelbach, C. (2009). On the adaptive flexibility of evaluative priming. submitted.Glenberg, A. M., Sato, M., Cattaneo, L., Riggio, L., Palumbo, D., & Buccino, G. (2008). Processing abstract language modulates motor

system activity. The Quarterly Journal of Experimental Psychology, 61, 905–919.Kelly, G. (1955). The psychology of personal constructs. New York: W.W. Norton.Kohler, I. (1956). Der Brillenversuch in der Wahrnehmungspsychologie mit Bemerkungen zur Lehre von der Adaptation [The mirror

glass experiment in perception psychology with comments on the study of adaptation]. Zeitschrift fur Experimentelle undAngewandte Psychologie, 3, 381–417.

Mar, R. A., Oatley, K., Hirsh, J., dela Paz, J., & Peterson, J. B. (2006). Bookworms versus nerds: Exposure to fiction versus non-fiction,divergent associations with social ability, and the simulation of fictional social worlds. Journal of Research in Personality, 40, 694–712.

Niedenthal, P. M., Barsalou, L. W., Winkielman, P., Krauth-Gruber, S., & Ric, F. (2005). Embodiment in attitudes, person perception,and emotion. Personality and Social Psychology Bulletin, 9, 184–211.

Pecher, D., & Zwaan, R. A. (2005). Grounding cognition: The role of perception and action in memory, language, and thinking.Cambridge: Cambridge University Press.

Proctor, R. W., & Cho, Y. S. (2006). Polarity correspondence: A general principle for performance of speeded binary classification tasks.Psychological Bulletin, 132, 416–442.

Semin, G. R., & Smith, E. R. (Eds.). (2008). Embodied grounding: Social, cognitive, affective, and neuroscientific approaches. NewYork: Cambridge University Press.

Stapel, D., & Semin, G. R. (2007). The magic spell of language. Journal of Personality and Social Psychology, 93, 23–33.Stevens, S. S. (1966). Matching functions between loudness and ten other continua. Perception & Psychophysics, 1, 5–8.Unkelbach, C. (2006). The learned interpretation of cognitive fluency. Psychological Science, 17, 339–345.Zwaan, R. A. (2004). The immersed experiencer: Toward an embodied theory of language comprehension. In B. H. Ross (Ed.), Thepsychology of learning and motivation (Vol. 44, pp. 35–62). New York: Academic Press.

Copyright # 2009 John Wiley & Sons, Ltd. Eur. J. Soc. Psychol. 39, 1156–1159 (2009)

DOI: 10.1002/ejsp