About language and about space

Download About language and about space

Post on 07-Nov-2014




0 download


Relation between language and space



The present volume consistsof chapters by participants in the Language and Space . In most casesthe chapters conferenceheld in Tucson, Arizona , 16- 19 March 1994 have beenwritten to reflect the numerous interactions at the conference , and for that reason we hope the book is more than just a compilation of isolated papers. The conferencewas truly interdisciplinary , including such domains as neurophysiology, , and linguistics. Neural , psychology, anthropology , cognitive science neuropsychology es mechanisms , and cultural factors were all grist for the , developmental process mill , as were semantics , syntax, and cognitive maps. The conferencehad its beginnings in a seemingly innocent conversation in 1990 betweentwo new colleaguesat the University of Arizona (Bloom and Peterson ), who MAP .) assumed of them confusions. One of left right wondered about the genesis ( that theseconfusions reflecteda languageproblem; the other (P. B.) was quite certain that they reflected a visual perceptual problem. Curiously, it was the perception researcherwho saw this issueas being mainly linguistic and the languageresearcher who saw it as mainly perceptual. In true academic form they decided that the best way to arrive at an answer would be to hold a seminar on the topic , which they did the very next year. Their seminar on languageand spacewas attended by graduate students , postdoctoral fellows, and many faculty membersfrom a variety of departments . Rather than answering the question that led to its inception, the seminar ? What aspectsof spacecan we raised other questions: How do we represent space ? And what role doesculture play in talk about? How do we learn to talk about space all thesematters? One seminar could not explore all of theseissuesin any depth; an enlarged group of interestedcolleagues(the four coeditors) felt that perhaps several workshops might . The Cognitive NeuroscienceProgram at the University of Arizona , in collaboration with the Cognitive ScienceProgram and the PsychologyDepartment, sponsored . Although two one-day workshops on the relations between space and language : other rise to still questions How does stimulating and helpful, the workshops gave




? How many kinds of spatial representations are there? the brain represent space ? Should What happensto spatial representationsafter various kinds of brain damage to closed restricted be and between relations of the tests language space experimental classlinguistic elementsor must the role of open-classelementsbe consideredas well? Given the scopeof thesequestion, we decidedto invite investigators from a variety . of disciplines to a major scientific conference , and Language and Spacetook shape . We do not imagine that the The conferencewas judged by all to be a great success chaptersin this book provide final answersto any of the questionswe first raised, but we are confident that they add much to the discussionand demonstrate the importance . We expectthat increasedattention of the relations betweenspaceand language will be given to this fascinating subject in the years ahead and hope that our conference , and this book , have made a significant contribution to its understanding. Meetings cannot be held without the efforts of a considerablenumber of people, . Our thanks to Pauline Smalley for all work and the support of many funding sources she did in organizing the conferenceand making sure participants got to the right place at the right time and to Wendy Wilkins , of Arizona State University, for her . We gratefully acknowledgethe gracious help both before and during the conference ' : McDonnell Pew Cognitive NeuroscienceProgram support of the conferences sponsors , the Flinn Foundation Cognitive Neuroscience Program, and the Cognitive ScienceProgram and Department of Psychology at the University of Arizona . We , which greatly thank the participants for their intellectual energy and enthusiasm ' of the MIT Pierce thank we . contributed to the conferences success Finally , Amy Pressfor her help with this volume. Editors Bloom and Petersontosseda coin one eveningover margaritas to determine whose name would go first.

Chapter of the Linguistic-Spatial Interface The Architecture ~ Ray Jackendoff



? More specifically How do we talk about what we see , how does the mind / brain encodespatial information (visual or otherwise), how does it encodelinguistic information , and how does it communicate betweenthe two? This chapter lays out some of the boundary conditions for a satisfactory answerto thesequestionsand illustrates the approach with somesampleproblems. The skeleton of an answer appears in figure 1.1. At the language end, speech perception converts auditory information into linguistic information , and speech production converts linguistic information into motor instructions to the vocal tract. Linguistic information includes at least somesort of phonetic/phonological encoding es of visual perception convert retinal information . ! At the visual end, the process of speech . into visual information , which includes at least some sort of retinotopic mapping. The connection betweenlanguageand vision is symbolized by the central it is clear there cannot be a direct relation double-headedarrow in figure 1.1. Because betweena retinotopic map and a phonological encoding, the solution to our problem lies in elaborating the structure of this double-headedarrow.

1.2 Representational ModularityThe overall hypothesisunder which I will elaborate figure 1.1 might be termed Representational , chapter I ) . , chapter 12; Jackendoff 1992 Modularity (Jackendoff 1987 formats distinct in information encodes mind brain that the The generalidea is many / " " for or languagesof the mind. There is a module of mind/ brain responsible each of these formats. For example, phonological structure and syntactic structure are distinct levels of encoding, with distinct and only partly commensurateprimitives and principles of combination. RepresentationalModularity therefore posits that the architecture . Each of the mind / brain devotesseparatemodules to thesetwo encodings

Ray Jackendoff

auditory signals ---.........

...- eye 4 ~ visual information information linguistic ~ motor signals ~ - - - -_ J C \ - - - "' -- Y - - - ---- I ~ ~ - -yVISION LANGUAGE

Figure 1.1 Coarse sketch of the relation betweenlanguageand vision.

of thesemodules is domain-specific (phonology and syntax, respectively ); and (with " in Fodor ' s " certain caveatsto follow shortly) each is informationally encapsulated . Representational modules differ from Fodorian modules in that they ) sense ( 1983 are individuated by the representationsthey processrather than by their function as faculties for input or output ; that is, they are at the scale of individual levels of , rather than being entire faculties such as languageperception. representation A conceptual difficulty with Fodorian Modularity is that it leavesunansweredhow ; modules communicate with each other and how they communicate with Fodor s ' central, nonmodular cognitive core. In particular , Fodor s languageperception module ' " derives " shallow representations - some form of syntactic structure; Fodor s " " " central faculty of " belief fixation operatesin terms of the languageof thought , a " " nonlinguistic encoding. But Fodor doesnot tell us how shallow representations are " " converted to the languageof thought, as they must be if linguistic communication is to affect belief fixation . In effect, the language module is so domain-specific and informationally encapsulatedthat nothing can get out of it to serve cognitive purposes .2 And without a theory of intermodular communication, it is impossible to approach the problem we are dealing with here, namely, how the languageand vision modules manageto interact with each other. es this difficulty by positing, in The theory of RepresentationalModularity address . modulesproposed above, a systemof interfacemodules addition to the representation An interface module communicatesbetweentwo levels of encoding, say Ll and L2 , by carrying a partial translation of information in Ll form into information in L2 : the phonologyform. An interfacemodule, like a Fodorian module, is domain-specific to-syntax interface module, for instance , knows only about phonology and syntax, -purpose audition . Such a module is also in not about visual perception or general : the phonology-to -syntax module dumbly takes whatever formationally encapsulated phonological inputs are available in the phonology representationmodule, translates the appropriate parts of them into (partial) syntactic structures, and delivers them to the syntax representation module, with no help or interference from , say, beliefs about the social context. In short, the communication among languagesof the mind es as well.3 is mediated by modular process

-Spatial The Architecture of the Linguistic Interfaceg-p

auditory ............ ........- phonology ~ .. motoreye ~ retinotopic. ~

~ syntax




/ ,haptic *,,action localization .... auditory.. ~

audition ,smell ,emotion ,... / , * structure / :..~ conceptual tresentation spatial rep ;

1.2 . Figure less sketch of coarse Slightly

the relation between language and vision .

The levelsof representationI will be working with here, and the interfaces among them, are sketchedin figure 1.2. Each label in figure 1.2 standsfor a level of representation served by a representation module. The arrows stand for interface modules. Double-headedarrows can be thought of either as interface modules that processbi directionally or as pairs of complementary unidirectional modules (the correct choice is an empirical question) . For instance , the phonology-syntax interface functions from left to right in speechperception and from right to left in speechproduction . " " Figure 1.2 expands the linguistic representation of figure 1.1 into three levels involved with language : the familiar levelsof phonology and syntax, plus conceptual structure, a central level of representation that interfaces with many other faculties. " " Similarly, visual representation in figure 1.1 is expandedinto levelsof retinotopic, ' imagistic, and spatial representation , corresponding roughly to Marr s ( 1982 ) primal sketch, 21 0 sketch, and 3 D model, respectively ; the last of theseagain is a central representationthat interfaceswith other faculties. In this picture, the effect of Fodor ian faculty -sized modules emergesthrough the linkup of a seriesof representation and interface modules; communication among Fodorian faculties is accomplishedby interface modules of exactly the same general character as the interface modules within faculties. The crucial interface for our purposeshere is that betweenthe most central levels of the linguistic and visual faculties, conceptual structure and spatial representation . Beforeexamining this interface, we have to discusstwo things: ( I ) the generalcharacter of interfaces betweenrepresentations(section 1.3); and (2) the general character of conceptual structure and spatial representationthemselves (sections 1.4 and 1.5) .

1.3 Character of Interface MappingsTo say that an interface module " translates " between two representations is , strictly speaking , inaccurate . In order to be more precise, let us focus for a moment on the

Ray Jackendotr

interface between phonology and syntax, the two best-understood levels of mental . representation It is obvious that there cannot be a complete translation betweenphonology and syntax. Many details of phonology, most notably the segmentalcontent of words, , many details of syntax, for instance the play no role at all in syntax. Conversely elaborate layering of specifiersand of arguments and adjuncts, are not reflected in phonology. In fact, a complete, information -preserving translation betweenthe two representationswould be pointless; it would in effect make them notational variants - which they clearly are not. The relation between phonology and syntax is actually something more like a partial homomorphism. The two representationsshare the notion of word (and perhaps .4 But ), and they share the linear order of words and morphemes morpheme segmentaland stressinformation in phonology has no direct counterpart in syntax; and syntactic category (N , V , PP, etc.) and case , number, gender , and person features 5 have no direct phonological counterparts. Moreover, syntactic and phonological constituent structures often fail to match. A classicexampleis given in ( I ) . ( I ) Phonological: [ Thisis the cat] [that ate the rat] [that ate the cheese ] Syntactic: [ Thisis [the cat [that ate [the rat [that ate [the cheese ]]]]]] The phonological bracketing, a flat tripartite structure, contrasts with the relentless , English articles cliticize phoright -embeddedsyntactic structure. At a smaller scale nologically to the following word , resulting in bracketing mismatches such as (2) . (2) Phonological: [the [ big]] [ house ] Syntactic: [the [ big [ house ]] Thus, in general, the phonology-syntax interface module createsonly partial correbetweenthesetwo levels . spondences A similar situation obtains with the interface between auditory information and phonological structure. The complex mappingbetweenwaveforms and phonetic segmentation in a sense the relative order of information : a particular auditory preserves cue may provide evidencefor a number of adjacent phonetic segments , and a particular be a number of phonetic segmentmay signaledby , but the adjacent auditory cues " bands" of the in an stream overlapping correspondenceprogress through speech orderly linear fashion. On the other hand, boundaries betweenwords, omnipresentin phonological structure, are not reliably detectable in the auditory signal; contrari -

The Architecture of the Linguistic - Spatial Interface

wise, the auditory signal contains information about the formant frequenciesof the ' speakers voice that are invisible to phonology. So again the interface module takes only certain information from each representation into account in establishing a betweenthem. correspondence These examples show that each level of representation has its own proprietary information , and that an interface module communicates only certain aspects of this information to the next level up- or downstream. Representational modules, : precisely to the extent that they then, are not entirely informationally encapsulated receiveinformation through interface modules, they are influenced by other parts of the mind.6 In addition to general principles of mapping, such as order preservation, an interface module can also make use of specialized learned mappings. The clearest instances of suchmappings are lexical items. For instance , the lexical item cat stipulates that the phonological struct...