summary of david kemmerer's cognitive neuroscience of...

Summary of David Kemmerer’sCognitive Neuroscience of Language

Under the guidance ofProf. Amitabha Mukerjee

By Group 4Bhuvesh, Anusha, Parth

January 19, 2016

SpeechProductionSelection of the Word andthe phonetic encoding

The Speech motor controland feed-back mechanism

SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Outline

1 Speech ProductionSelection of the Word and the phonetic encodingThe Speech motor control and feed-back mechanism

2 Speech Perception

3 Sign Language

4 Object Nouns

5 Action Verbs

6 Abstract Words

Group 4 Kemmerer Book Summary Slide 2 of 66



SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Speech Production and Perception




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Speech Production

Average adult human has a lexicon of 50,000 to 100,000words.

Speech production involves the coordination of about 80muscles.

The correct word to be spoken in milliseconds with veryhigh accuracy.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Division into Stages

The production of speech has been broadly divided into twostages:

The selection of the word and the encoding of theirphonetic forms

The Speech motor control and feed-back mechanism.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

The lemma model

Theory proposed by W.J.M Levelt in the 1970s withexperiments being done till date. A feedforward mechanismwhich states that this stage is divided into two processes:

Lexicon selection

Form Encoding




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Lexicon Selection

The concepts are stored in the form of a graph.

The concepts are represented as nodes.

The edges between the nodes signify the relations betweenthe concepts.

The middle temporal gyrus is mainly involved in lexiconselection.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Lexicon Selection: Cont.

The lexical concepts are in the forms of nodes and areconnected to related lexical concepts and the lemmas.

When a lemma i.e. a real life concept or object is to beconverted into a lexical entity, the nodes connected to thelemma are activated.

Several lemmas and lexical concept nodes are activated inparallel and the lexical concept with the highest degree ofactivation is selected as the correct lexical concept.

Other nodes activated help in the context formation andhence aid in the formation of sentences.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Lexicon Selection Example

Figure : Graphical Representation of Concepts (from page 151of [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Form encoding

Once the lexicon is selected, the task is to assign to reachthe phonological representation of the word.A three step pipeline1 The morphemes are attached to the root lexical concept.

election of the correct morpheme is controlled by the leftposterior STG/STS (Wernickes area), posterior MTG; the leftanterior insula; and the right SMA.

2 The completes word is divided into individual phonemes.3 The phonemes are joined to form syllables. The last two

steps are performed mainly by the broca’s area




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Form Encoding Example

Figure : The three stage pipeline (from page 152 of [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Mental Syllabary

Existence of a mental syllabary which is dictionary fromsyllables to their phonetic encodings called the articulatoryscore.

After syllabification, the articulatory score of the syllablesare sent to the motor systems for articulation.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Word Frequency Effect

The words which are more frequent in the language areeasily recalled.Lemma Model states the stage responsible for this effect isthe form encoding and articulation stage and not at lemmaselection.

Homophone effect as an experimental proof (from page 154of [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

The DIVA Model

Directions Into Velocities of Articulators Model of speechmotor control.

Proposed by Frank Guenther, a researcher at BostonUniversity in mid 1990s.

Starts where the Lemma model leaves off, i.e. with thephonetic encoding.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Division

The architecture is divided into two subsystems:

1 Feedforward control: Activating motor commands forarticulation and transmitting these commands to the vocalapparatus through the subcortical nuclei.

2 Feedback control:Using auditory and somatosensoryinput form the self produced speech to recognize errors anddecide the corrections to be sent to the articulatory part.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Feedforward Control

The system is organised as:

The Speech Sound Map: Repository of speech soundrepresentations i.e. equivalent to the index of the dictionaryin the Mental Syllabary concept from the lemma model.Present in the left posterior IFG and ventral premotorcortex.

The Articulatory Velocity and Position Maps: Containvocal tract representations of the larynx, lips, jaw, tongue,and palate. Present in ventral primary motor cortex.

Initiation Map: A module that sends a go signal. Present inthe Supplementary motor area (SMA).




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Feedback Control

The Feedback control systems is divided into two feedbackcircuits:

1 The auditory feedback circuit

2 The somatosensory feedback circuit




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

The auditory feedback circuit

The Auditory Target Map: Receives the acousticexpectations from the Speech Sound Map.

The Auditory State Map: Receives the speech auditoryinput.

The Auditory Error Map: A module that computesdiscrepancies between the anticipated sounds stored in theAuditory Target Map and the actual sounds produced,stored in the Auditory State Maps.

Feedback Control Map: Updates articulatory commandsaccording to the sensory feedback from the Error Map.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

The somatosensory feedback circuit

The Somatosensory Target Map: Receives the tactile andproprioceptive expectations from the Speech Sound Map.

The Somatosensory State Map: Receives the representstactile and proprioceptive input from speech production.

The Somatosensory Error Map: A module that computesdiscrepancies between the anticipated tactile andproprioceptive sensations and the actual sensationsproduced.

Feedback Control Map: Updates articulatory commandsaccording to the sensory feedback from the Error Maps.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Dual Stream Model of Speech Perception

Figure : Dual Stream Model of Speech Perception (from page 114of [1])Group 4 Kemmerer Book Summary Slide 20 of 66



SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

SIGN LANGUAGE




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Sign Language: Preliminaries

Sign language is one of the integral communicationsystems for the deaf people.

There are currently about 120 different sign languages.

Some examples are Indian Sign Language, American SignLanguage, British Sign Language etc.

Deafs communicate by signs(src: rochellebarlow.com)




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Parameters

The three main factors which play a great role in producingdifferent signs are :

Handshape

Location

Movement




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Different Signs in ISL (taken from page 248)

Figure : Signs differentiated by movement,location,handshape




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Structural Aspects

Phonology : a whole sign is composed of individualfeatures or elements.

Morphology : deals with internal grammar of words (canbe compounding, derivation or inflection).

Syntax : the rules used for producing and joining words.

Non-manual signs : while conveying a sign we tilt ourhead/body and use facial expressions.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Signs differentiated by Inflection

Figure : (A)I SHOW YOU (B)YOU SHOW ME (C)HE SHOWS YOU(from page 250 of [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Signs differentiated by Syntax

Figure : Syntax of Force differs from Give (from page 252 of [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Wada Testing

In 1986 Damasio et. al. conducted an experiment on a27-year-old woman where it was observed that when her lefthemisphere was stopped from functioning properly by adrug, she showed aphasia (even though in normal life sheknew both ASL and English).This proved that sign language depends mainly on lefthemisphere of the brain.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Sign Language and Visuospatial Cognition

Sign Language and Visuospatial Cognition go in hand inhand.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Supporting Experiment for VisuospatialCognition

Experiment : In 1996 Hickok et al. did an experiment with 13LHD (left hemisphere damaged) patients and 10 RHD (righthemisphere damaged) patients.

Observation : The LHD patients exhibited impaired signlanguage but proper visuospatial cognition and the RHDpatients had just the opposite of this.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Sign Language and Symbolic Gesture

Sign language and sybmolic gesture goes hand in hand.Experiment : In 2004, a 56-year old left temporoparietalstroke patient named Charles could not name a toothbrushwhen he was shown its picture, but when he was told to actout how a person brushes, he did so properly starting fromputting the paste to brushing up and down.

Conclusion : The route responsible for symbolic gesture isdifferent from the one used for sign production.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Object Nouns and Action Verbs




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Object Nouns

Object Nouns are names for concrete objects e.g.meercat ,(animal) , guava(fruit), pen(tool), aeroplane(vehicle) and soon. There are two competing theories to explain therepresentations of meanings of object nouns in the brain :1.Amodal Symbolic Model2.Grounded Cognition Model




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Amodal Symbolic Model

Maintains that the semantic system( where concepts arerepresented and processed) and modality-specific systemsfor action and perception are completely different.[1]

Was popular from 1970s to 1990s

Lacks strong support from modern neuroscience studies.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Grounded Cognition Model

Maintains that concepts are anchored in modality-specificsystems, such that understanding word meanings involvesactivating high-level perceptual and motor representations.[1]

Dates back to ancient philosophers such as Epicurus(341–270 BC)

Was resurrected in 1990s

Empirically supported by recent neuroscience studies.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words


Research has supported the hypothesis that neuralsubstrates of the meaning of object nouns are the same asthat associated with high-level perception.e.g Color featuresare stored in the same brain area that is associated withhigh-level color perception and so on.

Color features =⇒ ventral temporal cortex especially areaV4α.Shape features

animal concepts =⇒ lateral portion of the mid-fusiform gyrustool concepts =⇒ medial portion of the mid-fusiform gyrus

Motion features =⇒ posterolateral temporal cortex.Motor features =⇒ aIPS/SMG and the vPMC.Auditory features =⇒ posterior superior/middle temporalcortexOlfactory and gustatory features =⇒ orbitofrontal cortex




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words





SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Hub and Spoke Model

Combines elements from both the Grounded CognitionModel and Amodal Symbolic Model.

It maintains that concepts are based not only onmodality-specific brain systems for perception and action,but also on a modality-invariant integrative device—thesemantic hub—in the anterior temporal lobes (ATLs).[1]




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Hub and Spoke Model




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

The semantic hub

It integrates the different modality-specific features(whichare located in different parts of the brain) into a singleconcept.

This makes it possible to determine whether an object fallswithin the scope of a concept or outside of it.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

The semantic hub




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Where is the semantic hub located?

There are several sources of evidence which position this"semantic hub" in ATLs(bilaterally):

SD(Semantic Dementia) patients show gradual degradationof object concepts with the atrophy of ATLs.

ATLs are activated when healthy subjects process objectconcepts.

Disrupting ATLs using rTMS reduces the ability to processobject concepts.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Design and Results




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

A New Challenge

Some cases of brain-damaged patients withcategory-specific deficits of object concepts have beenreported with deficits in three major domains:animals, tools,fruits/vegetables.

Animal concepts =⇒ bilateral damage to the ventral andmedial sectors of the mid- to-anterior temporal lobes.

Fruit/Vegetable concepts =⇒ unilateral left-hemispheredamage to relatively more posterior areas, including themid-fusiform gyrus.[1]

Tool concepts =⇒ unilateral left-hemisphere damage tothe posterior lateral temporal region (pMTG), inferiorparietal region (aIPS/SMG), and/or inferior frontal region(vPMC)[1]




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Two approaches

2 approaches to explain category-specific deficits:1 Differential Weighting Hypothesis

different domains of object concepts are characterized bydifferent mixtures and "weightings" of modality-specificfeaturesthis causes them to gravitate, over the course of cognitivedevelopment, toward different networks of brain regions.Thus, disrupting a particular region that is functionally moreimportant for one conceptual domain than for others may besufficient to generate a category-specific deficit.[1]

2 Distributed Domain-Specific HypothesisDifferent conceptual domains are innately programmed tohave segregated neural implementations not only at the levelof modality-specific systems for perception and action, butalso at a more abstract level of pure semantic structure.Thus, a particular type of category-specific deficit would mostlikely result from damage to the correspondingcategory-specific component of the system that is restrictedexclusively to representing conceptual knowledge.Group 4 Kemmerer Book Summary Slide 45 of 66



SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

An illustration




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Verb Diversity

Some examples of action verbs given in [1] :

rapid motion : dash, rush, scramble

leisurely motion : amble, drift, stroll

smooth motion : glide, slide, slink, slip

awkward motion : limp, lurch, stagger, stumble




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Involvement of left PLTC to process Action verbs

Experiment : In 2008 Kemmerer et.al. gave 5 classes ofverbs to people : running , hitting , cutting , speaking, changeof state and told them to distinguish verbs within each class.For e.g. limp is more like trudge than stroll.

Observation: fMRI showed that everyone used the leftposterolateral temporal cortex (PLTC). To strengthen hisclaim, Kemmerer increased it to 6 tasks with 226 patients.

Conclusion : The left PLTC is very important in processingaction verbs.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Motor Cortex

Figure : Action zones in motor cortex(which is in frontal lobe) of amonkey (page 315 [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Overview of Models

There are 3 models which have been discussed.

Grounded Cognition Model : semantic features thatinvolve “visual motion patterns and motor specifications”depend on same neural systems that support “high-levelaction preparation”.

Hub and Spoke Model : The semantic hub binds thecortically distributed sensory and motor features ofconcepts. Details are already described.

Semantic Somatotopy Hypothesis : The motor featuresof verbs are present in “somatotopically mapped frontalareas”. As per the book, verbs for face-related actions(lick)depend on those ventral areas which control these type ofactions, and verbs for arm-related actions (pick) depend onthe lateral and dorsolateral areas which control these typeof actions.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Semantic Somatotopy Hypothesis

Figure : Motor features of verbs for face-related actions (lick) areindicated by blue nodes in inferior frontal region, arm-relatedactions by purple nodes in lateral frontal region, leg-relatedactions(kick) by yellow nodes in dorsal frontal region (pg. 317 [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Activation patterns evoked by hockey sentences

Figure : The left dorsal premotor cortex was engaged in expertsand fans but not novices. This effect correlated positively withcomprehension efficiency. (pg. 319 [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Activation patterns evoked by hockey sentences

Figure : The dorsal primary sensory-motor cortex was engagedbilaterally in novices but not experts or fans. This effect correlatednegatively with comprehension efficiency. (pg. 319 [1])




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Abstract Words




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Abstract Words

Abstract words are words without a direct correspondenceto a real world physical objects.

The concepts they represent are high-level(more abstract)as compared to those represented by object nouns oraction verbs.

E.g. freedom, love, war, sad, word, abstract, word, conceptand so on

Abstract concepts are harder to imagine and take greatertime to process as compared to concrete words.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Two Competing Models

According to the Dual Coding Modelall word meanings are based on both modality-specificrepresentations (non- verbal codes) and lexical associations(verbal codes), but they differ with regard to the relativepreponderances of the two types of information.Concrete concepts draw more or less equally on bothsystems, whereas abstract concepts rely primarily on theverbal system. [1]

According to the Context Availability ModelAll word meanings are amodal in format, but they differ withregard to how hard they are to pin down.Concrete concepts are fairly stable and insensitive to context,whereas abstract concepts are more variable and sensitive tocontext. [1]




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Dual coding model




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Analysis contd.

A meta-analysis of 19 PET/fMRI studies by Wang et al.(2010) revealed that the following brains regions were moreactive while processing concrete words as compared toabstract words

The left fusiform gyrus (associated with visual shape andcolor representations).

The bilateral posterior cingulate gyrus (associated withvisual imagery, spatial attention, navigation, and episodicmemory).

The left angular gyrus (associated with various integrativefunctions).

[1] These findings are more compatible with the the DualCoding Model.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Analysis contd.

For the abstract> concrete contrast the following brainregions were found:

The left inferior frontal gyrus (IFG, associated withauditory–verbal short-term memory and the strategiccontrol of semantic processing).

The left middle/superior sector of the anterior temporal lobe(ATL, associated with high-level spoken languagecomprehension and amodal semantic structures).

These findings can be handled equally well by the DualCoding Model and the Context Availability Model. [1]




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Evidence for Context Availability Model

To investigate the role of IFG in abstract word processing ,Hoffman et al. (2010) conducted a study to evaluateunderstanding of abstract words in(a) absence of any context (b) presence of context in bothbrain damaged patients and patients having undergonerTMS in the IFG region.This study is strongly compatible with Context AvailabilityModel as compared to Dual Coding model.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words





SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Some interesting things I learned..

Our brain isn’t "magical". i.e. A small brain damage canimpair us, which is not much different from a malfunctioningmachine

Most of the studies referenced were surprisingly new ! Thisis a frontier in brain science and there is a lot left to beexplored.

How "common sense" shapes science and vice versa.




SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

References

Cognitive Neuroscience of Language. By DavidKemmerer.

Other useful sources:https://en.wikipedia.org/wiki/Sign_language


https://en.wikipedia.org/wiki/Sign_language



SpeechPerception

Sign Language

Object Nouns

Action Verbs

Abstract Words

Conclusion


summary of david kemmerer's cognitive neuroscience of...

Documents