prosody-driven sentence processing: an event-related brain potential study ann pannekamp, ulrike...

Prosody-driven Sentence Processing: An Event-related

Brain Potential Study

Ann Pannekamp, Ulrike Toepel, Kai Alter, Anja Hahne and Angela D. Friederici

Presented by Laura Matzen, 9/1/2005

Pannekamp et al. (2005)

• Goal of study:– Determine what causes the closure positive

shift (CPS)

– Is this effect driven by prosody alone or by other factors?

Pannekamp et al. (2005)

• Basic Design:– Systematically reduce linguistic content of

sentences

– Record ERPs in each condition to see if CPS is present in all cases or to see how it changes

Background onEvent-related Potentials (ERPs)

Background

• Closure Positive Shift (CPS)– seen in response to normal spoken sentences– somehow related to intonation contour

• Intonation contour– “sentence melody”– provides information about syntactic structure,

sentence mode– fundamental frequency – F0

“Actually, we were the ones who said we didn’t want a regular cake, so you can blame it on us.”

Frequency

Intonational Phrases (IPh)

• One or more in each sentence

• Group and organize words into phrases

• Structure is usually determined by syntax


• “...defined as containing at least one nuclear accent and a boundary tone at their right edges”

• Pitch contour drops at the end, resets at start of the next IPh

• Last syllable usually lengthened

• Often followed by a pause

Examples

• She went to the store, then she picked up the dry cleaning on the way home.

• She went to the store, then she dropped the kids off at school, then she went home.


• Seem to be secondary to syntax in some ways– supporting role

• BUT, listeners can identify prosodic boundaries in the absence of semantic and syntactic information (de Rooij, 1975)


• Beckman (1996)“the prosodic structure of the utterance has to

be seen as a full grammatical property also requiring its own parsing”

• However, it might be harder to process prosody in absence of other linguistic information– Off-line behavioral studies can’t address this

issue

Steinhauer et al. (1999)

• Studied prosody with ERPs– Two sentence types: 1 or 2 IPh boundaries– Saw positive-going waveform in response to

all boundaries – CPS

– Possible Confound:• Close relationship between prosody and syntax• CPS could still be related to processing syntactic

boundaries, not just prosodic boundaries

Steinhauer & Friederici (2001)

• ERP study with delexicalized speech– Filtered to removed phonemic, semantic and

syntactic information– Only prosodic info left (pitch, amplitude,

rhythm)

CPS in this case could only be caused by prosody

Steinhauer & Friederici (2001)

• Results– Strange-looking CPS at first boundary– No CPS at second boundary– Contingent negative variation (CNV) across

whole sentence

Is this because the sentences are so unnatural?

Is this task completely different from language processing?

Meyer et al. (2002)

• fMRI study– Very different responses to natural speech

and delexicalized speech

– Response to prosody stronger in right hemisphere

• “strong evidence that pitch processing in the absence of additional linguistic information such as syntax and/or semantics takes place in right hemisphere regions”

Current Experiments

• Motivations:– Investigate partial replication of CPS findings

by Steinhauer & Friederici (2001)• Why didn’t they get the expected results?• Try using more natural stimuli

– See if CPS shifts to the right hemisphere as segmental information is removed

Experiments

• Experiment 1: Normal sentences– Normal semantic, syntactic and phonemic info

• Experiment 2: Jabberwocky sentences– Remove semantic information

• Experiment 3: Pseudo sentences– Remove semantic and syntactic information

• Experiment 4: Hummed sentences– Remove semantic, syntactic, and phonemic

info (only prosody left)

Expt 1: Normal Sentences

A1: [Kevin promises mom to sleep]IPh1

[and to be a good boy for a while.]

B1: [Kevin promises]IPh1 [mom to kiss]IPh2

[and to be a good boy for a while.]

Expt 1: Normal Sentences

A1: one IPh boundary at 1950 msec

B1: two IPh boundaries at 950 and 2700 msec• “First part” (subject + verb) of sentence longer than A1• High boundary tone at end of first part• Longer pause after first part

• Both conditions have IPh boundary after second verb (marked by high boundary tone)

Expt 2: Jabberwocky Sentences

All content words replaced with pseudo words

A2: [The bater rabels Onna to lubol]IPh1

[and the rado to nupe.]

B2: [The bater rabels ]IPh1 [Onna to lubol]IPh2

[and the rado to nupe.]

Expt 2: Jabberwocky Sentences


B2: two IPh boundaries at 1100 and 2600 msec

• “First part” (subject + verb) of sentence longer than A2• High boundary tone at end of first part• Longer pause after first part


Expt 3: Pseudo Sentences

All function and content words replaced with pseudo words

A3: [Bater saklimm Onna ko labei keg ]IPh1

[nug som Rado lie nupes.]

B3: [Bater saklimm]IPh1 [Onna ko labei keg]IPh2 [nug som Rado lie nupes.]

Expt 3: Pseudo Sentences


B3: two IPh boundaries at 920 and 2400 msec• “First part” (subject + verb) of sentence longer than A3• High boundary tone at end of first part• Longer pause after first part


Expt 4: Hummed Sentences

A4: [mm mmm mmmm mm mmmm]IPh1

[mmmm mmm mmm mmm mmmmm.]

B4: [mm mmm]IPh1 [mmmm mm mmm]IPh2 [mmmm mmm mmm mmm mmmmm.]

Expt 4: Hummed Sentences


B4: two IPh boundaries at 850 and 2150 msec• Longer pause after first part• High boundary tone at end of first part• (First part itself is NOT longer in this case)


Procedure• Subjects only saw one type of sentence

• Auditory presentation for all sentences

• Trial followed by a probe word– Subjects had to say whether word was in

sentence or not• (Words were placed randomly into hummed filler

sentences)

Data Collection

• 23 Ag/AgCl electrodes

• 200 msec prestimulus baseline

• Averages computed over whole sentences – 4500 msec segments

Results

A1: IPh at 1950 ms

Response at 2000 ms

------------------------------

B1: IPh1 at 950 ms

Response at 1500 ms

IPh2 at 2700 ms

Response at 2700 ms

A2: IPh at 2100 ms

Response at 2200 ms

------------------------------

B2: IPh1 at 1100 ms

Response at 1500 ms

IPh2 at 2600 ms

Response at 2800 ms

A3: IPh at 2000 ms

Response at ms

------------------------------

B3: IPh1 at 920 ms

Response at 1500 ms

IPh2 at 2400 ms

Response at 2500 ms

A4: IPh at 1850 ms

Response at 2200 ms

------------------------------

B4: IPh1 at 850 ms

Response at 1000 ms

IPh2 at 2150 ms

Response at 2000-2500 ms

Negative peak from 500-1000 ms

Waveforms measured from offset of 1st sentence fragment

Their interpretation...

• Positivity resembles CPS

• Observed in all experimental conditions– Related to processing of prosodic boundaries

• Scalp distribution changes across conditions

Their interpretation...

• CPS associated with first IPh seems related to the amount of segmental content in sentences– Moved forward and rightward as info

decreased

• CPS associated with second IPh seems UNrelated to amount of segmental content

Scalp Distributions – IPh1

• Experiment 1: CPS over whole head

• Experiment 2: CPS at midline and lateral sites, also moves to anterior sites

• Experiment 3: Moves to right anterior sites

• Experiment 4: CPS broadly distributed

Scalp Distributions

• First IPh:

– CPS moved anterior sites as linguistic information decreased

– CPS for psuedo sentences moved rightward, but not for hummed sentences

(possible explanation – left hemisphere processing timing of hums?)

Scalp Distributions• Second IPh:

– CPS distributed broadly over midline sites• also over lateral sites for normal and pseudo

sentence (why not for other types of sentences?)

– CPS moved rightward for hummed sentences ONLY

(CPS not seen here at all in previous study- conclude that naturalness is important)

Negativity

• Early negativity in hummed sentences– Different processing mechanism for pure

prosody?– Search for early accent (that isn’t there)?

– Different task demands?

Their Conclusions

• CPS is independent of expt manipulations– Seems to be related to perception of prosodic

boundaries– Dependent exclusively on prosody

“The observed differences in the scalp distribution of the CPS as a function of the segmental content of the acoustic speech stream suggest that prosodic processing interacts with other information types involving different systems.”

Questions• Why did scalp distribution change?

(It doesn’t seem to change in a systematic way)

• Why are there different results for the 1st and 2nd IPh boundaries?

• Are the differences in timing meaningful?

• Why not compare sentences with no IPh boundaries? (one phrase)

• Is it reasonable to say that these are “natural” manipulations?

prosody-driven sentence processing: an event-related brain potential study ann pannekamp, ulrike...

Documents