where to next? pronoun interpretation as a side effect of discourse direction
DESCRIPTION
Where to next? Pronoun interpretation as a side effect of discourse direction. Hannah Rohde, Andy Kehler, & Jeff Elman UC San Diego. CUNY Conference on Human Sentence Processing, March 29-31 2007. Bill. He. John. He. Transfer Verb. Goal (to-phrase). Ambiguous Pronoun Prompt. - PowerPoint PPT PresentationTRANSCRIPT
Where to next? Pronoun interpretation as a side effect of discourse direction
Hannah Rohde, Andy Kehler, & Jeff ElmanUC San Diego
CUNY Conference on Human Sentence Processing, March 29-31 2007
2
Transfer of possession (Stevenson et al. 1994)
(1) handed a book to . ______________.John Bill HeJohn He recommended it
Transfer Verb
Source (subject)
Goal (to-phrase)
AmbiguousPronoun Prompt
thanked JohnBill He
50/50: Goal continuations / Source continuations No subject preference or grammatical parallelism Two explanations considered:
Thematic Role Preference Event Structure Bias
3
Outline
Background: Rohde et al. 2006 Test Thematic-Role and Event-Structure biases Alternative account: Discourse Coherence
Experiments 1 & 2: test predictions of a coherence- based model using story continuations
Preliminary results: discourse effects in relative clause attachment
4
(2) handed a book to . ______ .JohnSOURCE BillGOAL He(3) was handing a book to . ___ .JohnSOURCE BillGOAL He
Explaining salience of Goal (Rohde et al. 2006)
0%
20%
40%
60%
80%
100%
Perfective Imperfective
% Interpretation
SourceGoal
Goal bias ~ side effect of Event Structure
Thematic role preference or event structure bias?
Equivalent thematic roles but different event structure
Effect of aspectF(1,48)=50.622p<0.0001
5
Establishing coherence: infer a relationship between the meanings expressed by two sentences (P&Q below) (Hobbs 1979, Kehler 2002)
Effects of coherence (Rohde et al. 2006)
(5) Matt passed a sandwich to David. He said thanks.
[Result: P Q]
David He
(4) Matt passed a sandwich to David. He didn’t want David to starve.
[Explanation: Q P]
Matt He
Causal relations (Explanation, Result, Violated Expectation)
6
(6) Matt passed a sandwich to David. He did so carefully.
[Elaboration: infer P from both S1 and S2]
Coherence cont.
[Occasion: infer initial state of event described in S2 to be final state of event described in S1]
(7) Matt passed a sandwich to David. He ate it up.David He
Matt He
Similarity relations (Parallel, Elaboration)
Contiguity relations (Occasion)
7
Goal bias following perfective context sentences limited to Occasion & Result (see Arnold 2001)
Interpretation as side effect of coherence distribution
Perfective Context Sentences
050
100150200
OccasionElaborationExplanationViolated-ExpResultParallel
CountSourceGoal
Discourse coherence effects (Rohde et al.)
8
Test predictions of a coherence-driven model More Occasion/Result more Goal resolutions More Explanation/Elaboration/Violated-Exp more
Source
Shift coherence shift interpretation
9
Experiment 1: objects-of-transfer
(8) John handed a book to Bill. He ___________ .
(9) John handed a bloody meat cleaver to Bill. He __ .
Proposal: elicit different continuations with different objects Stimuli: normal and bizarre objects
Predictions: If… Abnormal objects more Explanations and Explanations Source bias More Source continuations for (9) than (8)
10
Methodology
Subjects: 69 monolingual English speakers Task: write 50 continuations, just like Rohde et al. Stimuli: 21 transfer-of-possession like Rohde et al.
(+ bizarre objects) Evaluation: judges assess coherence/interpretation Analysis:
Effect of within-subject factor of Object Type on Coherence (Elab/Expl/Occ/Par/Res/Viol-Exp) Pronoun interpretation (Source/Goal)
Mixed-effects logistic regression Controls for random effects of Subject and Item
11
0%
20%
40%
60%
80%
100%
Normal Abnormal
% Coherence
ExpElabV-EResOccPar
Coherence varies by object
Results
0.81
0.820.99
0.050.17
0.71
Consistent prob(Source|coh)Exp 1Rohde et al.Coherence
Elaboration
0.87Violated-expectation
0.75Explanation0.99
0.16Result0.20Occasion
0.45Parallel
p<0.0001Source Source
Goal} Goal}
12
Results
0%
20%
40%
60%
80%
100%
Normal Abnormal
% Interpretation
SourceGoal
Subjects: F(1,68)= 0.052p<0.820
Items: F(1,20)=0.111p<0.743
No effect of object type on pronoun interpretation
13
Experiment 2: ‘What next?’ or ‘Why?’
(10) John handed a book to Bill. He ___________ .
Stimuli & Design: identical to Rohde et al. 2006
Predictions: “What next?” more Occasions Goal bias “Why?” more Explanations Source bias
Instructions: write continuations answering either “What happened next?” or “Why?”
14
Methodology
Subjects: 42 monolingual English speakers Task: identical to Rohde et al. 2006 (w/instructions) Stimuli: identical to Rohde et al. 2006 Evaluation: judges assess coherence/interpretation
Analysis: Effect of between-subject factor of Instruction Type
on coherence distribution & pronoun interpretation
15
0%
20%
40%
60%
80%
100%
What next? Why?
% Coherence
ExpElabV-EResOccPar
Results
Coherence varies w/instruction (p<0.0001)
Goal Source
0.81
0.811.00
0.110.28
0.46
Consistent prob(Source|coh)Exp 2Rohde et al.Coherence
Elaboration
0.87Violated-expectation
0.75Explanation0.99
0.16Result0.20Occasion
0.45Parallel
16
Results
Effect of Instruction type on pronoun interpretation
0%
20%
40%
60%
80%
100%
What next? Why?
% Interpretation
SourceGoal
F(1,20)=52.672p<0.0001
17
Predicting pronoun interpretation
€
=%Exp * p(SR |Exp) + %Elab* p(SR |Elab) + %V - E * p(SR | V - E) +
%Occ* p(SR |Occ) + %Res * p(SR |Res) + %Par* p(SR |Par)
p(Source)Coherence
Elaboration0.81
0.820.99
V-E
Explanation
0.050.17
ResultOccasion
0.45Parallel
Predict % Source Resolutions in Exp 2 using: Exp2 coherence breakdown Exp1 conditional probabilities
€
(1) %SR =
€
* p(SR |Coh)
€
coh∑
€
%coh
18
Capturing subject variation
0%
20%
40%
60%
80%
100%
1 6 11 16 21 26 31 36 41 Participant
Observed%Source
Predicted%Sourceusing (1)
linear regression R2=0.604 F(1,40)=61.097, p<0.0001
€
* p(SR |Coh)
€
coh∑
€
(1) %SRi =
€
%cohi for subject i in Exp 2
“What next” “Why”
19
R2=0.604, F(1,40)=61.097*Exp1: average across verbal aspects & object types
R2=0.586, F(1,40)=51.165*Exp1: imp, abnormal objects
R2=0.561, F(1,40)=51.165*Exp1: perf, abnormal objects
R2=0.627, F(1,40)=67.371*Exp1: imp, normal objects
R2=0.606, F(1,40)=61.612*Exp1: perf, normal objects
R2 value/ANOVAConditional Probability Estimator
* Indicates p<0.0001
Consistency of biases across conditions
20
Shift coherence Shift pronoun interpretation No model relying only on surface-level cues can
account for observed variation, since stimuli were near-identical (Exp 1) or identical (Exp 2)
Need richer models incorporating discourse-level factors (see Wolf et al. 2004; Kertz et al. 2006)
Summary
21
lowhigh Relative clause attachment ambiguity
What else can discourse do for you?
(11) Beth babysits the children of the musician who ____
Proposal: try to shift RC attachment using verbs that require Explanations and that attribute
cause to the referent occupying higher NP
musical prodigies themselves. arethe children
Function of a relative clause
(12) John despises the employee who is always late.
Implicit Causality (NP2 IC) verbs attribute cause to direct object
at the club downtown.playsthe musician
22
Predictions & results
0%
20%
40%
60%
80%
100%
nonIC ICVerb Type
% Attachment
lowhigh
F(1,51)=31.082 p<0.0001
Further evidence that discourse influences interpretation
nonIC: Beth babysits the children of the musician who _____
0%
20%
40%
60%
80%
100%
nonIC ICVerb Type
% Coherence
ModExpElabOccResPar
p<0.0001
IC: Beth despises the children of the musician who ______
playsat the club downtown.
the musician low
and yell during rehearsals. screamthe children
high
23
References
Arnold, J. E. (2001) The effects of thematic roles on pronoun use and frequency of reference. Discourse Processes, 31(2): 137-162.
Chambers, G. C. & Smyth, R. (1998) Structural parallelism and discourse coherence: A test of Centering Theory. Journal of Memory and Language, 39: 593-608.
Crawley, R., Stevenson, R., & Kleinman, D. (1990) The use of heuristic strategies in the interpretation of pronouns. Journal of Psycholinguistic Research, 4: 245–264.
Kameyama, M. (1996) Indefeasible semantics and defeasible pragmatics. In M. Kanazawa, C. Pinon, and H. de Swart, editors, Quantifiers, Deduction, and Context. CSLI Stanford, pp. 111-138.
Hobbs, J. R. (1979) Coherence and coreference, Cognitive Science, 3:67-90. Hobbs, J. R. (1990) Literature and Cognition. CSLI Lecture Notes 21. Stanford, CA. Kehler, A. (2002) Coherence, reference, and the theory of grammar. CSLI Publications, Stanford, CA.Kertz, L., Kehler, A., & Elman, J. (2006) Grammatical and Coherence-Based Factors in Pronoun
Interpretation. 28th Annual Conference of the Cognitive Science Society, Vancouver, July 2006.Moens, M. & Steedman, M. (1988) Temporal ontology and temporal reference. Computational
Linguistics 14(2):15-28.Smyth, R. H. (1994) Grammatical determinants of ambiguous pronoun resolution. Journal of
Psycholinguistic Research, 23: 197-229.Stevenson, R., Crawley R., & Kleinman D. (1994) Thematic roles, focusing and the representation of
events. Language and Cognitive Processes, 9:519–548.Wolf, F., Gibson, E. & Desmet, T. (2004) Coherence and pronoun resolution. Language and Cognitive
Processes, 19(6): 665-675
24
Variation by instruction and aspect
(17) John was giving a book to Bill. He ___________ .(16) John gave a book to Bill. He ___________ .
…“What happened next?”
Interpretation (instr x aspect interaction p<0.0001)
0%
20%
40%
60%
80%
100%
whatnext?
why? whatnext?
why?
perfective imperfective
SourceGoal
0%
100%
whatnext?
why? whatnext?
why?
perfective imperfective
SourceGoal
Coherence (instr x aspect interaction p<0.0001)
0%
20%
40%
60%
80%
100%
Whatnext?
Why? Whatnext?
Why?
Perfective Imperfective
ParV-EElabExpResOcc0%
100%
Whatnext?
Why? Whatnext?
Why?
Perfective Imperfective
ParV-EElabExpResOcc
25
Discourse coherence effectsPerfective Context Sentences
0
50
100
150
200
OccasionElaborationExplanationViolated-Exp
Result Parallel
Count
SourceGoal
Imperfective Context Sentences
0
50
100
150
200
OccasionElaborationExplanation
Result
Violated-ExpParallel
Count
SourceGoal