where to next? pronoun interpretation as a side effect of discourse direction

Where to next? Pronoun interpretation as a side effect of discourse direction

Hannah Rohde, Andy Kehler, & Jeff ElmanUC San Diego

CUNY Conference on Human Sentence Processing, March 29-31 2007

2

Transfer of possession (Stevenson et al. 1994)

(1) handed a book to . ______________.John Bill HeJohn He recommended it

Transfer Verb

Source (subject)

Goal (to-phrase)

AmbiguousPronoun Prompt

thanked JohnBill He

50/50: Goal continuations / Source continuations No subject preference or grammatical parallelism Two explanations considered:

Thematic Role Preference Event Structure Bias

3

Outline

Background: Rohde et al. 2006 Test Thematic-Role and Event-Structure biases Alternative account: Discourse Coherence

Experiments 1 & 2: test predictions of a coherence- based model using story continuations

Preliminary results: discourse effects in relative clause attachment

4

(2) handed a book to . ______ .JohnSOURCE BillGOAL He(3) was handing a book to . ___ .JohnSOURCE BillGOAL He

Explaining salience of Goal (Rohde et al. 2006)

0%

20%

40%

60%

80%

100%

Perfective Imperfective

% Interpretation

SourceGoal

Goal bias ~ side effect of Event Structure

Thematic role preference or event structure bias?

Equivalent thematic roles but different event structure

Effect of aspectF(1,48)=50.622p<0.0001

5

Establishing coherence: infer a relationship between the meanings expressed by two sentences (P&Q below) (Hobbs 1979, Kehler 2002)

Effects of coherence (Rohde et al. 2006)

(5) Matt passed a sandwich to David. He said thanks.

[Result: P Q]

David He

(4) Matt passed a sandwich to David. He didn’t want David to starve.

[Explanation: Q P]

Matt He

Causal relations (Explanation, Result, Violated Expectation)

6

(6) Matt passed a sandwich to David. He did so carefully.

[Elaboration: infer P from both S1 and S2]

Coherence cont.

[Occasion: infer initial state of event described in S2 to be final state of event described in S1]

(7) Matt passed a sandwich to David. He ate it up.David He

Matt He

Similarity relations (Parallel, Elaboration)

Contiguity relations (Occasion)

7

Goal bias following perfective context sentences limited to Occasion & Result (see Arnold 2001)

Interpretation as side effect of coherence distribution

Perfective Context Sentences

050

100150200

OccasionElaborationExplanationViolated-ExpResultParallel

CountSourceGoal

Discourse coherence effects (Rohde et al.)

8

Test predictions of a coherence-driven model More Occasion/Result more Goal resolutions More Explanation/Elaboration/Violated-Exp more

Source

Shift coherence shift interpretation

9

Experiment 1: objects-of-transfer

(8) John handed a book to Bill. He ___________ .

(9) John handed a bloody meat cleaver to Bill. He __ .

Proposal: elicit different continuations with different objects Stimuli: normal and bizarre objects

Predictions: If… Abnormal objects more Explanations and Explanations Source bias More Source continuations for (9) than (8)

10

Methodology

Subjects: 69 monolingual English speakers Task: write 50 continuations, just like Rohde et al. Stimuli: 21 transfer-of-possession like Rohde et al.

(+ bizarre objects) Evaluation: judges assess coherence/interpretation Analysis:

Effect of within-subject factor of Object Type on Coherence (Elab/Expl/Occ/Par/Res/Viol-Exp) Pronoun interpretation (Source/Goal)

Mixed-effects logistic regression Controls for random effects of Subject and Item

11

0%

20%

40%

60%

80%

100%

Normal Abnormal

% Coherence

ExpElabV-EResOccPar

Coherence varies by object

Results

0.81

0.820.99

0.050.17

0.71

Consistent prob(Source|coh)Exp 1Rohde et al.Coherence

Elaboration

0.87Violated-expectation

0.75Explanation0.99

0.16Result0.20Occasion

0.45Parallel

p<0.0001Source Source

Goal} Goal}

12

Results

0%

20%

40%

60%

80%

100%

Normal Abnormal

% Interpretation

SourceGoal

Subjects: F(1,68)= 0.052p<0.820

Items: F(1,20)=0.111p<0.743

No effect of object type on pronoun interpretation

13

Experiment 2: ‘What next?’ or ‘Why?’

(10) John handed a book to Bill. He ___________ .

Stimuli & Design: identical to Rohde et al. 2006

Predictions: “What next?” more Occasions Goal bias “Why?” more Explanations Source bias

Instructions: write continuations answering either “What happened next?” or “Why?”

14

Methodology

Subjects: 42 monolingual English speakers Task: identical to Rohde et al. 2006 (w/instructions) Stimuli: identical to Rohde et al. 2006 Evaluation: judges assess coherence/interpretation

Analysis: Effect of between-subject factor of Instruction Type

on coherence distribution & pronoun interpretation

15

0%

20%

40%

60%

80%

100%

What next? Why?

% Coherence

ExpElabV-EResOccPar

Results

Coherence varies w/instruction (p<0.0001)

Goal Source

0.81

0.811.00

0.110.28

0.46

Consistent prob(Source|coh)Exp 2Rohde et al.Coherence

Elaboration

0.87Violated-expectation

0.75Explanation0.99

0.16Result0.20Occasion

0.45Parallel

16

Results

Effect of Instruction type on pronoun interpretation

0%

20%

40%

60%

80%

100%

What next? Why?

% Interpretation

SourceGoal

F(1,20)=52.672p<0.0001

17

Predicting pronoun interpretation

€

=%Exp * p(SR |Exp) + %Elab* p(SR |Elab) + %V - E * p(SR | V - E) +

%Occ* p(SR |Occ) + %Res * p(SR |Res) + %Par* p(SR |Par)

p(Source)Coherence

Elaboration0.81

0.820.99

V-E

Explanation

0.050.17

ResultOccasion

0.45Parallel

Predict % Source Resolutions in Exp 2 using: Exp2 coherence breakdown Exp1 conditional probabilities

€

(1) %SR =

€

* p(SR |Coh)

€

coh∑

€

%coh

18

Capturing subject variation

0%

20%

40%

60%

80%

100%

1 6 11 16 21 26 31 36 41 Participant

Observed%Source

Predicted%Sourceusing (1)

linear regression R2=0.604 F(1,40)=61.097, p<0.0001

€

* p(SR |Coh)

€

coh∑

€

(1) %SRi =

€

%cohi for subject i in Exp 2

“What next” “Why”

19

R2=0.604, F(1,40)=61.097*Exp1: average across verbal aspects & object types

R2=0.586, F(1,40)=51.165*Exp1: imp, abnormal objects

R2=0.561, F(1,40)=51.165*Exp1: perf, abnormal objects

R2=0.627, F(1,40)=67.371*Exp1: imp, normal objects

R2=0.606, F(1,40)=61.612*Exp1: perf, normal objects

R2 value/ANOVAConditional Probability Estimator

* Indicates p<0.0001

Consistency of biases across conditions

20

Shift coherence Shift pronoun interpretation No model relying only on surface-level cues can

account for observed variation, since stimuli were near-identical (Exp 1) or identical (Exp 2)

Need richer models incorporating discourse-level factors (see Wolf et al. 2004; Kertz et al. 2006)

Summary

21

lowhigh Relative clause attachment ambiguity

What else can discourse do for you?

(11) Beth babysits the children of the musician who ____

Proposal: try to shift RC attachment using verbs that require Explanations and that attribute

cause to the referent occupying higher NP

musical prodigies themselves. arethe children

Function of a relative clause

(12) John despises the employee who is always late.

Implicit Causality (NP2 IC) verbs attribute cause to direct object

at the club downtown.playsthe musician

22

Predictions & results

0%

20%

40%

60%

80%

100%

nonIC ICVerb Type

% Attachment

lowhigh

F(1,51)=31.082 p<0.0001

Further evidence that discourse influences interpretation

nonIC: Beth babysits the children of the musician who _____

0%

20%

40%

60%

80%

100%

nonIC ICVerb Type

% Coherence

ModExpElabOccResPar

p<0.0001

IC: Beth despises the children of the musician who ______

playsat the club downtown.

the musician low

and yell during rehearsals. screamthe children

high

23

References

Arnold, J. E. (2001) The effects of thematic roles on pronoun use and frequency of reference. Discourse Processes, 31(2): 137-162.

Chambers, G. C. & Smyth, R. (1998) Structural parallelism and discourse coherence: A test of Centering Theory. Journal of Memory and Language, 39: 593-608.

Crawley, R., Stevenson, R., & Kleinman, D. (1990) The use of heuristic strategies in the interpretation of pronouns. Journal of Psycholinguistic Research, 4: 245–264.

Kameyama, M. (1996) Indefeasible semantics and defeasible pragmatics. In M. Kanazawa, C. Pinon, and H. de Swart, editors, Quantifiers, Deduction, and Context. CSLI Stanford, pp. 111-138.

Hobbs, J. R. (1979) Coherence and coreference, Cognitive Science, 3:67-90. Hobbs, J. R. (1990) Literature and Cognition. CSLI Lecture Notes 21. Stanford, CA. Kehler, A. (2002) Coherence, reference, and the theory of grammar. CSLI Publications, Stanford, CA.Kertz, L., Kehler, A., & Elman, J. (2006) Grammatical and Coherence-Based Factors in Pronoun

Interpretation. 28th Annual Conference of the Cognitive Science Society, Vancouver, July 2006.Moens, M. & Steedman, M. (1988) Temporal ontology and temporal reference. Computational

Linguistics 14(2):15-28.Smyth, R. H. (1994) Grammatical determinants of ambiguous pronoun resolution. Journal of

Psycholinguistic Research, 23: 197-229.Stevenson, R., Crawley R., & Kleinman D. (1994) Thematic roles, focusing and the representation of

events. Language and Cognitive Processes, 9:519–548.Wolf, F., Gibson, E. & Desmet, T. (2004) Coherence and pronoun resolution. Language and Cognitive

Processes, 19(6): 665-675

24

Variation by instruction and aspect

(17) John was giving a book to Bill. He ___________ .(16) John gave a book to Bill. He ___________ .

…“What happened next?”

Interpretation (instr x aspect interaction p<0.0001)

0%

20%

40%

60%

80%

100%

whatnext?

why? whatnext?

why?

perfective imperfective

SourceGoal

0%

100%

whatnext?

why? whatnext?

why?

perfective imperfective

SourceGoal

Coherence (instr x aspect interaction p<0.0001)

0%

20%

40%

60%

80%

100%

Whatnext?

Why? Whatnext?

Why?

Perfective Imperfective

ParV-EElabExpResOcc0%

100%

Whatnext?

Why? Whatnext?

Why?

Perfective Imperfective

ParV-EElabExpResOcc

25

Discourse coherence effectsPerfective Context Sentences

0

50

100

150

200

OccasionElaborationExplanationViolated-Exp

Result Parallel

Count

SourceGoal

Imperfective Context Sentences

0

50

100

150

200

OccasionElaborationExplanation

Result

Violated-ExpParallel

Count

SourceGoal