jointly identifying temporal relations with markov logic katsumasa yoshikawa †, sebastian riedel...
TRANSCRIPT
![Page 1: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/1.jpg)
Jointly Identifying Temporal Relations with Markov Logic
Katsumasa Yoshikawa†, Sebastian Riedel‡, Masayuki Asahara†, Yuji Matsumoto†
†Nara Institute of Science and Technology, Japan‡ University of Massachusetts, Amherst
ACL-IJCNLP2-7 August, 2009 Suntec Singapore
![Page 2: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/2.jpg)
2
Outline
Background and Motivation Related work of temporal relation
identification Proposed global approach with Markov
Logic Experimental setup and highlighted data Summary and future work
![Page 3: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/3.jpg)
3
Background and Motivation Temporal Relation Identification (temporal ordering)
Identifying temporal orders of events and time expressions in a document
introduction
PresentPast Future
Document Creation Time(August 2009)
became
BEFORE
2003
Essential work for document understanding
With the introduction of the TimeBank corpus (Pustejovsky et al., 2003), machine learning approaches to temporal ordering became possible.
![Page 4: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/4.jpg)
4
Outline
Background and Motivation Related work of temporal relation
identification Proposed global approach with Markov
Logic Experimental setup and highlighted data Summary and future work
![Page 5: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/5.jpg)
5
EVENT / TIME
AFTER
IAFTER
ENDS
DURING
BEGUN_BY
SIMULTANEOUS
BEGINS
ENDED_BY
IBEFORE
BEFORE
>
mi
oi
f
d
si
=
s
c
fi
o
m
<
after
met-by
overlapped-by
finishes
during
started-by
equal
startscontains
finished-by
overlapsmeets
before
TimeML(11 Labels)
Allen’s(13 Labels)
INCLUDES
Allen‘s Temporal Logic [Allen 1983] TimeML and TimeBank [Pustejovsky et al. 2003]
We regard temporal ordering as a classification task With TimeML, the TimeBank corpus was created
![Page 6: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/6.jpg)
6
TempEval (SemEval 2007 Task 15)
Temporal Relation Identification in SemEval 2007 Shared Task (TempEval)
Six temporal relation labels Main Label (BEFORE, AFTER , OVERLAP) Sub-Label (BEFORE-OR-OVERLAP, OVERLA
P-OR-AFTER, VAGUE) TempEval includes three types of tasks (A,
B, and C)
![Page 7: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/7.jpg)
7
introduction
DCT(August 2009)
became
2003
OVERLAP
Task A of TempEval
Temporal relations between events and time expressions that occur within the same sentence
Past FuturePresent
With the introduction of the TimeBank corpus (Pustejovsky et al., 2003), machine learning approaches to temporal ordering became possible.
![Page 8: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/8.jpg)
8
introduction
DCT(August 2009)
became
2003
Task B of TempEval
Temporal relations between events and the Document Creation Time (DCT)
BEFORE BEFORE
Past FuturePresent
With the introduction of the TimeBank corpus (Pustejovsky et al., 2003), machine learning approaches to temporal ordering became possible.
![Page 9: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/9.jpg)
9
BEFOREcreated
DCT(August 2009)
became
2003
Task C of TempEval
Temporal relations between the main events of adjacent sentences
Past FuturePresent
The TimeBank corpus was created (Pustejovsky et al., 2003). As a result, machine learning approaches to temporal ordering became possible.
![Page 10: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/10.jpg)
10
Issues of the TempEval Participants Local approaches with machine learning are empl
oyed by many participants in TempEval Considering only a single relation at a time Local approach cannot take into account the other
relations
A global approach can be useful in that case
EVENT 1 EVENT 2BEFORE (Task C)AFTER ? (Task C)
DCT
EVENT 1
BEFORE (Task B)
EVENT 2
AFTER (Task B)DCT
![Page 11: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/11.jpg)
11
A global approach can be useful in that case
Issues of the TempEval Participants Local approaches with machine learning are empl
oyed by many participants in TempEval Considering only a single relation at a time Local approach cannot take into account the other
relations
EVENT 1 EVENT 2BEFORE (Task C)
DCT
BEFORE (Task B) AFTER (Task B)
![Page 12: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/12.jpg)
12
Outline
Background and Motivation Related work and task reviews of temporal
relation identification Proposed global approach with Markov
Logic Experimental setup and highlighted data Summary and future work
![Page 13: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/13.jpg)
13
Overview of Our Global Approach
Ensure consistency among the multiple relations with hard and soft constraints based on the transition rules
Jointly identify the three types of relations in TempEval Learning one global model for the three tasks
Global approach with Markov Logic
![Page 14: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/14.jpg)
14
Markov Logic[Richardson and Domingos, 2006]
A Statistical Relational Learning framework An expressive template language of Marko
v Networks Not only hard but also soft constraints A Markov Logic Network (MLN) is a set of p
airs (φ, w) where φ is a formula in first-order logic w is a real number weight
Higher weight stronger constraint
![Page 15: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/15.jpg)
15
※ e1 and e2 are events
An Example of Markov Logic Networks hasPastTense(a) : indicates that an event a has past tense beforeDCT(a) : indicates that an event a happens before the DCT before(a,b) : indicates that an event a happens before another
event b
ID Weight function Weigh value Ground formula
(A1) wa(e1) 3.1 hasPastTense(e1) beforeDCT(e1)⇒
(A2) wa(e2) -0.9 hasPastTense(e2) beforeDCT(e2)⇒
(B1) wb(e1,e2) 1.7 beforeDCT(e1) ^ ¬ beforeDCT(e2) before(e1, e2)⇒
hasPastTense(e1)
beforeDCT(e1)
wa(e1)
beforeDCT(e2)
wb(e1,e2)
before (e1,e2) hasPastTense(e2)
wa(e2)grounding
![Page 16: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/16.jpg)
16
Global Feature Representation (Predicate Definition)
relE2T(e, t, r) : the relation r between an event e and a time expression t relDCT(e, r) : the relation r between an event a and the DCT relE2E(e1, e2, r) : the relation r between two events e1 and e2 relT2T(t1, t2, r) : the relation r between two time expressions t1 and t2 dctOrder(t, r) : the relation r between a time expression t and the DCT
EVENT (e1)
DCT
TIME (t1)
EVENT (e2)
TIME (t2)
relE2E(C)
relDCT(B)
relE2T(A)
dctOrder dctOrder
relT2T
relDCT(B)
relE2T(A)
![Page 17: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/17.jpg)
17
We jointly solve the three tasks of TempEval We use global features named Joint formulae A joint formula is based on a transition rule
EVENT (e1)
DCT
EVENT(e2)
BEFORE AFTER
BEFORE
BEFORE & AFTER BEFORE⇒
EVENT (e2)
DCT
EVENT(e1)
AFTER
BEFOREBEFORE
BEFORE & AFTER BEFORE⇒
If e1 happens before DCT and e2 happens after DCT => then e1 is before e2
If e1 happens before DCT and e1 happens after e2, => then e2 happens before DCT
Global Feature Representation (Transition Rules)
B→C C→B
![Page 18: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/18.jpg)
18
Global Feature Representation (Templates of the all Joint Formulae)
Tasks Joint Formula (first-order logic)
A→B dctOrder(t1,r) & relE2T(e1, t1, r1) relDCT(e1,r2)⇒
B→A dctOrder(t1,r) & relDCT(e1,r1) relE2T(e1, t1, r2)⇒
B →C relDCT(e1, r1) & relDCT(e2, r2) relE2E(e1, e2, r3)⇒
C→B relDCT(e2, r1) & relE2E(e1,e2, r2) relDCT(e1, r3)⇒
A→C relE2T(e1,t1,r1) & relT2T(t1,t2,r2) & relE2T(e2,t2,r3) relE2E(e1,e2,r4)⇒
C→A relE2T(e2,t2,r2) & relT2T(t1,t2,r1) & relE2E(e1,e2,r3) relE2T(e1,t1,r4)⇒
They are developed with events, time expressions and relations
![Page 19: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/19.jpg)
19
Global Feature Representation (Templates of the all Joint Formulae)
Tasks Joint Formula (first-order logic)
A→B dctOrder(t1,r) & relE2T(e1, t1, r1) relDCT(e1,r2)⇒
B→A dctOrder(t1,r) & relDCT(e1,r1) relE2T(e1, t1, r2)⇒
B →C relDCT(e1, BEFORE) & relDCT(e2, AFTER) relE2E(⇒ e1, e2, BEFORE)
C→B relDCT(e1, BEFORE) & relE2E(e1,e2, AFTER) relDCT(⇒ e2, BEFORE)
A→C relE2T(e1,t1,r1) & relT2T(t1,t2,r2) & relE2T(e2,t2,r3) relE2E(e1,e2,r4)⇒
C→A relE2T(e2,t2,r2) & relT2T(t1,t2,r1) & relE2E(e1,e2,r3) relE2T(e1,t1,r4)⇒
They are developed with events, time expressions and relations
![Page 20: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/20.jpg)
20
Outline
Background and Motivation Related work and task reviews of temporal
relation identification Proposed global approach with Markov
Logic Experimental setup and highlighted data Summary and future work
![Page 21: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/21.jpg)
21
Experimental Setup
Use a MLN Engine “Markov thebeast” Weight learning : MIRA Inference : Cutting Plane Inference
(base solver: ILP) [Riedel, 2008] Employ the local features referred to the ea
rly work in TempEval [SemEval, 2007] Select joint formulae as global features Use the same data and evaluation scheme
s of TempEval
![Page 22: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/22.jpg)
22
Comparison of Local and Global
Local Global
Task A 0.613 0.662 (+0.049)
Task B 0.789 0.799 (+0.010)
Task C 0.533 0.552 (+0.019)
All 0.667 0.689 (+0.022)
Results with 10-fold cross validation on training data
Over all tasks, Global is better than Local On Task A, Global model outperformed Local one.
ρ< 0.01 (McNemar’s test, 2-tailed)
※All scores denote F1-value
![Page 23: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/23.jpg)
23
Results with the other systems on test data (F1-value)
Comparison to State-of-the-art
Outperformed the others on Tasks A and C Always performed better than the best pure m
achine-learning based system (CU-TMP[Bethard and Martin, 2007])
Other Systems
TempEval Best
TempEval Average CU-TMP
Task A 0.62 0.56 0.61
Task B 0.80 0.74 0.75
Task C 0.55 0.51 0.54
Our Systems
Local Global
0.62 0.65
0.74 0.76
0.53 0.57※All scores denote F1-value
![Page 24: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/24.jpg)
24
Outline
Background and motivation Related work and task reviews of temporal
relation identification Proposed global approach with Markov
Logic Experimental setup and highlighted data Summary and future work
![Page 25: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/25.jpg)
25
Summary
We proposed a global framework with Markov Logic for Temporal Relation Identification
Our global model with joint formulae successfully improved the performances of the identifications
Our approach reported the competitive results among all participants in TempEval
![Page 26: Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute](https://reader036.vdocuments.us/reader036/viewer/2022062712/56649c7d5503460f94932f42/html5/thumbnails/26.jpg)
26
Future Work
Issues inherent to the task and the dataset Low inter annotator agreement Low transitive connectivity Small size
Semi-supervised approaches ease some issues
TRAIN DEV TEST TOTAL
Task A 1359 131 169 1659
Task B 2330 227 331 2888
Task C 1597 147 258 2002
Numbers of labeled relations for all tasks and datasets