modeling semantic relations expressed by prepositions
DESCRIPTION
Modeling Semantic Relations Expressed by Prepositions . Vivek Srikumar and Dan Roth University of Illinois, Urbana-Champaign. Prepositions trigger relations. John enjoyed the visit to the zoo in NYC. E njoy Agent / Enjoyer : John Cause / Thing-enjoyed : the visit to the zoo in NYC Visit - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/1.jpg)
Modeling Semantic Relations Expressed by Prepositions
Vivek Srikumar and Dan RothUniversity of Illinois, Urbana-Champaign
![Page 2: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/2.jpg)
2
Prepositions trigger relations
John enjoyed the visit to the zoo in NYC. • Enjoy– Agent/Enjoyer: John– Cause/Thing-enjoyed: the visit to the zoo in NYC
• Visit– Agent: John– Destination: the zoo in NYC
Q: Where is the zoo located? A: NYC.
![Page 3: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/3.jpg)
3
Talk outline
1. Ontology of preposition relations
2. Two models for predicting preposition relations
3. Experiments
![Page 4: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/4.jpg)
4
ONTOLOGY OF PREPOSITION RELATIONS
![Page 5: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/5.jpg)
5
Examples of preposition relations
Possessor
Species
![Page 6: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/6.jpg)
6
Preposition Sense Disambiguation
Eg. State of Illinois vs. University of Illinois
• The Preposition Project [Litkowski and Hargraves, 2005]
– Word sense for 34 prepositions– Based on preposition definitions in Oxford
Dictionary of English
![Page 7: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/7.jpg)
7
Mapping from senses to relationslive at Conway House
at:1(1)
stopped at 9 PMat:2(2)
cooler in eveningin:3(2)
drive at 50 mphat:5(3)
came on Sep. 26th
on:17(8)
the camp on the islandon:7(2)
look at the watchat:9(5)
Location
Temporal
ObjectOfVerb
Numeric
...
![Page 8: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/8.jpg)
8
An inventory of preposition relations
• Labels that act as the predicate – Semantically related senses of prepositions merged– ~250 senses 32 relation labels
• Word sense disambiguation data, re-labeled– SemEval 2007 shared task gives relation labeled data
• ~16K training and ~8K test instances• 34 prepositions
![Page 9: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/9.jpg)
9
TWO MODELS FOR PREDICTING PREPOSITION RELATIONS
“zoo in NYC” Location(zoo, NYC)
![Page 10: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/10.jpg)
10
Poor care led to her death from flu.
Cause
death fluGovernor Object
Relation
Structure of prepositions
![Page 11: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/11.jpg)
11
Relation depends on argument types
Poor care led to her death from flu.
Cause(death, flu)
Poor care led to her death from pneumonia.
How do we generalize the classifier to unseen arguments in the same “type”?
![Page 12: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/12.jpg)
12
Why are types important?
• Goes beyond words– Abstract flu and pneumonia into the same group
• Some semantic relations hold only for certain types of entities
• Two notions of type • WordNet hypernyms• Distributional word clusters
– Allow for multiple meanings and concept hierarchies
![Page 13: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/13.jpg)
13
WordNet IS-A hierarchy
pneumonia
=> respiratory disease
=> disease
=> illness
=> ill health
=> pathological state
=> physical condition
=> condition
=> state
=> attribute
=> abstraction
=> entity
More general, but less discrimniative
Picking the right level in this hierarchy can generalize pneumonia and flu
Picking incorrectly will over-generalize
![Page 14: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/14.jpg)
14
Poor care led to her death from flu.
Cause
death flu
experience disease
Governor Object
Governor type
Object type
Relation
Structure of prepositions
![Page 15: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/15.jpg)
15
Two models
• Model 1 – Predict only relation label: Multi-class– Use features from all possible governor and object
candidates• Also types
• Model 2 uses features from the structure– Predict full structure: relation and arguments• Also types
![Page 16: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/16.jpg)
16
Model 1: Predict relation labelPoor care led to her death from flu.
her
led
death
flu
Attribute
Source
CausePaint from resin
Weak from asthma
Candidate from Montreal..
Governor Object
Governor type Object type
Relation
lead
produce
travel
herchange in state
event
state
killing
point in time
ending
Features from all sources
contagious disease
communicable disease
disease
![Page 17: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/17.jpg)
17
Model 2: Predict full structurePoor care led to her death from flu.
her
led
death
flu
contagious disease
communicable disease
disease
Attribute
Cause
Source
Paint from resin
Weak from asthma
Candidate from Montreal..
lead
produce
travel
herchange in state
event
state
killing
point in time
ending
Governor Object
Object type
Relation
Governor type
![Page 18: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/18.jpg)
18
Poor care led to her death from flu.
Cause
death flu
experience disease
Governor Object
Governor type
Object type
Relation
Structure of prepositions
![Page 19: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/19.jpg)
19
Learning Model 2: Latent inference
• Standard inference: Find an assignment to the full structure
• Latent inference: Given an example with annotated
• “Complete the structure given current model”
![Page 20: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/20.jpg)
20
Learning Model 2
• Initialize weight vector using Model 1
• Repeat– Use latent inference with current weight to
“complete” all missing pieces– Train with Structured SVM• During training, the learning algorithm is penalized
more if it makes a mistake on
Generalization of Latent Structure SVM [Yu & Joachims ’09]
![Page 21: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/21.jpg)
21
Poor care led to her death from flu.
Cause
death flu
experience disease
Governor Object
Governor type
Object type
Relation
Preposition Sense and Relations
from:12(9)Sense [Hovy et al, 2010]
![Page 22: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/22.jpg)
22
EXPERIMENTS
![Page 23: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/23.jpg)
23
Accuracy of relation labeling
Model 1 Model 286.5
87
87.5
88
88.5
89
89.5
90
90.5
Baseline+ types+ joint sense
Model size: 5.41 million non-zero weights
Model size: 2.21 million non-zero weights
Using types gives improvement, helps model 1 more
Model 2 helps
Enforcing coherence with preposition sense gives best results
![Page 24: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/24.jpg)
24
What do we have?
Input Relation Governor type Object typeDied of pneumonia Cause Experience Disease Suffering from flu Cause Experience DiseaseRecovered from flu StartState Change Disease
Governor, object and their types as a certificate for the choice of relation label
![Page 25: Modeling Semantic Relations Expressed by Prepositions](https://reader036.vdocuments.us/reader036/viewer/2022062501/56816376550346895dd45370/html5/thumbnails/25.jpg)
25
Conclusion
• Prepositions express a diverse set of relations– An ontology of preposition relations– Can enrich existing PropBank/FrameNet
representation• Models for predicting preposition relations– Arguments and types help
Data, word clusters, software available (soon)
Questions?