text processing 1 anaphora resolution: recent developments massimo poesio
Post on 17-Dec-2015
222 Views
Preview:
TRANSCRIPT
Outline
Analysis of errors made by mainstream systems
Joint Entity Detection and Tracking Lexical and Commonsense knowledge AR tools (Global models)
Error analysis (Soon et al)
Errors most affecting precision: Prenominal modifiers identified as mentions and
other errors in mention identification String match but noun phrases refer to different
entities Errors most affecting recall:
Errors in mention identification (11%) Errors in SEMCLASS determination (10%) Need more features (63.3%)
Soon et al examples of errors:
Tarnoff, a former Carter administration official and president of the Council on foreign relations, is expected to be named [undersecretary] for political affairs … Former. Sen Tim Wirth is expected to get a newly created [undersecretary] post for global affairs
[Ms Washington and Mr. Dingell] have been considered [allies] of [the Securities exchanges], while [banks] and [future exchanges] often have fought with THEM
Mention detection errors in GUITAR (Kabadjov, 2007)
[The bow] (see detail, below right) is decorated with a complicated arrangement of horses and lions’ heads.
Above the lions’ heads are four sphinxes.
Three pairs of lions clamber up the section from the point where [the sheath and bow] are joined.
Joint Entity Detection and Tracking
Daume and Marcu 2005: Mention identification, classification, and linking take place at the same time
Denis and Balridge 2007: ILP
Lexical and commonsense knowledge
Nominals the most common type of anaphoric expression
Main source of errors with nominals: lack of commonsense knowledge
Lexical and commonsense knowledge: outline
Charniak, Sidner: the semantic network assumption
Formal models: Hobbs et al 1992, Poesio 1993, Lascarides & Asher 1998, Gardent & Kohlase 1999, Kehler
Using information about semantic relations: Byron 2002, Kehler 2004, Ng, Yang
Ponzetto & Strube [Simone] Pattern-based approaches (Poesio et al,
Markert & Nissim, Versley)
Basic errors in GUITAR: synonyms & hyponyms
Toni Johnson pulls a tape measure across the front of what was once [a stately Victorian home].
…..
The remainder of [THE HOUSE] leans precariously against a sturdy oak tree.
Most of the 10 analysts polled last week by Dow Jones International News Service in Frankfurt .. .. expect [the US dollar] to ease only mildly in November
…..
Half of those polled see [THE CURRENCY] …
Early work: the semantic network assumption (Charniak, Sidner)
CAR
VEHICLE
WHEELS
ISA
HAS
I saw [a car] come in. … THE VEHICLE was moving very slowly … THE WHEELS were moving very slowly …
Basic errors: NE
[Bach]’s air followed. Mr. Stolzman tied [the composer] in by proclaiming him the great improviser of the 18th century ….
[The FCC] …. [the agency]
Modifiers
FALSE NEGATIVE:
A new incentive plan for advertisers …
…. The new ad plan ….
FALSE NEGATIVE:
The 80-year-old house
….
The Victorian house …
(Actual example of error from Soon et al on ACE)
<COREF ID="set_5">The quality</COREF> that 's coming out of <COREF ID="set_8">software</COREF> from <COREF ID="set_3">India</COREF> now is exceeding <COREF ID="set_5">the quality</COREF> coming out of <COREF ID="set_8">software</COREF> from the United States .
More complex cases of modification
The cabinet was actually made for [a Polish general, [Count Jan Klemens Branicki]]. An inventory of Count Branicki's possessions made at his death describes both the corner cupboard and the objects displayed on its shelves: a collection of mounted Chinese porcelain and clocks, some embellished with porcelain flowers.The drawing of the corner cupboard, or more probably anengraving of it, must have caught Branicki's attention.
Dubois was commissioned through a Warsaw dealer to construct the cabinet for THE POLISH ARISTOCRAT.
Lexical & encyclopedic knowledge in anaphora resolution
Arguably the most important source of errors, especially for nominals
Importance of this sort of information perceived very early on (e.g., Charniak and Winograd’s use of frame-based representations to model simple inferences)
But methods used to deploy it too labour-intensive and hard to evaluate
Current challenge: to find ways of extracting and deploying this knowledge automatically
Formal models
To recover the preferred interpretation of “John hid Bill’s car keys. He was drunk”
(Hobbs, 1979) You need to (defeasibly) infer:
That utterances may be used to explain previous statements by providing a cause for certain events;
That somebody being drunk may cause other people not to want that person to drive;
That one way to prevent somebody driving is to hide that person’s car keys
These analyses plausible …
… and implementable (e.g., in Hobbs’ TACITUS, SRI’s Core Language Engine)
… But very brittle Need theories of ‘robust inferencing’ Also: need to develop ways of acquiring
such knowledge Most work in the past ten years has
focused on this.
Alternatives to hand-coding for a specific domain
Using an existing resource, even if hand-coded E.g., WordNet: Harabagiu, Poesio &
Vieira Extracting knowledge from corpora Extracting knowledge from resources
such as lexica and encyclopedias Recently: Wikipedia
Using Wordnet
WordNet was the first publically available repository of ‘semantic-network’ – style knowledge
Good coverage of hyponymy relations (around 60%)
Not so good for meronymy. Also, knowledge in WordNet often hard to find.
The case of HOUSE
ARTIFACT
HOUSING BUILDING
HOUSE HOME ROOM
WALL FLOOR
IS-A IS-A
IS-AIS-A PART-OF
PART-OF PART-OF
Extracting lexical information from corpora for anaphora resolution
Using vector-based representations Poesio et al 1998: LSA-style vector
models for synonymy Using patterns
UNSUPERVISED RELATION EXTRACTION USING PATTERNS
HEARST 1998: HYPONYMYNP {, NP}* {,} or other NPbruises …… broken bones, and other INJURIESHYPONYM (bruise, injury)
POESIO ET AL, 2002: MERONYMYthe N of the N is ….the WHEEL of the CAR is …MERONYM (wheel, car)
Used for anaphora resolution by Markert & Nissim (2005), Versley (2006, 2007) (German)
Experiment: adding Web patterns to Soon et al baseline
Versley, 2007 (Johns Hopkins workshop)
Using web patterns to resolve references to nominals
Knowledge about semantic relations
Eckert & Strube 2001, Byron 2002: distinguish between abstract and concrete references
Dagan et al 1997, Kehler et al 2004: selectional restrictions for pronoun resolution
Encyclopedic knowledge in IDC
[The FCC] took [three specific actions] regarding [AT&T]. By a 4-0 vote, it allowed AT&T to continue offering special discount packages to big customers, called Tariff 12, rejecting appeals by AT&T competitors that the discounts were illegal. ….. …..[The agency] said that because MCI's offer had expired AT&T couldn't continue to offer its discount plan.
Why Wikipedia may help addressing the encyclopedic knowledge problem
http://en.wikipedia.org/wiki/FCC:
The Federal Communications Commission (FCC) is an independent United States government agency, created, directed, and empowered by Congressional statute (see 47 U.S.C. § 151 and 47 U.S.C. § 154).
Using Wikipedia for anaphora resolution
Using category structure Using disambiguation pages Using links
Problems with categories
Category structure is becoming unusable (too many categories)
Two solutions: Find ways of pruning the category tree Use other types of information in
Wikipedia
Results (ACE-02 BNews)
Recall Precision F-Score
SoonEtAl 51.2% 69.9% 59.1%
WikiCats 56.8% 68.0% 61.9%
WikiOther 56.5% 67.9% 61.7%
All 60.7% 65.4% 63.0%
From mentions to entities
<ANAPHOR (j), ANTECEDENT (i)>
Soon et al: <mention (j), mention (i)>
Global models: <mention (j), entity (coref chain) (i)>
Entity features
Global models make it possible to use information about entities / coref chains to decide on antecedent Culotta et al: first-order features
E.g., Agree in gender with all mentions in chain / most mentions in chain / some mention in chain
Finkel and Manning: transitivity
GuiTAR: A general-purpose tool
XML in / XML out Implemented in Java
Data- and Application-level integration into NLP systems
New preprocessing modules / AR algorithms can easily be integrated
Version 3.03: Mitkov’s pronoun resolution algorithm (1998) Vieira and Poesio’s algorithm for DD resolution
(2000) + statistical DN classifier Boncheva et al algorithm for NE resolution
MAS-XML EXAMPLE
<s> … <ne id="ne139" cat="the-np" per="per3“ gen=”neut” num=”plur”>
<W P="DT">The</W><mod id="m89" type=”pre”>
<W P="JJ">fragile</W></mod><nphead>
<W P="NNS">eggs</W></nphead>
</ne> …</s>
<ante current="ne139" rel="ident"><anchor antecedent="ne112" />
</ante>
GUITAR STRENGHTS / WEAKNESSES
Strenghts Well engineered Modular Nice discourse model Interface to lexical knowledge
Weaknesses Very basic learning architecture No commonsense knowledge Algorithms other than DDs not high performance
top related