computational discourse f17 - carnegie mellon university
TRANSCRIPT
ComputationalDiscourse
11-711AlgorithmsforNLP7 December2017
WhatIsDiscourse?
Discourse isthecoherentstructureoflanguageabovethelevelofsentencesorclauses.Adiscourseisacoherentstructuredgroupofsentences.
Whatmakesapassagecoherent?
Apracticalanswer:Ithasmeaningfulconnectionsbetweenitsutterances.
CoverofShel Silverstein’sWheretheSidewalkEnds (1974)
ApplicationsofComputationalDiscourse
• Automaticessaygrading• Automaticsummarization• Meetingunderstanding• Dialoguesystems
Kindsofdiscourseanalysis
• Discourse,monologue,dialogue,(conversation)
• Discourse(SLP Ch.21)vs.(Spoken)DialogueSystems(SLP Ch.24)
Discoursemechanismsvs.Coherenceofthought
• “Longer-range”analysis(discourse)vs.“deeper”analysis(realsemantics):– JohnboughtacarfromBill– BillsoldacartoJohn– Theywerebothhappywiththetransaction
Coherence,Cohesion• Coherencerelations:– JohnhidBill’scarkeys.Hewasdrunk.– JohnhidBill’scarkeys.Helikesspinach.
• Entity-basedcoherence(Centering)andlexicalcohesion:– Johnwenttothestoretobuyapiano– Hehadgonetothestoreformanyyears– Hewasexcitedthathecouldfinallyaffordapiano– Hearrivedjustasthestorewasclosingforthedayversus– Johnwenttothestoretobuyapiano– Itwasastorehehadgonetoformanyyears– Hewasexcitedthathecouldfinallyaffordapiano– ItwasclosingforthedayjustasJohnarrived
CohesioninNLP
DiscourseSegmentation
Goal:Givenrawtext,separateadocumentintoalinearsequenceofsubtopics.
Pyramidfromcommons.wikimedia.org
Discoursesegmentation:TextTiling
• Usingdipsincohesion tosegmenttext.
SupervisedDiscourseSegmentation
Ourinstances:placemarkersbetweensentences(orparagraphsorclauses)Ourlabels:yes(markerisadiscourseboundary)orno(markerisnotadiscourseboundary)Whatfeaturesshouldweuse?• Discoursemarkersorcuewords• Wordoverlapbefore/afterboundary• Numberofcoreferencechainsthatcrossboundary• Others?
CoherenceinNLP
CoherenceRelationsS1:JohnwenttothebanktodeposithispaycheckS2:HethentookabustoBill’scardealershipS3:HeneededtobuyacarS4:Thecompanyheworksfornowisn’tnearabuslineS5:HealsowantedtotalkwithBillabouttheirsoccerleague
SomeCoherenceRelationsHowcanwelabeltherelationshipsbetweenutterancesinadiscourse?Afewexamples:• Explanation:InferthatthestateoreventassertedbyS1 causesorcouldcausethestateoreventassertedbyS0.
• Occasion:AchangeofstatecanbeinferredfromtheassertionofS0,whosefinalstatecanbeinferredfromS1,orviceversa.
• Parallel:Inferp(a1,a2,…) fromtheassertionofS0andp(b1,b2,…) fromtheassertionofS1,whereaiandbi aresimilarforalli.
RSTCoherenceRelations
RSTformalrelationdefinition
• Relationname:Evidence• Constr onN:RnotbelievingNenoughforW• Constr onS:RbelievesS,orwould• Constr onN+S:R’sbelievingSwouldincreaseR’sbelievingN
• Effects:R’sbeliefofNisincreased
AutomaticCoherenceAssignment
Givenasequenceofsentencesorclauses,wewanttoautomatically:• determinecoherencerelationsbetweenthem(coherencerelationassignment)
• extractatreeorgraphrepresentinganentirediscourse(discourseparsing)
AutomaticCoherenceAssignment
Verydifficult.Oneexistingapproachistousecuephrases.
JohnhidBill’scarkeysbecause hewasdrunk.Thescarecrowcametoaskforabrain.Similarly,the
tinmanwantsaheart.1) Identifycuephrasesinthetext.2) Segmentthetextintodiscoursesegments.3) Classifytherelationshipbetweeneach
consecutivediscoursesegment.
AutomaticCoherenceAssignment
• “Discourseparsing”?• Usecuephrases/discoursemarkers– although,but,because,yet,with,…– butoftenimplicit,asincarkeyexample
• Useabduction,defeasibleinference– Allmenaremortal–Maxwasmortal–MaybeMaxwasaman
• Thecitydeniedthedemonstratorsapermitbecausethey(feared/advocated)violence
Pragmatics
Pragmatics
Pragmaticsisabranchoflinguisticsdealingwithlanguageuseincontext.
Whenadiplomatsaysyes,hemeans‘perhaps’;Whenhesaysperhaps,hemeans‘no’;Whenhesaysno,heisnotadiplomat.
(VariouslyattributedtoVoltaire,H.L.Mencken,andCarlJung)
Quotefromhttp://plato.stanford.edu/entries/pragmatics/
InContext?
• Socialcontext– Socialidentities,relationships,andsetting
• Physicalcontext–Where?Whatobjectsarepresent?Whatactions?
• Linguisticcontext– Conversationhistory
• Otherformsofcontext– Sharedknowledge,etc.
SpeechActs
(Direct) Speech Acts
• Mood of a sentence indicates relation between speaker and the concept (proposition) defined by the LF
• There can be operators that represent these relations:
• ASSERT: the proposition is proposed as a fact
• YN-QUERY: the truth of the proposition is queried
• COMMAND: the proposition describes a requested action
• WH-QUERY: the proposition describes an object to be identified
Indirect Speech Acts
• Can you pass the salt?
• It’s warm in here.
Austin,Howtodothingswithwords
• Inadditiontojustsayingthings,sentencesperformactions.
• Whenthesesentencesareuttered,theimportantthingisnottheirtruthvalue,butthefelicitousness oftheaction (e.g.,doyouhavetheauthority todoit):– InamethisshiptheQueenElizabeth.– Itakethismantobemyhusband.– Ibequeaththiswatchtomybrother.– Ideclarewar.
• http://en.wikipedia.org/wiki/J._L._Austin
Performative sentences• Youcantellwhethersentencesareperformativebyadding“hereby”:– IherebynamethisshiptheQueenElizabeth.– I herebytakethismantobemyhusband.– Iherebybequeaththiswatchtomybrother.– Iherebydeclarewar.
• Non-performative sentencesdonotsoundgoodwithhereby:– Birdsherebysing.– ThereisherebyfightinginSyria.
Austincontinued
• Locution:saysomewords• Illocution:anactionperformedin sayingwords– Ask,promise,command
• Perlocution:anactionperformedby sayingwords,probablytheeffectthatanillocutionhasonthelistener.– Persuade,convince,scare,elicitananswer,etc.
Searle’sspeechactsSearle(1975) hassetupthefollowingclassificationofillocutionaryspeechacts:
• assertives =speechactsthatcommitaspeakertothetruthoftheexpressed proposition,e.g.recitinga creed
• directives =speechactsthataretocausethehearertotakeaparticularaction,e.g.requests,commandsandadvice
• commissives =speechactsthatcommitaspeakertosomefutureaction,e.g.promisesandoaths
• expressives =speechactsthatexpressthespeaker'sattitudesandemotionstowardstheproposition,e.g.congratulations,excusesandthanks
• declarations =speechactsthatchangetherealityinaccordwiththepropositionofthedeclaration,e.g.baptisms,pronouncingsomeoneguiltyorpronouncingsomeonehusbandandwife
• http://en.wikipedia.org/wiki/Speech_act
Searleexample
• Indirectspeechacts:– Canyoupassthesalt?• Hastheformofaquestion,buttheeffectofadirective.
SpeechActsinNLP
Task-OrientedDialogue
• Makingtravelreservations(flight,hotelroom,etc.)
• Schedulingameeting.• Taskorienteddialoguesthatarefrequentlydonewithcomputers:– Findingoutwhenthenextbusis.–Makingapaymentoverthephone.
Waystoaskforaroom
• I’dliketomakeareservation• I’mcallingtomakeareservation• Doyouhaveavacancyon...• CanIreservearoom• Isitpossibletoreservearoom
Domain-specificspeechacts:travelscheduling(NESPOLE!Project)
(aprimitiveversionofthespeechtranslation)
• 61.2.3olangITAlangITAPrvIRST“Telefonoperprenotaredellestanzeperquattro colleghi”
• 61.2.3olang ITAlang ENGPrv IRST“Iamcallingtobooksomeroomsforfourcolleagues”
• 61.2.3IFPrv IRSTc:request-action+reservation+room (room-spec=(room,quantity=some),for-whom=(colleague,quantity=4))
• comments:dial-oo5-spkB-roca0-02-3
Task-orienteddialogueactsrelatedtonegotiation
• Suggest– Irecommendthishotel.
• Offer– Icansendsomebrochures.– HowaboutifIsendsomebrochures.
• Accept– Sure.Thatsoundsfine.
• Reject– No.Idon’tlikethatone.
ExamplesofSpeechActinventoriesusedinlanguagetechnologies
• Theseinventoriesareactuallyannotationschemes.
• Theyareusedforcorpusannotation.• Thecorpusannotationisusedforautomatedlearning.
• Theyarehighlydevelopedandcheckedforintercoder agreement.– Butstilltakealongtimetolearn.
Examplesoftask-orientedspeechacts• Identifyself:
– ThisisLori– MynameisLori– I’mLori– Lorihere
• Soundcheck:Canyouhearme?• Metadialogueact:Thereisaproblem.• Greet:Hello.• Request-information:–Whereareyougoing.– Tellmewhereyouaregoing.
Examplesoftask-orientedspeechacts• Backchannel:– Soundsyoumaketoindicatethatyouarestilllistening– ok,m-hm
• Apologize/replytoapology• Thank/replytothanks• Requestverification/Verify– Sothat’s2:00?Yes.2:00.
• Resumetopic– Backtotheaccommodations….
• Answerayes/noquestion:yes,no.
DAMSLDialogueActMarkupinSeveralLayers
• Fortask-orientedornon-task-orienteddialogue.• However,muchofthedevelopmentwasrelatedtotask-
orienteddialogues:– Trainscorpus– Maptask corpus– Meetingschedulingcorpus
• Althoughithasbeenusedfornon-task-orienteddialogue:– Switchboardcorpus(JHUworkshop1997)– SpanishCallHome corpus(ClarityProject,Waibel,Levin,Lavie)– Textmessagecorpus(Proprietaryproject,Levin,Rudnicky,Tenny)
• Whatarethelayers?– Forwardfunction:offer,ask– Backwardfunction:backchannel,accept,reject
Forwardlookingfunctions• Statement
– Assert– Reassert– Other-statement
• Influencing-addressee-future-action– Open-option– Action-directive
• Info-request• Committing-speaker-future-action
– Offer– Commit
• ConventionalOpeningClosing• Explicit-performative• Exclamation• Other-forward-function
Backwardlookingfunctions• Agreement
– Accept– Acceptpart– Maybe– Rejectpart– Reject– Hold
• Understanding– Signalnon-understanding– Signalunderstanding
• Acknowledge• Repeat• Complete
– Correctmisspeaking• Answer
Now,afamousbadidea(linkedtoagoodidea)
Grice’sMaxims
• Whydothesemakesense?– Areyou21?– Yes.I’m25.
– I’mhungry.– I’llgetmykeys.
–WherecanIgetcigarettes?– Thereisagasstationacrossthestreet.
Grice’sMaxims• Whyarethesestrange?
– (Thestudentsareallgirls.)– Somestudentsaregirls.
– (Therearesevennon-stopflights.)– Therearethreenon-stopflights.
• Jurafsky andMartin,page820
– (Inaletterofrecommendationforajob)– Istronglypraisetheapplicant’simpeccablehandwriting.
Grice’sCooperativePrinciple
• “Makeyourcontributionsuchasitisrequired,atthestageatwhichitoccurs,bytheacceptedpurposeordirectionofthetalkexchangeinwhichyouareengaged.”
• TheCooperativePrincipleisgoodandright.
• Ontheotherhand,wehavetheMaxims:
Grice’sactualMaxims• MaximofQuality
– Trytosaysomethingtrue;donotsaysomethingfalseorforwhichyoulackevidence.
• MaximofQuantity– Sayasmuchasisrequiredtobeinformative– Donotmakeyourcontributionmoreinformativethanrequired
• MaximofRelevance– BeRelevant
• MaximofManner– Beperspicuous– Avoidambigtuity– Bebrief– Beorderly
Flouting theCooperativePrinciple
• “Nicethrow.”(saidafterterriblethrow)
• “Ifyourunalittleslower,you’llnevercatchuptotheball.”(duringmediocrepursuitofball)
• Youcan indeedimplysomethingbyclearlyviolatingtheprinciple.– TheMaximsstill suck.
Flout ≠Flaunt
• Flout:openlydisregard(arule,laworconvention).
• Flaunt:display(something)ostentatiously,especiallyinordertoprovokeenvyoradmirationortoshowdefiance.– Source:Google
MypaperontheMaxims
• Grice'sMaxims:"DotheRightThing" byRobertE.Frederking.ArguesthattheGricean maximsaretoovaguetobeusefulfornaturallanguageprocessing.[fromWikipediaarticle]
• “Iusedtothinkyouwereaniceguy.”– Actualquotefromagradstudent,afterreadingthepaper
Referenceresolution
ReferenceResolution:example• VictoriaChen,CFOofMegabucksBankingCorpsince2004,sawherpay jump20%,to$1.3million,asthe37-year-oldalsobecametheDenver-basedcompany’spresident.Ithasbeentenyearssinceshe cametoMegabucks fromrivalLotsaloot.
• Shouldgive4coreference chains:– {VictoriaChen,CFOofMegabucksBankingCorpsince2004,her,the37-year-old,theDenver-basedcompany’spresident,she}
– {MegabucksBankingCorp,theDenver-basedcompany,Megabucks}
– {herpay}– {Lotsaloot}.
CoreferenceResolution
Marypickeduptheball.Shethrewittome.
Referenceresolution
Marypickeduptheball.Shethrewittome.
(Co)ReferenceResolution
• Determiningthereferentofareferringexpression.Anaphora,antecedentscorefer.
• 1961FordFalcon:it,this,that,thiscar,thecar,theFord,theFalcon,myfriend’scar,…
• Coreference chainsarepartofcohesion• Note:otherkindsofreferents:– AccordingtoDoug,SuejustboughttheFordFalcon• Butthat turnedouttobealie• Butthat wasfalse• That struckmeasafunnywaytodescribethesituation• That causedafinancialproblemforSue
TypesofReferringExpressions• IndefiniteNPs:a/an,some,this,ornothing– newentities;specific/non-specificambiguity
• DefiniteNPs:usuallythe– anentityidentifiablebythehearer
• Pronouns:he,them,it,etc.Alsocataphora.– strongconstraintsontheiruse– canbebound:Everystudentimprovedhisgrades
• Demonstratives:this,that• Names:construedtobeunique,buttheyaren’t– IsthattheBobinLTIortheBobintheLaneCenter?
Informationstructure:given/new
• Wherearemyshoes?Yourshoes areinthecloset• What’sinthecloset?– ??Yourshoes areinthecloset.– Yourshoes areinthecloset.
• Definiteness/pronoun,length,positioninS
Complications
• Inferrables:Somecar.…adoor…theengine…• Generics:AtCMUyouhavetoworkhard.• Pleonastic/clefts/extraposition:– Itisraining.Itwasmewhocalled.Itwasgoodthat…
ReferenceResolution
Goal:determinewhatentitiesarereferredtobywhichlinguisticexpressions.
Thediscoursemodelcontainsoureligiblesetofreferents.
Pronouns:FiltersandPreferences
Pronounreferenceresolution:filters
• Agreementinnumber,person,gender• Pittsburghdialect:yinz=youse=y’all• UKdialect:Newcastleareaphysicalteam.
– Lcanhave>2numbers,>3persons,or>3genders• Bindingtheory:reflexive required/prohibited:– JohnboughthimselfanewFord.[himself=John]– JohnboughthimanewFord.[him!=John]– JohnsaidthatBillboughthimanewFord.[him!=Bill]– JsaidthatBboughthimselfanewF.[himself=Bill]– HesaidthatheboughtJanewFord.[both he!=J]
Pronounreferenceresolution:preferences• Recency:preferenceformostrecentreferent• GrammaticalRole:subj>obj>others– BillywenttothebarwithJim.Heorderedrum.
• Repeatedmention:Billyhadbeendrinkingfordays.Hewenttothebaragaintoday.Jimwentwithhim.Heorderedrum.
• Parallelism:JohnwentwithJimtoonebar.Billwentwithhimtoanother.
• Verbsemantics:Johnphoned/criticizedBill.Helostthelaptop.
• Selectional restrictions:Johnparkedhiscarinthegarageafterdrivingitaroundforhours.
Threecomputationalapproachestopronouns
PNref.res.1:HobbsAlgorithm
• Algorithmforwalkingthroughparsesofcurrentandprecedingsentences
• Simple,oftenusedasbaseline
• Requiresparser,morphgenderandnumber– plusheadrulesandWordNet forNPgender
• Implementsbindingtheory,recency,andgrammaticalrolepreferences
PNref.res.2:Centeringtheory• Claim:asingleentityis“centered”ineachS• Backward-lookingcenter,Forward-lookingcenters• Cb =mosthighlyrankedCf usedfromprev.S• Rank:Subj>ExistPredNom>Obj>IndObj-Obl>DemAdvPP
• Definedtransitions:(Cp isfrontofCf list)
Rule1:IfanyCf usedasPron+1,thenCb(n+1) mustbeProtooRule2:Rank:Continue>Retain>Smooth>Rough
U1:JohnsawaFordatthedealershipCb:NILCf:John,Ford,dealership
U2:HeshowedittoBob[Bob!=he]He=John,it={Ford,dealership}Cb=John• (it->Ford)=>Cf:{John,Ford,Bob}=>CONTINUE[tie-winner]• (it->dealership)=>Cf:{John,dealer,Bob}=>CONTINUE
U3:Heboughtit[dealership isnowunavailable]He={John,Bob},it=Ford• (he->John)=>Cb=John,Cf={John,Ford}=>CONTINUE[Win]• (he->Bob)=>Cb=Bob,Cf={Bob,Ford}=>SMOOTH
Centeringtheory
• SamerequirementsasHobbs• ImplementsGrammaticalRole,Recency,andRepeatedMention
• Canmakemistakes:– Bobopenedanewdealershiplastweek– JohntookalookattheFordsinhislot[Cb=Bob]– Heendedupbuyingone• He=Bob=>CONTINUE,He=John=>SMOOTH
PNref.res.3:Log-linearmodel
• Supervised:hand-labelled coref corpus• Rule-basedfilteringofnon-referentialpronouns• Features,valuesforHe inU3:
GeneralReferenceResolution
GeneralCoreference Resolution• VictoriaChen,CFOofMegabucksBankingCorpsince2004,sawherpay jump20%,to$1.3million,asthe37-year-oldalsobecametheDenver-basedcompany’spresident.Ithasbeentenyearssinceshe cametoMegabucks fromrivalLotsaloot.
• Shouldgive4coreference chains:– {VictoriaChen,CFOofMegabucksBankingCorpsince2004,her,the37-year-old,theDenver-basedcompany’spresident,she}
– {MegabucksBankingCorp,theDenver-basedcompany,Megabucks}
– {herpay}– {Lotsaloot}.
President Park Geun-hye of South Korea ordered the country’s military on Monday to deliver a strong and immediate response to any North Korean provocation, the latest turn in a war of words that has become a test of resolve for the relatively unproven leaders in both the North and South.“I consider the current North Korean threats very serious,” Ms. Park told the South’s generals. “If the North attempts any provocation against our people and country, you must respond strongly at the first contact with them without any political consideration.“As top commander of the military, I trust your judgment in the face of North Korea’s unexpected surprise provocation,” she added.Since Kim Jong-un took power after the death of his father, Kim Jong-il, in late 2011, the North has taken a series of provocative steps and amplified threats against Washington and Seoul to much louder and more menacing levels. The North has launched a three-stage rocket, tested a nuclear device and threatened to hit major American cities with nuclear-armed ballistic missiles. And Mr. Kim has declared that the Korean Peninsula has reverted to a “state of war.”
President Park Geun-hye of South Korea ordered the country’s military on Monday to deliver a strong and immediate response to any North Korean provocation, the latest turn in a war of words that has become a test of resolve for the relatively unproven leaders in both the North and South.“I consider the current North Korean threats very serious,” Ms. Park told the South’s generals. “If the North attempts any provocation against our people and country, you must respond strongly at the first contact with them without any political consideration.“As top commander of the military, I trust your judgment in the face of North Korea’s unexpected surprise provocation,” she added.Since Kim Jong-un took power after the death of his father, Kim Jong-il, in late 2011, the North has taken a series of provocative steps and amplified threats against Washington and Seoul to much louder and more menacing levels. The North has launched a three-stage rocket, tested a nuclear device and threatened to hit major American cities with nuclear-armed ballistic missiles. And Mr. Kim has declared that the Korean Peninsula has reverted to a “state of war.”
President Park Geun-hye of South Korea ordered the country’s military on Monday to deliver a strong and immediate response to any North Korean provocation, the latest turn in a war of words that has become a test of resolve for the relatively unproven leaders in both the North and South.“I consider the current North Korean threats very serious,” Ms. Park told the South’s generals. “If the North attempts any provocation against our people and country, you must respond strongly at the first contact with them without any political consideration.“As top commander of the military, I trust your judgment in the face of North Korea’s unexpected surprise provocation,” she added.Since Kim Jong-un took power after the death of his father, Kim Jong-il, in late 2011, the North has taken a series of provocative steps and amplified threats against Washington and Seoul to much louder and more menacing levels. The North has launched a three-stage rocket, tested a nuclear device and threatened to hit major American cities with nuclear-armed ballistic missiles. And Mr. Kim has declared that the Korean Peninsula has reverted to a “state of war.”
President Park Geun-hye of South Korea ordered the country’s military on Monday to deliver a strong and immediate response to any North Korean provocation, the latest turn in a war of words that has become a test of resolve for the relatively unproven leaders in both the North and South.“I consider the current North Korean threats very serious,” Ms. Park told the South’s generals. “If the North attempts any provocation against our people and country, you must respond strongly at the first contact with them without any political consideration.“As top commander of the military, I trust your judgment in the face of North Korea’s unexpected surprise provocation,” she added.Since Kim Jong-un took power after the death of his father, Kim Jong-il, in late 2011, the North has taken a series of provocative steps and amplified threats against Washington and Seoul to much louder and more menacing levels. The North has launched a three-stage rocket, tested a nuclear device and threatened to hit major American cities with nuclear-armed ballistic missiles. And Mr. Kim has declared that the Korean Peninsula has reverted to a “state of war.”
President Park Geun-hye of South Korea ordered the country’s military on Monday to deliver a strong and immediate response to any North Korean provocation, the latest turn in a war of words that has become a test of resolve for the relatively unproven leaders in both the North and South.“I consider the current North Korean threats very serious,” Ms. Park told the South’s generals. “If the North attempts any provocation against our people and country, you must respond strongly at the first contact with them without any political consideration.“As top commander of the military, I trust your judgment in the face of North Korea’s unexpected surprise provocation,” she added.Since Kim Jong-un took power after the death of his father, Kim Jong-il, in late 2011, the North has taken a series of provocative steps and amplified threats against Washington and Seoul to much louder and more menacing levels. The North has launched a three-stage rocket, tested a nuclear device and threatened to hit major American cities with nuclear-armed ballistic missiles. And Mr. Kim has declared that the Korean Peninsula has reverted to a “state of war.”
High-LevelRecipeforCoreferenceResolution
1. ParsethetextandidentifyNPs;then2. ForeverypairofNPs,carryoutbinary
classification:coreferential ornot?3. Collecttheresultsintocoreferential chains
Whatdoweneed?-Achoiceofclassifier-Lotsoflabeleddata-Features
Features?• EditdistancebetweenthetwoNPs• ArethetwoNPsthesameNERtype?• Appositivesyntax– “AlanShepherd,thefirstAmericanastronaut…”
• Proper/definite/indefinite/pronoun• Gender• Number• Distanceinsentences• NumberofNPsbetween• Grammaticalrole• etc.
MoreCoreference Resolution
• Combinebest:ENCORE(BoLinetal2010)• MLforCross-DocCoref (Rushin Shahetal2011)
EntityLinking
Apple updateditsinvestorrelationspagetodaytonotethatitwillannounceitsearningsforthesecondfiscalquarter(firstcalendarquarter)of2015onMonday,April27.
Newstextfromhttp://www.macrumors.com/2015/03/30/apple-to-announce-q2-2015-earnings-on-april-27/
OneApproachtoEntityLinking
Usesupervisedlearning:Trainonknownreferencestoeachentity.Usefeaturesfromcontext(bagofwords,syntax,etc.).
Questions?