lexical semantics and word sense disambiguationkathy/nlp/2017/classslides/class12...lexical...
TRANSCRIPT
![Page 1: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/1.jpg)
LexicalSemanticsandWordSenseDisambiguation
![Page 2: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/2.jpg)
Announcements• Midtermsampleques/onsonwebsite• Nextclass:midtermreviewforpartoftheclass.PostyourwishesfortopicsforthereviewonPiazza
• HW1gradesout.Meanis81.Nicegoing!• Followingtopics:seman/cparsing,thentodistributedseman/csandwordembeddings,neuralnets.
![Page 3: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/3.jpg)
Polysemy• ThebankisconstructedfromredbrickIwithdrewthemoneyfromthebank• Arethosethesamesense?• OrconsiderthefollowingWSJexample• Whilesomebanksfurnishspermonlytomarriedwomen,othersarelessrestric/ve• Whichsenseofbankisthis?• Isitdis/nctfrom(homonymouswith)theriverbanksense?• Howaboutthesavingsbanksense?
![Page 4: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/4.jpg)
![Page 5: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/5.jpg)
Polysemy• Asinglelexemewithmul/plerelatedmeanings(bankthebuilding,bankthefinancialins/tu/on)• Mostnon-rarewordshavemul/plemeanings• Thenumberofmeaningsisrelatedtoitsfrequency• Verbstendmoretopolysemy• Dis/nguishingpolysemyfromhomonymyisn’talwayseasy(ornecessary)
![Page 6: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/6.jpg)
MetaphorandMetonymy• Specifictypesofpolysemy• Metaphor:• GermanywillpullSloveniaoutofitseconomicslump.• Ispent2hoursonthathomework.• Metonymy• TheWhiteHouseannouncedyesterday.• Thischaptertalksaboutpart-of-speechtagging• Bank(building)andbank(financialins/tu/on)
![Page 7: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/7.jpg)
Howdoweknowwhenawordhasmorethanonesense?• ATISexamples• Whichflightsservebreakfast?• DoesAmericaWestservePhiladelphia?
• The“zeugma”test:
• ?DoesUnitedservebreakfastandSanJose?
![Page 8: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/8.jpg)
Synonyms• Wordthathavethesamemeaninginsomeorallcontexts.• filbert/hazelnut• couch/sofa• big/large• automobile/car• vomit/throwup• Water/H20
• Twolexemesaresynonymsiftheycanbesuccessfullysubs/tutedforeachotherinallsitua/ons• Ifsotheyhavethesameproposi+onalmeaning
![Page 9: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/9.jpg)
Synonyms• Buttherearefew(orno)examplesofperfectsynonymy.• Whyshouldthatbe?• Evenifmanyaspectsofmeaningareiden/cal• S/llmaynotpreservetheacceptabilitybasedonno/onsofpoliteness,slang,register,genre,etc.
• Example:• WaterandH20
![Page 10: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/10.jpg)
Somemoreterminology• Lemmasandwordforms• Alexemeisanabstractpairingofmeaningandform• Alemmaorcita+onformisthegramma/calformthatisusedtorepresentalexeme.• Carpetisthelemmaforcarpets• Dormiristhelemmaforduermes.
• Specificsurfaceformscarpets,sung,duermesarecalledwordforms
• Thelemmabankhastwosenses:• Instead,abankcanholdtheinvestmentsinacustodialaccountintheclient’sname• Butasagricultureburgeonsontheeastbank,theriverwillshrinkevenmore.
• Asenseisadiscreterepresenta/onofoneaspectofthemeaningofaword
![Page 11: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/11.jpg)
Synonymyisarelationbetweensensesratherthanwords
• Considerthewordsbigandlarge• Aretheysynonyms?• Howbigisthatplane?• WouldIbeflyingonalargeorsmallplane?
• Howabouthere:• MissNelson,forinstance,becameakindofbigsistertoBenjamin.
• ?MissNelson,forinstance,becameakindoflargesistertoBenjamin.
• Why?• bighasasensethatmeansbeingolder,orgrownup• largelacksthissense
![Page 12: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/12.jpg)
Antonyms• Sensesthatareoppositeswithrespecttoonefeatureoftheirmeaning• Otherwise,theyareverysimilar!• dark/light• short/long• hot/cold• up/down• in/out• Moreformally:antonymscan• defineabinaryopposi/onoratoppositeendsofascale(long/short,fast/slow)• Bereversives:rise/fall,up/down
![Page 13: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/13.jpg)
![Page 14: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/14.jpg)
Hyponymy• Onesenseisahyponymofanotherifthefirstsenseismorespecific,deno/ngasubclassoftheother• carisahyponymofvehicle• dogisahyponymofanimal• mangoisahyponymoffruit
• Conversely• vehicleisahypernym/superordinateofcar• animalisahypernymofdog• fruitisahypernymofmango
superordinate vehicle fruit furniture mammal
hyponym car mango chair dog
![Page 15: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/15.jpg)
Hypernymymoreformally• Extensional:• Theclassdenotedbythesuperordinate• extensionallyincludestheclassdenotedbythehyponym
• Entailment:• AsenseAisahyponymofsenseBifbeinganAentailsbeingaB
• Hyponymyisusuallytransi/ve• (AhypoBandBhypoCentailsAhypoC)
![Page 16: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/16.jpg)
![Page 17: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/17.jpg)
II.WordNet• Ahierarchicallyorganizedlexicaldatabase• On-linethesaurus+aspectsofadic/onary
• Versionsforotherlanguagesareunderdevelopment
Category Unique Forms
Noun 117,097
Verb 11,488
Adjective 22,141
Adverb 4,601
![Page 18: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/18.jpg)
WordNet
• Whereitis:• hdps://wordnet.princeton.edu/
![Page 19: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/19.jpg)
FormatofWordnetEntries
![Page 20: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/20.jpg)
WordNetNounRelations
![Page 21: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/21.jpg)
WordNetVerbRelations
![Page 22: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/22.jpg)
WordNetHierarchies
![Page 23: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/23.jpg)
Howis“sense”deJinedinWordNet?• Thesetofnear-synonymsforaWordNetsenseiscalledasynset(synonymset);it’stheirversionofasenseoraconcept• Example:chumpasanountomean• ‘apersonwhoisgullibleandeasytotakeadvantageof’
• Eachofthesesensessharethissamegloss• ThusforWordNet,themeaningofthissenseofchumpisthislist.
![Page 24: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/24.jpg)
Wordnetexample
![Page 25: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/25.jpg)
WordSenseDisambiguation
![Page 26: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/26.jpg)
WordSenseDisambiguation(WSD)• Given• awordincontext,• Afixedinventoryofpoten/alwordsenses
• decidewhichsenseofthewordthisis.• English-to-SpanishMT• InventoryissetofSpanishtransla/ons
• SpeechSynthesis• Inventoryishomographswithdifferentpronuncia/onslikebassandbow
• Automa/cindexingofmedicalar/cles• MeSH(MedicalSubjectHeadings)thesaurusentries
![Page 27: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/27.jpg)
TwovariantsofWSDtask• LexicalSampletask• Smallpre-selectedsetoftargetwords• Andinventoryofsensesforeachword• All-wordstask• Everywordinanen/retext• Alexiconwithsensesforeachword• Sortoflikepart-of-speechtagging• Excepteachlemmahasitsowntagset
![Page 28: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/28.jpg)
Approaches• Supervised• Semi-supervised• Unsupervised
• Dic/onary-basedtechniques• Selec/onalAssocia/on
• Lightlysupervised• Bootstrapping• PreferredSelec/onalAssocia/on
![Page 29: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/29.jpg)
SupervisedMachineLearningApproaches
• Supervisedmachinelearningapproach:• atrainingcorpusof?• usedtotrainaclassifierthatcantagwordsinnewtext• Justaswesawforpart-of-speechtagging,sta/s/calMT.
• Summaryofwhatweneed:• thetagset(“senseinventory”)• thetrainingcorpus• Asetoffeaturesextractedfromthetrainingcorpus• Aclassifier
![Page 30: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/30.jpg)
![Page 31: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/31.jpg)
SupervisedWSD1:WSDTags• What’satag?
![Page 32: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/32.jpg)
WordNet} hdp://www.cogsci.princeton.edu/cgi-bin/webwn
![Page 33: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/33.jpg)
WordNetBassThenoun``bass''has8sensesinWordNet1. bass-(thelowestpartofthemusicalrange)2. bass,basspart-(thelowestpartinpolyphonicmusic)3. bass,basso-(anadultmalesingerwiththelowestvoice)4. seabass,bass-(fleshoflean-fleshedsaltwaterfishofthefamilySerranidae)5. freshwaterbass,bass-(anyofvariousNorthAmericanlean-fleshedfreshwaterfishesespeciallyof
thegenusMicropterus)6. bass,bassvoice,basso-(thelowestadultmalesingingvoice)7. bass-(thememberwiththelowestrangeofafamilyofmusicalinstruments)8. bass-(nontechnicalnameforanyofnumerousediblemarineandfreshwaterspiny-finnedfishes)
![Page 34: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/34.jpg)
Inventoryofsensetagsforbass
![Page 35: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/35.jpg)
SupervisedWSD2:Getacorpus
• Lexicalsampletask:• Line-hard-servecorpus-4000examplesofeach• Interestcorpus-2369sense-taggedexamples
• Allwords:• Seman+cconcordance:acorpusinwhicheachopen-classwordislabeledwithasensefromaspecificdic/onary/thesaurus.• SemCor:234,000wordsfromBrownCorpus,manuallytaggedwithWordNetsenses
• SENSEVAL-3compe//oncorpora-2081taggedwordtokens
![Page 36: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/36.jpg)
SupervisedWSD3:Extractfeaturevectors
• Weaver(1955)• Ifoneexaminesthewordsinabook,oneata/measthroughanopaquemaskwithaholeinitonewordwide,thenitisobviouslyimpossibletodetermine,oneata/me,themeaningofthewords.[…]Butifonelengthenstheslitintheopaquemask,un/lonecanseenotonlythecentralwordinques/onbutalsosayNwordsoneitherside,thenifNislargeenoughonecanunambiguouslydecidethemeaningofthecentralword.[…]Theprac/calques/onis:``WhatminimumvalueofNwill,atleastinatolerablefrac/onofcases,leadtothecorrectchoiceofmeaningforthecentralword?''
![Page 37: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/37.jpg)
• dishes• bass
![Page 38: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/38.jpg)
• washingdishes.• simpledishesincluding• convenientdishesto• ofdishesand• freebasswith• poundbassof• andbassplayer• hisbasswhile
![Page 39: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/39.jpg)
![Page 40: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/40.jpg)
![Page 41: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/41.jpg)
![Page 42: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/42.jpg)
![Page 43: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/43.jpg)
• “Inourhouse,everybodyhasacareerandnoneofthemincludeswashingdishes,”hesays.• Inher/nykitchenathome,Ms.Chenworksefficiently,s/r-fryingseveralsimpledishes,includingbraisedpig’searsandchckenliverswithgreenpeppers.• Postquickandconvenientdishestofixwhenyourinahurry.• Japanesecuisineoffersagreatvarietyofdishesandregionalspecial/es
![Page 44: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/44.jpg)
• Weneedmoregoodteachers–rightnow,thereareonlyahalfadozenwhocanplaythefreebasswithease.• Thoughs/llafarcryfromthelake’srecord52-poundbassofadecadeago,“youcouldfilletthesefishagain,andthatmadepeoplevery,veryhappy.”Mr.Paulsonsays.• Anelectricguitarandbassplayerstandofftooneside,notreallypartofthescene,justasasortofnodtogringoexpecta/onsagain.• LowecaughthisbasswhilefishingwithproBillLeeofKilleen,Texas,whoiscurrentlyin144thplacewithtwobassweighing2-09.
![Page 45: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/45.jpg)
Featurevectors• Asimplerepresenta/onforeachobserva/on(eachinstanceofatargetword)• Vectorsofsetsoffeature/valuepairs• I.e.filesofcomma-separatedvalues
• ThesevectorsshouldrepresentthewindowofwordsaroundthetargetHowbigshouldthatwindowbe?
![Page 46: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/46.jpg)
Twokindsoffeaturesinthevectors
• Colloca+onalfeaturesandbag-of-wordsfeatures• Colloca+onal• Featuresaboutwordsatspecificposi/onsneartargetword• Ovenlimitedtojustwordiden/tyandPOS
• Bag-of-words• Featuresaboutwordsthatoccuranywhereinthewindow(regardlessofposi/on)• Typicallylimitedtofrequencycounts
![Page 47: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/47.jpg)
Examples• Exampletext(WSJ)• Anelectricguitarandbassplayerstandofftoonesidenotreallypartofthescene,justasasortofnodtogringoexpecta/onsperhaps• Assumeawindowof+/-2fromthetarget
![Page 48: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/48.jpg)
Examples• Exampletext• Anelectricguitarandbassplayerstandofftoonesidenotreallypartofthescene,justasasortofnodtogringoexpecta/onsperhaps• Assumeawindowof+/-2fromthetarget
![Page 49: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/49.jpg)
Collocational
• Posi/on-specificinforma/onaboutthewordsinthewindow• guitarandbassplayerstand• [guitar,NN,and,CC,player,NN,stand,VB]• Wordn-2,POSn-2,wordn-1,POSn-1,Wordn+1POSn+1…• Inotherwords,avectorconsis/ngof• [posi/onnword,posi/onnpart-of-speech…]
![Page 50: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/50.jpg)
Bag-of-words• Informa/onaboutthewordsthatoccurwithinthewindow.• Firstderiveasetoftermstoplaceinthevector.• Thennotehowoveneachofthosetermsoccursinagivenwindow.
![Page 51: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/51.jpg)
Co-OccurrenceExample• Assumewe’vesedledonapossiblevocabularyof12wordsthatincludesguitarandplayerbutnotandandstand
• guitarandbassplayerstand• [0,0,0,1,0,0,0,0,0,1,0,0]• Whicharethecountsofwordspredefinedase.g.,• [fish,fishing,viol,guitar,double,cello…
![Page 52: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/52.jpg)
ClassiJiers• OncewecasttheWSDproblemasaclassifica/onproblem,thenallsortsoftechniquesarepossible• NaïveBayes(theeasiestthingtotryfirst)• Decisionlists• Decisiontrees• Neuralnets• Supportvectormachines• Nearestneighbormethods…
![Page 53: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/53.jpg)
ClassiJiers• Thechoiceoftechnique,inpart,dependsonthesetoffeaturesthathavebeenused• Sometechniquesworkbeder/worsewithfeatureswithnumericalvalues• Sometechniquesworkbeder/worsewithfeaturesthathavelargenumbersofpossiblevalues• Forexample,thefeaturethewordtotheleAhasafairlylargenumberofpossiblevalues
![Page 54: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/54.jpg)
NaïveBayes• ŝ=p(s|V),or
• WheresisoneofthesensesSpossibleforawordwandVtheinputvectoroffeaturevaluesforw• Assumefeaturesindependent,soprobabilityofVistheproductofprobabili/esofeachfeature,givens,so• p(V)sameforanyŝ
• Then
)|1()|( s
n
jv jpsVp ∏
==
)|1()(maxargˆ s
n
jv jpsp
Sss ∏
=∈=
)()()|(maxarg
VpspsVp
Ss∈maxargSs∈
![Page 55: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/55.jpg)
• Howdowees/matep(s)andp(vj|s)?• p(si)ismax.likelihoodes/matefromasense-taggedcorpus(count(si,wj)/count(wj))–howlikelyisbanktomean‘financialins/tu/on’overallinstancesofbank?
• P(vj|s)ismax.likelihoodofeachfeaturegivenacandidatesense(count(vj,s)/count(s))–howlikelyisthepreviouswordtobe‘river’whenthesenseofbankis‘financialins/tu/on’
• Calculatetakethehighestscoringsenseasthemostlikelychoice
)|1()(maxargˆ s
n
jv jpsp
Sss ∏
=∈=
![Page 56: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/56.jpg)
NaïveBayesTest• Onacorpusofexamplesofusesofthewordline,naïveBayesachievedabout73%correct
• Good?
![Page 57: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/57.jpg)
DecisionLists:anotherpopularmethod• Acasestatement….
![Page 58: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/58.jpg)
LearningDecisionLists• Restricttheliststorulesthattestasinglefeature(1-decisionlistrules)• Evaluateeachpossibletestandrankthembasedonhowwelltheywork.• Gluethetop-Nteststogetherandcallthatyourdecisionlist.
![Page 59: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/59.jpg)
Yarowsky
• Onabinary(homonymy)dis/nc/onusedthefollowingmetrictorankthetests
• Thisgivesabout95%onthistest…
€
P(Sense1 |Feature)P(Sense2 |Feature)
![Page 60: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/60.jpg)
WSDEvaluationsandbaselines• Invivoversusinvitroevalua/on• Invitroevalua/onismostcommonnow• Exactmatchaccuracy• %ofwordstaggediden/callywithmanualsensetags
• Usuallyevaluateusingheld-outdatafromsamelabeledcorpus• Problems?• Whydowedoitanyhow?
• Baselines• Mostfrequentsense• TheLeskalgorithm
![Page 61: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/61.jpg)
MostFrequentSense• Wordnetsensesareorderedinfrequencyorder• So“mostfrequentsense”inwordnet=“takethefirstsense”• SensefrequenciescomefromSemCor
![Page 62: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/62.jpg)
Ceiling• Humaninter-annotatoragreement• Compareannota/onsoftwohumans• Onsamedata• Givensametaggingguidelines
• Humanagreementsonall-wordscorporawithWordnetstylesenses• 75%-80%
![Page 63: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/63.jpg)
UnsupervisedMethodsWSD:Dictionary/Thesaurusmethods• TheLeskAlgorithm• Selec/onalRestric/ons
![Page 64: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/64.jpg)
SimpliJiedLesk
![Page 65: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/65.jpg)
OriginalLesk:pinecone
![Page 66: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/66.jpg)
CorpusLesk• Addcorpusexamplestoglossesandexamples• Thebestperformingvariant
![Page 67: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/67.jpg)
DisambiguationviaSelectionalRestrictions• “Verbsareknownbythecompanytheykeep”• Differentverbsselectfordifferentthema/croles
washthedishes(takeswashable-thingaspa/ent)servedeliciousdishes(takesfood-typeaspa/ent)
• Method:anotherseman/cadachmentingrammar• Seman/cadachmentrulesareappliedassentencesaresyntac/callyparsed,e.g.VP-->VNPVàserve<theme>{theme:food-type}
• Selec/onalrestric/onviola/on:noparse
![Page 68: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/68.jpg)
• Butthismeanswemust:• Writeselec/onalrestric/onsforeachsenseofeachpredicate–oruseFrameNet• Servealonehas15verbsenses
• Obtainhierarchicaltypeinforma/onabouteachargument(usingWordNet)• Howmanyhypernymsdoesdishhave?• Howmanywordsarehyponymsofdish?
• Butalso:• Some/messelec/onalrestric/onsdon’trestrictenough(Whichdishesdoyoulike?)• Some/mestheyrestricttoomuch(Eatdirt,worm!I’lleatmyhat!)
• Canwetakeasta/s/calapproach?
![Page 69: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/69.jpg)
Semi-supervisedBootstrapping• Whatifyoudon’thaveenoughdatatotrainasystem…• Bootstrap• Pickawordthatyouasananalystthinkwillco-occurwithyourtargetwordinpar/cularsense• Grepthroughyourcorpusforyourtargetwordandthehypothesizedword• Assumethatthetargettagistherightone
![Page 70: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/70.jpg)
Bootstrapping• Forbass• Assumeplayoccurswiththemusicsenseandfishoccurswiththefishsense
![Page 71: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/71.jpg)
Sentencesextractingusing“Jish”and“play”
![Page 72: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/72.jpg)
Wheredotheseedscomefrom?1) Handlabeling2) “Onesenseperdiscourse”:• Thesenseofawordishighlyconsistentwithina
document-Yarowsky(1995)• Truefortopicdependentwords• NotsotrueforotherPOSlikeadjec/vesand
verbs,e.g.make,take• Krovetz(1998)“Morethanonesenseper
discourse”arguesitisn’ttrueatallonceyoumovetofine-grainedsenses
3) Onesensepercolloca/on:• Awordreoccurringincolloca/onwiththesame
wordwillalmostsurelyhavethesamesense.Slide adapted from Chris Manning
![Page 73: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/73.jpg)
StagesintheYarowskybootstrappingalgorithm
![Page 74: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/74.jpg)
Problems• GiventhesegeneralMLapproaches,howmanyclassifiersdoIneedtoperformWSDrobustly• Oneforeachambiguouswordinthelanguage
• Howdoyoudecidewhatsetoftags/labels/sensestouseforagivenword?• Dependsontheapplica/on
![Page 75: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/75.jpg)
WordNetBass• Taggingwiththissetofsensesisanimpossiblyhardtaskthat’sprobablyoverkillforanyrealis/capplica/on
1. bass-(thelowestpartofthemusicalrange)2. bass,basspart-(thelowestpartinpolyphonicmusic)3. bass,basso-(anadultmalesingerwiththelowestvoice)4. seabass,bass-(fleshoflean-fleshedsaltwaterfishofthefamilySerranidae)5. freshwaterbass,bass-(anyofvariousNorthAmericanlean-fleshedfreshwaterfishesespeciallyofthegenus
Micropterus)6. bass,bassvoice,basso-(thelowestadultmalesingingvoice)7. bass-(thememberwiththelowestrangeofafamilyofmusicalinstruments)8. bass-(nontechnicalnameforanyofnumerousediblemarineandfreshwaterspiny-finnedfishes)
![Page 76: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/76.jpg)
SensevalHistory• ACL-SIGLEXworkshop(1997)• YarowskyandResnikpaper
• SENSEVAL-I(1998)• LexicalSampleforEnglish,French,andItalian
• SENSEVAL-II(Toulouse,2001)• LexicalSampleandAllWords• Organiza/on:Kilkgarriff(Brighton)
• SENSEVAL-III(2004)• SENSEVAL-IV->SEMEVAL(2007)• SEMEVAL(2010)• SEMEVAL2017:hdp://alt.qcri.org/semeval2017/index.php?id=tasks
SLIDE ADAPTED FROM CHRIS MANNING
![Page 77: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/77.jpg)
WSDPerformance• Varieswidelydependingonhowdifficultthedisambigua/ontaskis• Accuraciesofover90%arecommonlyreportedonsomeoftheclassic,ovenfairlyeasy,WSDtasks(pike,star,interest)• Sensevalbroughtcarefulevalua/onofdifficultWSD(manysenses,differentPOS)• Senseval1:morefinegrainedsenses,widerrangeoftypes:• Overall:about75%accuracy• Nouns:about80%accuracy• Verbs:about70%accuracy
![Page 78: Lexical Semantics and Word Sense Disambiguationkathy/NLP/2017/ClassSlides/Class12...Lexical Semantics and Word Sense Disambiguation Announcements • Midterm sample ques/ons on website](https://reader031.vdocuments.us/reader031/viewer/2022022508/5acc8ca17f8b9a27628c9522/html5/thumbnails/78.jpg)
Summary• LexicalSeman/cs• Homonymy,Polysemy,Synonymy• Thema/croles• Computa/onalresourceforlexicalseman/cs• WordNet• Task• Wordsensedisambigua/on• Next:seman/cparsing