basic parsing with context-free grammarskathy/nlp/2017/classslides/...basic parsing with...
TRANSCRIPT
![Page 1: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/1.jpg)
BasicParsingwithContext-FreeGrammars
1 Some slides adapted from Julia Hirschberg and Dan Jurafsky
Announcementsbull HW2togoouttodayNextTuesdaymostimportantforbackgroundtoassignment
bull Signupforpolleverywhere
bull Todaywrap-upfromlastclassandstartonparsing
2
Wrap-uponsyntax
3
GrammarEquivalencebull Canhavedifferentgrammarsthatgeneratesamesetofstrings(weakequivalence)bull Grammar1NPrarrDetPNandDetPrarra|thebull Grammar2NPrarraN|NPrarrtheN
bull CanhavedifferentgrammarsthathavesamesetofderivaLontrees(strongequivalence)bull WithCFGspossibleonlywithuselessrulesbull Grammar2NPrarraN|NPrarrtheNbull Grammar3NPrarraN|NPrarrtheNDetPrarrmany
bull Strongequivalenceimpliesweakequivalence
NormalFormsampcbull Thereareweaklyequivalentnormalforms(ChomskyNormalFormGreibachNormalForm)
bull TherearewaystoeliminateuselessproducLonsandsoon
ChomskyNormalFormACFGisinChomskyNormalForm(CNF)ifallproducLonsareofoneoftwoforms
bull ArarrBCwithABCnonterminalsbull ArarrawithAanonterminalandaaterminal
EveryCFGhasaweaklyequivalentCFGinCNF
NobodyUsesSimpleCFGs(ExceptIntroNLPCourses)
bull AllmajorsyntacLctheories(ChomskyLFGHPSGTAG-basedtheories)representbothphrasestructureanddependencyinonewayoranother
bull AllsuccessfulparserscurrentlyusestaLsLcsaboutphrasestructureandaboutdependency
bull DerivedependencythroughldquoheadpercolaLonrdquoforeachrulesaywhichdaughterishead
MassiveAmbiguityofSyntaxbull Forastandardsentenceandagrammarwithwidecoveragethereare1000sofderivaLons
bull Examplebull ThelargeportraitpaintertoldthedelegaLonthathesentmoneyordersinaleYeronWednesday
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 2: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/2.jpg)
Announcementsbull HW2togoouttodayNextTuesdaymostimportantforbackgroundtoassignment
bull Signupforpolleverywhere
bull Todaywrap-upfromlastclassandstartonparsing
2
Wrap-uponsyntax
3
GrammarEquivalencebull Canhavedifferentgrammarsthatgeneratesamesetofstrings(weakequivalence)bull Grammar1NPrarrDetPNandDetPrarra|thebull Grammar2NPrarraN|NPrarrtheN
bull CanhavedifferentgrammarsthathavesamesetofderivaLontrees(strongequivalence)bull WithCFGspossibleonlywithuselessrulesbull Grammar2NPrarraN|NPrarrtheNbull Grammar3NPrarraN|NPrarrtheNDetPrarrmany
bull Strongequivalenceimpliesweakequivalence
NormalFormsampcbull Thereareweaklyequivalentnormalforms(ChomskyNormalFormGreibachNormalForm)
bull TherearewaystoeliminateuselessproducLonsandsoon
ChomskyNormalFormACFGisinChomskyNormalForm(CNF)ifallproducLonsareofoneoftwoforms
bull ArarrBCwithABCnonterminalsbull ArarrawithAanonterminalandaaterminal
EveryCFGhasaweaklyequivalentCFGinCNF
NobodyUsesSimpleCFGs(ExceptIntroNLPCourses)
bull AllmajorsyntacLctheories(ChomskyLFGHPSGTAG-basedtheories)representbothphrasestructureanddependencyinonewayoranother
bull AllsuccessfulparserscurrentlyusestaLsLcsaboutphrasestructureandaboutdependency
bull DerivedependencythroughldquoheadpercolaLonrdquoforeachrulesaywhichdaughterishead
MassiveAmbiguityofSyntaxbull Forastandardsentenceandagrammarwithwidecoveragethereare1000sofderivaLons
bull Examplebull ThelargeportraitpaintertoldthedelegaLonthathesentmoneyordersinaleYeronWednesday
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 3: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/3.jpg)
Wrap-uponsyntax
3
GrammarEquivalencebull Canhavedifferentgrammarsthatgeneratesamesetofstrings(weakequivalence)bull Grammar1NPrarrDetPNandDetPrarra|thebull Grammar2NPrarraN|NPrarrtheN
bull CanhavedifferentgrammarsthathavesamesetofderivaLontrees(strongequivalence)bull WithCFGspossibleonlywithuselessrulesbull Grammar2NPrarraN|NPrarrtheNbull Grammar3NPrarraN|NPrarrtheNDetPrarrmany
bull Strongequivalenceimpliesweakequivalence
NormalFormsampcbull Thereareweaklyequivalentnormalforms(ChomskyNormalFormGreibachNormalForm)
bull TherearewaystoeliminateuselessproducLonsandsoon
ChomskyNormalFormACFGisinChomskyNormalForm(CNF)ifallproducLonsareofoneoftwoforms
bull ArarrBCwithABCnonterminalsbull ArarrawithAanonterminalandaaterminal
EveryCFGhasaweaklyequivalentCFGinCNF
NobodyUsesSimpleCFGs(ExceptIntroNLPCourses)
bull AllmajorsyntacLctheories(ChomskyLFGHPSGTAG-basedtheories)representbothphrasestructureanddependencyinonewayoranother
bull AllsuccessfulparserscurrentlyusestaLsLcsaboutphrasestructureandaboutdependency
bull DerivedependencythroughldquoheadpercolaLonrdquoforeachrulesaywhichdaughterishead
MassiveAmbiguityofSyntaxbull Forastandardsentenceandagrammarwithwidecoveragethereare1000sofderivaLons
bull Examplebull ThelargeportraitpaintertoldthedelegaLonthathesentmoneyordersinaleYeronWednesday
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 4: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/4.jpg)
GrammarEquivalencebull Canhavedifferentgrammarsthatgeneratesamesetofstrings(weakequivalence)bull Grammar1NPrarrDetPNandDetPrarra|thebull Grammar2NPrarraN|NPrarrtheN
bull CanhavedifferentgrammarsthathavesamesetofderivaLontrees(strongequivalence)bull WithCFGspossibleonlywithuselessrulesbull Grammar2NPrarraN|NPrarrtheNbull Grammar3NPrarraN|NPrarrtheNDetPrarrmany
bull Strongequivalenceimpliesweakequivalence
NormalFormsampcbull Thereareweaklyequivalentnormalforms(ChomskyNormalFormGreibachNormalForm)
bull TherearewaystoeliminateuselessproducLonsandsoon
ChomskyNormalFormACFGisinChomskyNormalForm(CNF)ifallproducLonsareofoneoftwoforms
bull ArarrBCwithABCnonterminalsbull ArarrawithAanonterminalandaaterminal
EveryCFGhasaweaklyequivalentCFGinCNF
NobodyUsesSimpleCFGs(ExceptIntroNLPCourses)
bull AllmajorsyntacLctheories(ChomskyLFGHPSGTAG-basedtheories)representbothphrasestructureanddependencyinonewayoranother
bull AllsuccessfulparserscurrentlyusestaLsLcsaboutphrasestructureandaboutdependency
bull DerivedependencythroughldquoheadpercolaLonrdquoforeachrulesaywhichdaughterishead
MassiveAmbiguityofSyntaxbull Forastandardsentenceandagrammarwithwidecoveragethereare1000sofderivaLons
bull Examplebull ThelargeportraitpaintertoldthedelegaLonthathesentmoneyordersinaleYeronWednesday
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 5: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/5.jpg)
NormalFormsampcbull Thereareweaklyequivalentnormalforms(ChomskyNormalFormGreibachNormalForm)
bull TherearewaystoeliminateuselessproducLonsandsoon
ChomskyNormalFormACFGisinChomskyNormalForm(CNF)ifallproducLonsareofoneoftwoforms
bull ArarrBCwithABCnonterminalsbull ArarrawithAanonterminalandaaterminal
EveryCFGhasaweaklyequivalentCFGinCNF
NobodyUsesSimpleCFGs(ExceptIntroNLPCourses)
bull AllmajorsyntacLctheories(ChomskyLFGHPSGTAG-basedtheories)representbothphrasestructureanddependencyinonewayoranother
bull AllsuccessfulparserscurrentlyusestaLsLcsaboutphrasestructureandaboutdependency
bull DerivedependencythroughldquoheadpercolaLonrdquoforeachrulesaywhichdaughterishead
MassiveAmbiguityofSyntaxbull Forastandardsentenceandagrammarwithwidecoveragethereare1000sofderivaLons
bull Examplebull ThelargeportraitpaintertoldthedelegaLonthathesentmoneyordersinaleYeronWednesday
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 6: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/6.jpg)
ChomskyNormalFormACFGisinChomskyNormalForm(CNF)ifallproducLonsareofoneoftwoforms
bull ArarrBCwithABCnonterminalsbull ArarrawithAanonterminalandaaterminal
EveryCFGhasaweaklyequivalentCFGinCNF
NobodyUsesSimpleCFGs(ExceptIntroNLPCourses)
bull AllmajorsyntacLctheories(ChomskyLFGHPSGTAG-basedtheories)representbothphrasestructureanddependencyinonewayoranother
bull AllsuccessfulparserscurrentlyusestaLsLcsaboutphrasestructureandaboutdependency
bull DerivedependencythroughldquoheadpercolaLonrdquoforeachrulesaywhichdaughterishead
MassiveAmbiguityofSyntaxbull Forastandardsentenceandagrammarwithwidecoveragethereare1000sofderivaLons
bull Examplebull ThelargeportraitpaintertoldthedelegaLonthathesentmoneyordersinaleYeronWednesday
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 7: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/7.jpg)
NobodyUsesSimpleCFGs(ExceptIntroNLPCourses)
bull AllmajorsyntacLctheories(ChomskyLFGHPSGTAG-basedtheories)representbothphrasestructureanddependencyinonewayoranother
bull AllsuccessfulparserscurrentlyusestaLsLcsaboutphrasestructureandaboutdependency
bull DerivedependencythroughldquoheadpercolaLonrdquoforeachrulesaywhichdaughterishead
MassiveAmbiguityofSyntaxbull Forastandardsentenceandagrammarwithwidecoveragethereare1000sofderivaLons
bull Examplebull ThelargeportraitpaintertoldthedelegaLonthathesentmoneyordersinaleYeronWednesday
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 8: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/8.jpg)
MassiveAmbiguityofSyntaxbull Forastandardsentenceandagrammarwithwidecoveragethereare1000sofderivaLons
bull Examplebull ThelargeportraitpaintertoldthedelegaLonthathesentmoneyordersinaleYeronWednesday
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 9: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/9.jpg)
9
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 10: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/10.jpg)
10
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 11: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/11.jpg)
PennTreebank(PTB)bull SyntacLcallyannotatedcorpusofnewspapertexts(phrasestructure)
bull ThenewspapertextsarenaturallyoccurringdatabutthePTBisnot
bull PTBannotaLonrepresentsaparLcularlinguisLctheory(butafairlyldquovanillardquoone)
bull ParLculariLesbull VeryindirectrepresentaLonofgrammaLcalrelaLons(needforheadpercolaLontables)
bull CompletelyflatstructureinNP(brownbaglunchpink-and-yellowchildseat)
bull HasflatSsflatVPs
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 12: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/12.jpg)
ExamplefromPTB((S(NP-SBJIt)(VPs(NP-PRD(NP(NPthelatestinvestmentcraze) (VPsweeping (NPWallStreet))) (NP(NParash) (PPof (NP(NPnewclosed-endcountryfunds) (NP(NPthose (ADJPpubliclytraded) poraolios) (SBAR(WHNP-37that) (S(NP-SBJT-37) (VPinvest (PP-CLRin (NP(NPstocks) (PPof (NPasingleforeigncountry)))))))))))
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 13: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/13.jpg)
SyntacticParsing
13
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 14: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/14.jpg)
SyntacticParsingbull DeclaraLveformalismslikeCFGsFSAsdefinethelegalstringsofalanguage--butonlytellyoulsquothisisalegalstringofthelanguageXrsquo
bull Parsingalgorithmsspecifyhowtorecognizethestringsofalanguageandassigneachstringone(ormore)syntacLcanalyses
14
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 15: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/15.jpg)
CFGExamplebull ManypossibleCFGsforEnglishhereisanexample(fragment)
bull SrarrNPVPbull VPrarrVNPbull NPrarrDetN|AdjNPbull Nrarrboy|girlbull Vrarrsees|likesbull Adjrarrbig|smallbull DetPrarra|the
bull bigthesmallgirlseesaboybull Johnlikesagirlbull Ilikeagirlbull Isleepbull Theolddogthefootstepsoftheyoung
the small boy likes a girl
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 16: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/16.jpg)
ModiMiedCFGS agrave NP VP VP agrave V S agrave Aux NP VP VP -gt V PP S -gt VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
NP -gt Pronoun Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 17: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/17.jpg)
Parse Tree for lsquoThe old dog the footsteps of the youngrsquo for Prior CFG
S
NP VP
NP V
DET NOM
N PP
DET NOM
N
The old dog the
footsteps of the young
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 18: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/18.jpg)
ParsingasaFormofSearchbull SearchingFSAs
bull Findingtherightpaththroughtheautomatonbull SearchspacedefinedbystructureofFSA
bull SearchingCFGsbull Findingtherightparsetreeamongallpossibleparsetrees
bull Searchspacedefinedbythegrammar
bull Constraintsprovidedbytheinputsentenceandtheautomatonorgrammar 18
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 19: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/19.jpg)
Top-DownParserbull BuildsfromtherootSnodetotheleavesbull ExpectaLon-basedbull Commonsearchstrategy
bull Top-downlei-to-rightbacktrackingbull TryfirstrulewithLHS=Sbull NextexpandallconsLtuentsinthesetreesrulesbull ConLnueunLlleavesarePOSbull BacktrackwhencandidatePOSdoesnotmatchinputstring 19
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 20: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/20.jpg)
RuleExpansionbull ldquoTheolddogthefootstepsoftheyoungrdquo
bull Wheredoesbacktrackinghappen
bull WhatarethecomputaLonaldisadvantages
bull Whataretheadvantages
20
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 21: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/21.jpg)
21
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 22: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/22.jpg)
Bottom-UpParsingbull ParserbeginswithwordsofinputandbuildsuptreesapplyinggrammarruleswhoseRHSmatches DetNVDetNPrepDetN TheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN TheolddogthefootstepsoftheyoungParseconLnuesunLlanSrootnodereachedornofurthernodeexpansionpossible
22
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 23: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/23.jpg)
DetNVDetNPrepDetNTheolddogthefootstepsoftheyoungDetAdjNDetNPrepDetN
23
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 24: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/24.jpg)
Bottom-upparsingbull WhendoesdisambiguaLonoccur
bull WhatarethecomputaLonaladvantagesanddisadvantages
24
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 25: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/25.jpg)
25
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 26: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/26.jpg)
Whatrsquosrightwrongwithhellip
bull Top-Downparsersndashtheyneverexploreillegalparses(egwhichcanrsquotformanS)--butwasteLmeontreesthatcannevermatchtheinput
bull BoYom-Upparsersndashtheyneverexploretreesinconsistentwithinput--butwasteLmeexploringillegalparses(withnoSroot)
bull Forbothfindacontrolstrategy--howexploresearchspaceefficientlybull Pursuingallparsesinparallelorbacktrackorhellipbull Whichruletoapplynextbull Whichnodetoexpandnext 26
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 27: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/27.jpg)
SomeSolutionsDynamicProgrammingApproachesndashUseacharttorepresentparltalresults
bull CKYParsingAlgorithmbull BoYom-upbull GrammarmustbeinNormalFormbull TheparsetreemightnotbeconsistentwithlinguisLctheory
bull EarlyParsingAlgorithmbull Top-downbull ExpectaLonsaboutconsLtuentsareconfirmedbyinputbull APOStagforawordthatisnotpredictedisneveradded
bull ChartParser 27
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 28: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/28.jpg)
EarleyParsingbull AllowsarbitraryCFGsbull Fillsatableinasinglesweepovertheinputwordsbull TableislengthN+1Nisnumberofwordsbull Tableentriesrepresent
bull CompletedconsLtuentsandtheirlocaLonsbull In-progressconsLtuentsbull PredictedconsLtuents 28
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 29: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/29.jpg)
Statesbull Thetable-entriesarecalledstatesandarerepresentedwithdoYed-rulesS-gtVP AVPispredicted
NP-gtDetNominal AnNPisinprogress
VP-gtVNP AVPhasbeenfound
29
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 30: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/30.jpg)
StatesLocationsbull ItwouldbenicetoknowwherethesethingsareintheinputsohellipS-gtVP[00] AVPispredictedatthe
startofthesentence
NP-gtDetNominal[12] AnNPisinprogressthe Detgoesfrom1to2
VP-gtVNP [03] AVPhasbeenfound starLngat0andendingat3
30
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 31: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/31.jpg)
Graphically
31
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 32: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/32.jpg)
Earleybull Aswithmostdynamicprogrammingapproachestheanswerisfoundbylookinginthetableintherightplace
bull InthiscasethereshouldbeanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
32
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 33: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/33.jpg)
EarleyAlgorithmbull Marchthroughchartlei-to-rightbull Ateachstepapply1of3operators
bull Predictorbull CreatenewstatesrepresenLngtop-downexpectaLons
bull Scannerbull MatchwordpredicLons(rulewithwordaierdot)towords
bull Completerbull WhenastateiscompleteseewhatruleswerelookingforthatcompletedconsLtuent
33
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 34: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/34.jpg)
Predictorbull Givenastate
bull Withanon-terminaltorightofdot(notapart-of-speechcategory)
bull Createanewstateforeachexpansionofthenon-terminalbull PlacethesenewstatesintosamechartentryasgeneratedstatebeginningandendingwheregeneraLngstateends
bull Sopredictorlookingatbull S-gtVP[00]
bull resultsinbull VP-gtVerb[00]bull VP-gtVerbNP[00]
34
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 35: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/35.jpg)
Scannerbull Givenastate
bull Withanon-terminaltorightofdotthatisapart-of-speechcategory
bull IfthenextwordintheinputmatchesthisPOSbull Createanewstatewithdotmovedoverthenon-terminalbull SoscannerlookingatVP-gtVerbNP[00]bull Ifthenextwordldquobookrdquocanbeaverbaddnewstate
bull VP-gtVerbNP[01]bull Addthisstatetochartentryfollowingcurrentonebull NoteEarleyalgorithmusestop-downinputtodisambiguatePOSOnlyPOSpredictedbysomestatecangetaddedtochart
35
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 36: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/36.jpg)
Completerbull Appliedtoastatewhenitsdothasreachedrightendofrole
bull Parserhasdiscoveredacategoryoversomespanofinput
bull Findandadvanceallpreviousstatesthatwerelookingforthiscategorybull copystatemovedotinsertincurrentchartentry
bull Givenbull NP-gtDetNominal[13]bull VP-gtVerbNP[01]
bull Addbull VP-gtVerbNP[03] 36
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 37: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/37.jpg)
Howdoweknowwearedonebull FindanSstateinthefinalcolumnthatspansfrom0ton+1andiscomplete
bull Ifthatrsquosthecaseyoursquoredonebull Sndashgtα[0n+1]
37
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 38: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/38.jpg)
Earleybull Morespecificallyhellip
1 Predictallthestatesyoucanupfront
2 Readaword
1 Extendstatesbasedonmatches2 AddnewpredicLons3 Goto2
3 LookatN+1toseeifyouhaveawinner 38
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 39: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/39.jpg)
Examplebull Bookthatflightbull WeshouldfindhellipanSfrom0to3thatisacompletedstatehellip
39
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 40: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/40.jpg)
CFGforFragmentofEnglishS agrave NP VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN V agrave dog | include | prefer | book
Nom -gt Adj Nom Aux agrave does Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 41: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/41.jpg)
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 42: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/42.jpg)
S agrave NP VP S -gt VP VP agrave V S agrave Aux NP VP PP -gt Prep NP NP agrave Det Nom
N agrave old | dog | footsteps | young | flight
NP agravePropN NP -gt Pro
V agrave dog | include | prefer | book Aux agrave does
Nom agrave N Prep agravefrom | to | on | of Nom agrave N Nom PropN agrave Bush | McCain |
Obama Nom agrave Nom PP Det agrave that | this | a| the VP agrave V NP VP -gt V NP PP VP -gt V PP VP -gt VP PP
Adj -gt old | green | red
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 43: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/43.jpg)
Example
43
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 44: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/44.jpg)
Example
44
Completer
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 45: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/45.jpg)
Example
45
Completer
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 46: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/46.jpg)
Example
46
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 47: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/47.jpg)
Detailsbull Whatkindofalgorithmsdidwejustdescribebull Notparsersndashrecognizers
bull ThepresenceofanSstatewiththerightaYributesintherightplaceindicatesasuccessfulrecogniLon
bull Butnoparsetreehellipnoparserbull Thatrsquoshowwesolve(not)anexponenLalprobleminpolynomialLme
47
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 48: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/48.jpg)
ConvertingEarleyfromRecognizertoParserbull WiththeaddiLonofafewpointerswehaveaparser
bull AugmenttheldquoCompleterrdquotopointtowherewecamefrom
48
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 49: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/49.jpg)
Augmentingthechartwithstructuralinformation
S8 S9
S10
S11
S13 S12
S8
S9 S8
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 50: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/50.jpg)
RetrievingParseTreesfromChartbull Allthepossibleparsesforaninputareinthetable
bull WejustneedtoreadoffallthebackpointersfromeverycompleteSinthelastcolumnofthetable
bull FindalltheS-gtX[0N+1]
bull FollowthestructuraltracesfromtheCompleter
bull OfcoursethiswonrsquotbepolynomialLmesincetherecouldbeanexponenLalnumberoftrees
bull Wecanatleastrepresentambiguityefficiently 50
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 51: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/51.jpg)
LeftRecursionvsRightRecursion
bull Depth-firstsearchwillneverterminateifgrammarisle9recursive(egNP--gtNPPP)
51
)( εαα ⎯rarr⎯ΑΒ⎯rarr⎯Α
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 52: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/52.jpg)
bull SoluLonsbull Rewritethegrammar(automaLcally)toaweaklyequivalentonewhichisnotlei-recursiveegThemanonthehillwiththetelescopehellipNPagraveNPPP(wantedNomplusasequenceofPPs)NPagraveNomPPNPagraveNomNomagraveDetNhellipbecomeshellipNPagraveNomNPrsquoNomagraveDetNNPrsquoagravePPNPrsquo(wantedasequenceofPPs)NPrsquoagraveebull Notsoobviouswhattheserulesmeanhellip
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 53: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/53.jpg)
bull Hardertodetectandeliminatenon-immediatele9recursion
bull NP--gtNomPPbull Nom--gtNP
bull Fixdepthofsearchexplicitly
bull Ruleorderingnon-recursiverulesfirstbull NP--gtDetNombull NP--gtNPPP
53
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 54: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/54.jpg)
AnotherProblemStructuralambiguity
bull MulLplelegalstructuresbull AYachment(egIsawamanonahillwithatelescope)
bull CoordinaLon(egyoungercatsanddogs)bull NPbrackeLng(egSpanishlanguageteachers)
54
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 55: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/55.jpg)
55
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 56: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/56.jpg)
56
NP vs VP Attachment
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 57: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/57.jpg)
bull SoluLonbull Returnallpossibleparsesanddisambiguateusingldquoothermethodsrdquo
57
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58
![Page 58: Basic Parsing with Context-Free Grammarskathy/NLP/2017/ClassSlides/...Basic Parsing with Context-Free Grammars Some slides adapted from Julia Hirschberg and Dan Jurafsky 1 . ... Normal](https://reader030.vdocuments.us/reader030/viewer/2022040108/5f058a007e708231d41376cf/html5/thumbnails/58.jpg)
SummingUpbull Parsingisasearchproblemwhichmaybeimplementedwithmanycontrolstrategiesbull Top-DownorBoYom-Upapproacheseachhaveproblemsbull Combiningthetwosolvessomebutnotallissues
bull Leirecursionbull SyntacLcambiguity
58