![Page 1: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/1.jpg)
Deep Learning and Lexical, Syntacticand Semantic Analysis
Wanxiang Che (HIT)Yue Zhang (SUTD)
CCL 2016 Tutorial 12016-10-14
![Page 2: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/2.jpg)
Part 3: Greedy Decoding
CCL 2016 Tutorial 22016-10-14
![Page 3: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/3.jpg)
Part 3.1: Greedy Decoding for Tagging
CCL 2016 Tutorial 32016-10-14
![Page 4: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/4.jpg)
WindowApproach
• Tasks– POS tagging, Chunking, NER, SRL
• Tagonewordata time• Feedafixed-sizewindowoftextaroundeachwordtotag
• Features– Words, POS tags, Suffix, Cascading, …
RonanCollobert,JasonWeston,LéonBottou,MichaelKarlen,Koray Kavukcuoglu,andPavel Kuksa.2011.NaturalLanguageProcessing(Almost)fromScratch.J.Mach.Learn.Res.12,2493-2537.
CCL 2016 Tutorial 42016-10-14
![Page 5: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/5.jpg)
WindowApproach
• Worksfineformosttasks• Howtodealwithlong-rangedependencies?
– E.g.inSRL,theverbofinterestmightbeoutsidethewindow!
RonanCollobert,JasonWeston,LéonBottou,MichaelKarlen,Koray Kavukcuoglu,andPavel Kuksa.2011.NaturalLanguageProcessing(Almost)fromScratch.J.Mach.Learn.Res.12,2493-2537.CCL 2016 Tutorial 52016-10-14
![Page 6: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/6.jpg)
SentenceApproach
• Tagonewordata time– addextrarelative position features
• Feedthewholesentencetothenetwork• Convolutionstohandlevariable-lengthinputs• Maxovertimetocapturemostrelevantfeatures
– Outputsafixed-sizedfeaturevector
RonanCollobert,JasonWeston,LéonBottou,MichaelKarlen,Koray Kavukcuoglu,andPavel Kuksa.2011.NaturalLanguageProcessing(Almost)fromScratch.J.Mach.Learn.Res.12,2493-2537.CCL 2016 Tutorial 62016-10-14
![Page 7: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/7.jpg)
SentenceApproach
RonanCollobert,JasonWeston,LéonBottou,MichaelKarlen,Koray Kavukcuoglu,andPavel Kuksa.2011.NaturalLanguageProcessing(Almost)fromScratch.J.Mach.Learn.Res.12,2493-2537.CCL 2016 Tutorial 72016-10-14
![Page 8: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/8.jpg)
Results
• Window approach: POS, Chunking,NER• Sentence approach: SRL• WLL: Word-LevelLog-Likelihood
RonanCollobert,JasonWeston,LéonBottou,MichaelKarlen,Koray Kavukcuoglu,andPavel Kuksa.2011.NaturalLanguageProcessing(Almost)fromScratch.J.Mach.Learn.Res.12,2493-2537.CCL 2016 Tutorial 82016-10-14
![Page 9: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/9.jpg)
CCGSupertagging
Lewis, M., & Steedman, M. (2014). Improved CCG Parsing with Semi-supervised Supertagging. TACL.Xu, W., Auli, M., & Clark, S. (2015). CCG Supertagging with a Recurrent Neural Network. ACL.Lewis, M., Lee, Kenton., & Zettlemoyer, L. (2016). LSTM CCG Parsing. NAACL.
CCL 2016 Tutorial 92016-10-14
![Page 10: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/10.jpg)
CCGSupertagging
• NoPOSfeature
• Reducewordsparsity
• Globalcontextinformation
Lewis, M., & Steedman, M. (2014). Improved CCG Parsing with Semi-supervised Supertagging. TACL.Xu, W., Auli, M., & Clark, S. (2015). CCG Supertagging with a Recurrent Neural Network. ACL.Lewis, M., Lee, Kenton., & Zettlemoyer, L. (2016). LSTM CCG Parsing. NAACL.
CCL 2016 Tutorial 102016-10-14
![Page 11: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/11.jpg)
CCGSupertagging
Lewis, M., & Steedman, M. (2014). Improved CCG Parsing with Semi-supervised Supertagging. TACL.Xu, W., Auli, M., & Clark, S. (2015). CCG Supertagging with a Recurrent Neural Network. ACL.Lewis, M., Lee, Kenton., & Zettlemoyer, L. (2016). LSTM CCG Parsing. NAACL.
CCL 2016 Tutorial 112016-10-14
![Page 12: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/12.jpg)
CCGSupertagging
• Parsingresults
Lewis, M., & Steedman, M. (2014). Improved CCG Parsing with Semi-supervised Supertagging. TACL.Xu, W., Auli, M., & Clark, S. (2015). CCG Supertagging with a Recurrent Neural Network. ACL.Lewis, M., Lee, Kenton., & Zettlemoyer, L. (2016). LSTM CCG Parsing. NAACL.
CCL 2016 Tutorial 122016-10-14
![Page 13: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/13.jpg)
Part 3.2: Greedy Search for ConstituentParsingwithRNN
CCL 2016 Tutorial 132016-10-14
![Page 14: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/14.jpg)
Constituent Parsing with RecursiveNN
• Our goal
RichardSocher,CliffLin,AndrewY.Ng,andChristopherD.Manning.ParsingNaturalScenesandNaturalLanguagewithRecursiveNeuralNetworks.ICML2011CCL 2016 Tutorial 142016-10-14
![Page 15: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/15.jpg)
RecursiveNN
• Inputs– Two candidate children’s representations
• Outputs– The semantic representation if the two nodes are merged– Score of how plausible the new node would be
RichardSocher,CliffLin,AndrewY.Ng,andChristopherD.Manning.ParsingNaturalScenesandNaturalLanguagewithRecursiveNeuralNetworks.ICML2011CCL 2016 Tutorial 152016-10-14
![Page 16: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/16.jpg)
RNN Definition
!score = 𝑈*𝑝
𝑝 = tanh(𝑊𝑐3𝑐4
+ 𝑏)
where W at all nodes of the tree are thesame
RichardSocher,CliffLin,AndrewY.Ng,andChristopherD.Manning.ParsingNaturalScenesandNaturalLanguagewithRecursiveNeuralNetworks.ICML2011CCL 2016 Tutorial 162016-10-14
![Page 17: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/17.jpg)
Parsing a sentence with an RNN
CCL 2016 Tutorial 172016-10-14
![Page 18: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/18.jpg)
Parsing a sentence with an RNN
CCL 2016 Tutorial 182016-10-14
![Page 19: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/19.jpg)
Parsing a sentence with an RNN
CCL 2016 Tutorial 192016-10-14
![Page 20: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/20.jpg)
Parsing a sentence with an RNN
CCL 2016 Tutorial 202016-10-14
![Page 21: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/21.jpg)
Discussion on Simple RNN
• The composition function with single weight matrix isthe same for all categories, punctuation, etc.
• It could capture some phenomena but not adequate formore complex, higher order composition
CCL 2016 Tutorial 212016-10-14
![Page 22: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/22.jpg)
Solution: Syntactically-Untied RNN (SU-RNN)
• Intuition– Condition the composition function on the syntacticcategories
• Allows for different composition functions for pairs ofsyntactic categories, e.g. Adv + AdjP, VP + NP
RichardSocher,JohnBauer,ChristopherD.ManningandAndrewY.Ng.ParsingwithCompositionalVectorGrammars.ACL2013.CCL 2016 Tutorial 222016-10-14
![Page 23: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/23.jpg)
Compositional Vector Grammars (CVG)
• PCFG + SU-RNN• PCFG
– Produce: k-best parsing trees
• SU-RNN– Re-ranking with SU-RNN
RichardSocher,JohnBauer,ChristopherD.ManningandAndrewY.Ng.ParsingwithCompositionalVectorGrammars.ACL2013.CCL 2016 Tutorial 232016-10-14
![Page 24: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/24.jpg)
Part 3.3: Transition-basedDependencyParsingwithGreedy Search
CCL 2016 Tutorial 242016-10-14
![Page 25: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/25.jpg)
DependencyParsing
•NeuralMaltParser
Chen, D., & Manning, C. D. (2014). A Fast and Accurate Dependency Parser using Neural Network. ACL.CCL 2016 Tutorial 252016-10-14
![Page 26: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/26.jpg)
DependencyParsing
Chen, D., & Manning, C. D. (2014). A Fast and Accurate Dependency Parser using Neural Network. ACL.CCL 2016 Tutorial 262016-10-14
![Page 27: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/27.jpg)
DependencyParsing
•ZPar features(ZhangandNivre,ACL2011)
Chen, D., & Manning, C. D. (2014). A Fast and Accurate Dependency Parser using Neural Network. ACL.CCL 2016 Tutorial 272016-10-14
![Page 28: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/28.jpg)
DependencyParsing
Chen, D., & Manning, C. D. (2014). A Fast and Accurate Dependency Parser using Neural Network. ACL.CCL 2016 Tutorial 282016-10-14
![Page 29: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/29.jpg)
DependencyParsing
•ChenandManningwithcombinedfeatures
Zhang, M., & Zhang, Y. (2015). Combining Discrete and Continuous Features for Deterministic Transition-based Dependency Parsing. EMNLP. CCL 2016 Tutorial 292016-10-14
![Page 30: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/30.jpg)
DependencyParsing
•ChenandManningwithcombinedfeatures
Zhang, M., & Zhang, Y. (2015). Combining Discrete and Continuous Features for Deterministic Transition-based Dependency Parsing. EMNLP. CCL 2016 Tutorial 302016-10-14
![Page 31: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/31.jpg)
DependencyParsing
•ChenandManningwithricherfeatures
•11 more LSTMs
Keperwasser, E., & Goldberg, Y. (2016). Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations. TACL.
![Page 32: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/32.jpg)
DependencyParsing
Keperwasser, E., & Goldberg, Y. (2016). Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations. TACL. CCL 2016 Tutorial 322016-10-14
![Page 33: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/33.jpg)
DependencyParsing
•Chen andManningwithless features
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., & Smith, N. A. (2015). Transition-Based Dependency Parsing with Stack Long Short-Term Memory. ACL. CCL 2016 Tutorial 332016-10-14
![Page 34: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/34.jpg)
DependencyParsing
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., & Smith, N. A. (2015). Transition-Based Dependency Parsing with Stack Long Short-Term Memory. ACL. CCL 2016 Tutorial 342016-10-14
![Page 35: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/35.jpg)
DependencyParsing
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., & Smith, N. A. (2015). Transition-Based Dependency Parsing with Stack Long Short-Term Memory. ACL. CCL 2016 Tutorial 352016-10-14
![Page 36: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/36.jpg)
DependencyParsing
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., & Smith, N. A. (2015). Transition-Based Dependency Parsing with Stack Long Short-Term Memory. ACL. CCL 2016 Tutorial 362016-10-14
![Page 37: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/37.jpg)
DependencyParsing
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., & Smith, N. A. (2015). Transition-Based Dependency Parsing with Stack Long Short-Term Memory. ACL.
Chinese parsing results (CTB5)English parsing results (SD)
CCL 2016 Tutorial 372016-10-14
![Page 38: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/38.jpg)
DependencyParsing
•Dyeretal.withcharacterbasedwordvector
Ballesteros, M., Dyer, C., & Smith, N. A. (2015). Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs. EMNLP. CCL 2016 Tutorial 382016-10-14
![Page 39: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/39.jpg)
DependencyParsing
Ballesteros, M., Dyer, C., & Smith, N. A. (2015). Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs. EMNLP. CCL 2016 Tutorial 392016-10-14
![Page 40: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/40.jpg)
DependencyParsing
Ballesteros, M., Dyer, C., & Smith, N. A. (2015). Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs. EMNLP. CCL 2016 Tutorial 402016-10-14
![Page 41: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/41.jpg)
Dependency Parsing
Ballesteros, M., Dyer, C., & Smith, N. A. (2015). Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs. EMNLP. CCL 2016 Tutorial 412016-10-14
![Page 42: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/42.jpg)
Part 3.4: SequencetoSequencewithGreedy Search
CCL 2016 Tutorial 422016-10-14
![Page 43: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/43.jpg)
ConstituentParsing
•Sequence tosequence
Vinyals, O., Kaiser, L., Koo, T., Petrov, S., & Sutskever, I. (2015). Grammar as a Foreign Language. ICLR.CCL 2016 Tutorial 432016-10-14
![Page 44: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/44.jpg)
ConstituentParsing
Vinyals, O., Kaiser, L., Koo, T., Petrov, S., & Sutskever, I. (2015). Grammar as a Foreign Language. ICLR.CCL 2016 Tutorial 442016-10-14
![Page 45: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/45.jpg)
ConstituentParsing
Vinyals, O., Kaiser, L., Koo, T., Petrov, S., & Sutskever, I. (2015). Grammar as a Foreign Language. ICLR.CCL 2016 Tutorial 452016-10-14
![Page 46: Deep Learning and Lexical, Syntactic and Semantic Analysisir.hit.edu.cn/~car/talks/ccl16-tutorial-dl4nlp/part3.pdfRecursive NN • Inputs – Two candidate children’srepresentations](https://reader034.vdocuments.us/reader034/viewer/2022042922/5f6e472fcb9d9528bc0ed9b8/html5/thumbnails/46.jpg)
ConstituentParsing
Vinyals, O., Kaiser, L., Koo, T., Petrov, S., & Sutskever, I. (2015). Grammar as a Foreign Language. ICLR.CCL 2016 Tutorial 462016-10-14