machinelearningfor language(technology( lecture11:! logisc...
TRANSCRIPT
![Page 1: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/1.jpg)
Machine Learning for Language Technology Lecture 11: Logis.c Regression
Marina San.ni Department of Linguis.cs and Philology Uppsala University, Uppsala, Sweden
Autumn 2014
Acknowledgement: Thanks to Prof. Joakim Nivre for course design and materials
1
![Page 2: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/2.jpg)
![Page 3: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/3.jpg)
![Page 4: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/4.jpg)
![Page 5: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/5.jpg)
![Page 6: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/6.jpg)
![Page 7: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/7.jpg)
![Page 8: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/8.jpg)
![Page 9: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/9.jpg)
![Page 10: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/10.jpg)
![Page 11: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/11.jpg)
![Page 12: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/12.jpg)
![Page 13: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/13.jpg)
![Page 14: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/14.jpg)
![Page 15: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/15.jpg)
”Our Linear” Classifiers and their Induc.ve biases (or… how to find the weights)
• Perceptron (online): minimizes error in the training set
• SVMs (batch): minimizes error in the training set and maximizes margin
• MIRA (online): minimizes error in the training set and maximizes margin
• Logis.c Regression (batch): maximizes the likelihood of the training data
![Page 16: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/16.jpg)
Appendix: Gradient Ascent
![Page 17: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/17.jpg)
![Page 18: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/18.jpg)
![Page 19: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/19.jpg)
![Page 20: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/20.jpg)
![Page 21: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/21.jpg)
![Page 22: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/22.jpg)
![Page 23: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/23.jpg)
![Page 24: MachineLearningfor Language(Technology( Lecture11:! Logisc ...santini.se/teaching/ml/2014/Lecture11_logistic_regression.pdf · Aside: Logistic Regression = Entropy Well known equivalence](https://reader035.vdocuments.us/reader035/viewer/2022071603/613e5ed459df642846167cc4/html5/thumbnails/24.jpg)
The end