e9 205 machine learning for signal...
TRANSCRIPT
![Page 1: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/1.jpg)
E9 205 Machine Learning for Signal Processing
11-10-2017Linear Models for Regression and
Classification
![Page 2: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/2.jpg)
Linear Regression
Bishop - PRML book (Chap 3)
❖ Solution to Maximum Likelihood problem is the least squares solution
Pseudo Inverse Based Solution
![Page 3: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/3.jpg)
Underfit
(Courtesy - Dr D. Vijayasenan, NITK)
![Page 4: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/4.jpg)
Overfit
(Courtesy - Dr D. Vijayasenan, NITK)
![Page 5: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/5.jpg)
Compromise
(Courtesy - Dr D. Vijayasenan, NITK)
![Page 6: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/6.jpg)
Regularized Least Squares
Bishop - PRML book (Chap 3)
❖ Optimize a modified cost function
![Page 7: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/7.jpg)
Regularized Least Squares
Bishop - PRML book (Chap 3)
![Page 8: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/8.jpg)
Linear Models for Classification
Bishop - PRML book (Chap 3)
❖ Optimize a modified cost function
![Page 9: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/9.jpg)
Least Squares for Classification
Bishop - PRML book (Chap 3)
❖ K-class classification problem
❖ With 1-of-K hot encoding, and least squares regression
![Page 10: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/10.jpg)
Logistic Regression
Bishop - PRML book (Chap 3)
❖ 2- class logistic regression
❖ K-class logistic regression
❖ Maximum likelihood solution
❖ Maximum likelihood solution
![Page 11: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/11.jpg)
Least Squares versus Logistic Regression
Bishop - PRML book (Chap 4)
![Page 12: E9 205 Machine Learning for Signal Processingleap.ee.iisc.ac.in/sriram/teaching/MLSP_17/slides/... · E9 205 Machine Learning for Signal Processing 11-10-2017 Linear Models for Regression](https://reader033.vdocuments.us/reader033/viewer/2022053005/5f08f4eb7e708231d4248aa7/html5/thumbnails/12.jpg)
Least Squares versus Logistic Regression
Bishop - PRML book (Chap 4)