introduction mohammad beigi department of biomedical engineering isfahan university...
TRANSCRIPT
![Page 2: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/2.jpg)
Pattern recognition and Machine Learning
Syllabus Introduction,Linear Models for classificationNeural Networks (MLP, RBF, SOM, LVQ, ADALINE) Kernel Methods & Support Vector Machines Statistical Pattern Recognition ? (HMM,EM, Clustering and unsupervised learning ? Feature Selection and Dimension reduction ?
![Page 3: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/3.jpg)
Pattern recognition and Machine Learning
TextsR. O. Duda, P. E. Hart, D. G. Stork,
Pattern Classification, 2nd edition, John Wiley & Sons, Inc., 2000.
M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
![Page 4: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/4.jpg)
• Midterm 25%• Final 40%• Computer assignments 10%• Final Programming Project 15%• Seminar 10%
Evaluation
![Page 5: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/5.jpg)
Human Perception
Humans have developed highly sophisticated skills for sensing their environment and taking actions according to what they observe, e.g
Understanding spoken wordsreading handwritingdistinguishing fresh food from its smell
We would like to give similar capabilities to machines
![Page 6: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/6.jpg)
What is Pattern Recognition? A pattern is an entity, vaguely defined, that could be given a name, e.g.,
fingerprint image, handwritten word, human face, speech signal, DNA sequence,
Pattern recognition is the study of how machines can observe the environment, learn to distinguish patterns of interest, make sound and reasonable decisions about the categories of the
patterns.
![Page 7: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/7.jpg)
Human and Machine Perception
We are often influenced by the knowledge of how patterns are modeled and recognized in nature when we develop pattern recognition algorithms. Research on machine perception also helps us gain deeper understanding and appreciation for pattern recognition systems in nature. Yet, we also apply many techniques that are purelynumerical and do not have any correspondence in naturalsystems.
![Page 8: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/8.jpg)
![Page 9: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/9.jpg)
Pattern Recognition Applications
![Page 10: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/10.jpg)
Pattern Recognition Applications
![Page 11: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/11.jpg)
Pattern Recognition Applications
![Page 12: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/12.jpg)
Pattern Recognition Applications
![Page 13: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/13.jpg)
Pattern Recognition Applications
![Page 14: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/14.jpg)
Pattern Recognition Applications
![Page 15: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/15.jpg)
Pattern Recognition Applications
![Page 16: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/16.jpg)
Pattern Recognition Applications
![Page 17: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/17.jpg)
Pattern Recognition Applications
Figure 9: Clustering of Microarray Data
![Page 18: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/18.jpg)
Pattern Recognition Applications
Figure 10: Brain Control Interface
![Page 19: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/19.jpg)
![Page 20: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/20.jpg)
![Page 21: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/21.jpg)
![Page 22: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/22.jpg)
![Page 23: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/23.jpg)
![Page 24: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/24.jpg)
![Page 25: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/25.jpg)
![Page 26: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/26.jpg)
![Page 27: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/27.jpg)
![Page 28: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/28.jpg)
![Page 29: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/29.jpg)
![Page 30: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/30.jpg)
![Page 31: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/31.jpg)
![Page 32: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/32.jpg)
![Page 33: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/33.jpg)
![Page 34: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/34.jpg)
![Page 35: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/35.jpg)
Regression: Polynomial Curve Fitting
t is continuous
![Page 36: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/36.jpg)
Sum-of-Squares Error Function
* min ( )w Arg E w Optimization Problem
![Page 37: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/37.jpg)
0th Order Polynomial
![Page 38: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/38.jpg)
1st Order Polynomial
![Page 39: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/39.jpg)
3rd Order Polynomial
![Page 40: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/40.jpg)
9th Order Polynomial
![Page 41: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/41.jpg)
Over-fitting
Root-Mean-Square (RMS) Error:
![Page 42: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/42.jpg)
Polynomial Coefficients
![Page 43: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/43.jpg)
Data Set Size: 9th Order Polynomial
![Page 44: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/44.jpg)
Data Set Size: 9th Order Polynomial
![Page 45: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/45.jpg)
Regularization ;ridge regression
Penalize large coefficient values
Shrinkage: reduce the order of method
~* min ( )w Arg E w
![Page 46: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/46.jpg)
Regularization:
![Page 47: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/47.jpg)
Regularization:
![Page 48: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/48.jpg)
Regularization: vs.
![Page 49: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/49.jpg)
Polynomial Coefficients
Optimization Problem: Finding optimum ,M
![Page 50: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/50.jpg)
Classification example: Handwritten Digit Recognition
28*28 Pixel image : 784 real numbers, training set: 1{ ,.... }Nx x1x
( ),y x t {1,..,9}t
![Page 51: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/51.jpg)
![Page 52: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/52.jpg)
![Page 53: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/53.jpg)
![Page 54: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/54.jpg)
Pattern recognition approaches
![Page 55: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/55.jpg)
Statistical Pattern recognition
![Page 56: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/56.jpg)
Statistical Pattern recognition
![Page 57: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/57.jpg)
Structural Pattern Recognition
![Page 58: Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University Majid.beigi@eng.ui.ac.ir](https://reader033.vdocuments.us/reader033/viewer/2022051214/56649e1b5503460f94b098f2/html5/thumbnails/58.jpg)
Neural Pattern Recognition