pattern classificationdpinto/pln/autumn2010/dhsch1.pdf · pattern classification, chapter 1 6 •...
TRANSCRIPT
![Page 1: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/1.jpg)
Pattern Classification
All materials in these slides were taken fromPattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley & Sons, 2000
with the permission of the authors and the publisher
![Page 2: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/2.jpg)
Chapter 1: Introduction to Pattern Recognition (Sections 1.1-1.6)
• Machine Perception• An Example• Pattern Recognition Systems• The Design Cycle• Learning and Adaptation• Conclusion
![Page 3: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/3.jpg)
Pattern Classification, Chapter 1
3
Machine Perception
• Build a machine that can recognize patterns:
• Speech recognition
• Fingerprint identification
• OCR (Optical Character Recognition)
• DNA sequence identification
![Page 4: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/4.jpg)
Pattern Classification, Chapter 1
4
An Example
• “Sorting incoming Fish on a conveyor according to species using optical sensing”
Sea bassSpecies
Salmon
![Page 5: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/5.jpg)
Pattern Classification, Chapter 1
5
• Problem Analysis
• Set up a camera and take some sample images to extract features
• Length• Lightness• Width• Number and shape of fins• Position of the mouth, etc…
• This is the set of all suggested features to explore for use in our classifier!
![Page 6: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/6.jpg)
Pattern Classification, Chapter 1
6
• Preprocessing
• Use a segmentation operation to isolate fishes from one another and from the background
• Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features
• The features are passed to a classifier
![Page 7: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/7.jpg)
Pattern Classification, Chapter 1
7
![Page 8: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/8.jpg)
Pattern Classification, Chapter 1
8
• Classification
• Select the length of the fish as a possible feature for discrimination
![Page 9: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/9.jpg)
Pattern Classification, Chapter 1
9
![Page 10: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/10.jpg)
Pattern Classification, Chapter 1
10
The length is a poor feature alone!
Select the lightness as a possible feature.
![Page 11: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/11.jpg)
Pattern Classification, Chapter 1
11
![Page 12: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/12.jpg)
Pattern Classification, Chapter 1
12
• Threshold decision boundary and cost relationship
• Move our decision boundary toward smaller values of lightness in order to minimize the cost (reduce the number of sea bass that are classified salmon!)
Task of decision theory
![Page 13: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/13.jpg)
Pattern Classification, Chapter 1
13
• Adopt the lightness and add the width of the fish
Fish xT = [x1, x2]
Lightness Width
![Page 14: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/14.jpg)
Pattern Classification, Chapter 1
14
![Page 15: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/15.jpg)
Pattern Classification, Chapter 1
15
• We might add other features that are not correlated with the ones we already have. A precaution should be taken not to reduce the performance by adding such “noisy features”
• Ideally, the best decision boundary should be the one which provides an optimal performance such as in the following figure:
![Page 16: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/16.jpg)
Pattern Classification, Chapter 1
16
![Page 17: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/17.jpg)
Pattern Classification, Chapter 1
17
• However, our satisfaction is premature because the central aim of designing a classifier is to correctly classify novel input
Issue of generalization!
![Page 18: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/18.jpg)
Pattern Classification, Chapter 1
18
![Page 19: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/19.jpg)
Pattern Classification, Chapter 1
19
Pattern Recognition Systems
• Sensing
• Use of a transducer (camera or microphone)• PR system depends of the bandwidth, the resolution
sensitivity distortion of the transducer
• Segmentation and grouping
• Patterns should be well separated and should not overlap
![Page 20: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/20.jpg)
Pattern Classification, Chapter 1
20
![Page 21: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/21.jpg)
Pattern Classification, Chapter 1
21
• Feature extraction• Discriminative features• Invariant features with respect to translation, rotation and
scale.
• Classification• Use a feature vector provided by a feature extractor to
assign the object to a category
• Post Processing• Exploit context input dependent information other than from
the target pattern itself to improve performance
![Page 22: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/22.jpg)
Pattern Classification, Chapter 1
22
The Design Cycle
• Data collection• Feature Choice• Model Choice• Training• Evaluation• Computational Complexity
![Page 23: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/23.jpg)
Pattern Classification, Chapter 1
23
![Page 24: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/24.jpg)
Pattern Classification, Chapter 1
24
• Data Collection
• How do we know when we have collected an adequately large and representative set of examples for training and testing the system?
![Page 25: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/25.jpg)
Pattern Classification, Chapter 1
25
• Feature Choice
• Depends on the characteristics of the problem domain. Simple to extract, invariant to irrelevant transformation insensitive to noise.
![Page 26: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/26.jpg)
Pattern Classification, Chapter 1
26
• Model Choice
• Unsatisfied with the performance of our fish classifier and want to jump to another class of model
![Page 27: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/27.jpg)
Pattern Classification, Chapter 1
27
• Training
• Use data to determine the classifier. Many different procedures for training classifiers and choosing models
![Page 28: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/28.jpg)
Pattern Classification, Chapter 1
28
• Evaluation
• Measure the error rate (or performance and switch from one set of features to another one
![Page 29: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/29.jpg)
Pattern Classification, Chapter 1
29
• Computational Complexity
• What is the trade-off between computational ease and performance?
• (How an algorithm scales as a function of the number of features, patterns or categories?)
![Page 30: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/30.jpg)
Pattern Classification, Chapter 1
30
Learning and Adaptation
• Supervised learning
• A teacher provides a category label or cost for each pattern in the training set
• Unsupervised learning
• The system forms clusters or “natural groupings” of the input patterns
![Page 31: Pattern Classificationdpinto/pln/Autumn2010/DHSch1.pdf · Pattern Classification, Chapter 1 6 • Preprocessing • Use a segmentation operation to isolate fishes from one another](https://reader033.vdocuments.us/reader033/viewer/2022052723/5f0e2bbe7e708231d43df2de/html5/thumbnails/31.jpg)
Pattern Classification, Chapter 1
31
Conclusion
• Reader seems to be overwhelmed by the number, complexity and magnitude of the sub-problems of Pattern Recognition
• Many of these sub-problems can indeed be solved
• Many fascinating unsolved problems still remain