outline course description, what is pattern recognition, cost of error, decision boundaries, the...
Post on 19-Dec-2015
216 views
TRANSCRIPT
![Page 1: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/1.jpg)
![Page 2: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/2.jpg)
OUTLINE
Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle
![Page 3: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/3.jpg)
The objective of the course is to make student familiar with general approaches such as – Bayes classification, – discriminant functions, – decision trees, – nearest neighbor rule, – neural networks for pattern recognition.
Course description
![Page 4: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/4.jpg)
Course Book:– R.O. Duda, P.E. Hart, and
D.G. Stork, Pattern Classification, New York: John Wiley, 2001,
Course description
![Page 5: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/5.jpg)
Course description
WEEKLY SCHEDULE AND PRE-STUDY PAGES
Week
Topics Pre-study Pages
1 Introduction Chapter 1 (main text)
2 Bayesian Decision Theory Chapter 2
3 Bayesian Decision Theory Chapter 2
4 Bayesian Decision Theory Chapter 2
5 Maximum – Likelihood and Bayesian Parameter Estimation Chapter 3
6 Maximum – Likelihood and Bayesian Parameter Estimation Chapter 3
7 Nonparametric Techniques Chapter 4
8 Nonparametric Techniques Chapter 4
9 Linear Discriminant Functions Chapter 5
10 Linear Discriminant Functions Chapter 5
11 Multilayer Neural Networks Chapter 6
12 Nonmetric Methods Chapter 8
13 Unsupervised Learning and Clustering Chapter 10
14 Unsupervised Learning and Clustering Chapter 10
![Page 6: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/6.jpg)
EVALUATION SYSTEM
IN-TERM STUDIES QUANTITY PERCENTAGE
Mid-terms 2 2x20=40
Assignment 3 30
Final Exam 1 30
TOTAL 100
Course description
![Page 7: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/7.jpg)
What is a pattern ?
an entity, vaguely defined, that could be given a name,
e.g.:– fingerprint image,– handwritten word,– human face,– speech signal,– DNA sequence,
![Page 8: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/8.jpg)
What is pattern recognition ?
a discipline which learn some theories and methods to design machines that can recognize patterns in noisy data or complex environment (Srihari,Govindaraju).
![Page 9: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/9.jpg)
What is pattern recognition ?
a scientific discipline whose aim is the classification of the objects into a lot of categories or classes. Pattern recognition is also a integral part in most machine intelligence system built for decision making (Sergios Theodoridis).
![Page 10: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/10.jpg)
What is pattern recognition ?
the act of taking in raw data and making an action based on the category of the pattern (Duda and Hart)
![Page 11: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/11.jpg)
What is pattern recognition ?
Pattern recognition is the study of how machines can:– observe the environment,– learn to distinguish patterns of interest,– make sound and reasonable decisions about the
categories of the patterns.
![Page 12: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/12.jpg)
What is pattern recognition ?
Some Applications:
![Page 13: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/13.jpg)
What is pattern recognition ?
Some Applications:
![Page 14: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/14.jpg)
What is pattern recognition ?
Some Applications:
![Page 15: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/15.jpg)
What is pattern recognition ?
Some Applications:
![Page 16: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/16.jpg)
What is pattern recognition ?
Some Applications:
![Page 17: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/17.jpg)
What is pattern recognition ?
Some Applications:
![Page 18: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/18.jpg)
An Example
Suppose that:– A fish packing plant wants
to automate the process of sorting incoming fish on a conveyor belt according to species,
– There are two species: Sea bass, Salmon.
![Page 19: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/19.jpg)
An Example
How to distinguish one specie from the other ? (length, width, weight, number and shape of fins, tail shape,etc.)
![Page 20: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/20.jpg)
An Example
Suppose somebody at the fish plant say us that:– Sea bass is generally longer than a salmon
Then our models for the fish:– Sea bass have some typical length, and this is
greater than that for salmon.
![Page 21: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/21.jpg)
An Example
Then length becomes a feature, We might attempt to classify the fish by
seeing whether or not the length of a fish exceeds some critical value (threshold value) l*.
![Page 22: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/22.jpg)
An Example
How to desie on the critical value (threshold value) ?
![Page 23: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/23.jpg)
An Example
How to desie on the critical value (threshold value) ?– We could obtain some training samples of
different types of fish,– make length measurements,– Inspect the results.
![Page 24: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/24.jpg)
An Example
Measurement results on the training sample related to two species.
![Page 25: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/25.jpg)
An Example
Can we reliably seperate sea bass from salmon by using length as a feature ?
Remember our model:
–Sea bass have some typical length, and this is greater than that for salmon.
![Page 26: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/26.jpg)
An Example
From histogram we can see that single criteria is quite poor.
![Page 27: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/27.jpg)
An Example
It is obvious that length is not a good feature.
What we can do to seperate sea bass from salmon?
![Page 28: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/28.jpg)
An Example
What we can do to seperate sea bass from salmon?
Try another feature: – average lightness of the fish scales.
![Page 29: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/29.jpg)
An Example
Can we reliably seperate sea bass from salmon by using lightness as a feature ?
![Page 30: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/30.jpg)
An Example
Lighness is better than length as a feature but again there are some problems.
![Page 31: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/31.jpg)
An Example
Suppose we also know that:– Sea bass are typically wider than salmon.
We can use more than one feature for our decision:– Lightness (x1) and width (x2)
![Page 32: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/32.jpg)
An Example
Each fish is now a point in two dimension.– Lightness (x1) and width (x2)
![Page 33: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/33.jpg)
An Example
Each fish is now a point in two dimension.– Lightness (x1) and width (x2)
![Page 34: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/34.jpg)
An Example
Each fish is now a point in two dimension.– Lightness (x1) and width (x2)
![Page 35: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/35.jpg)
Cost of error
Cost of different errors must be considered when making decisions,
We try to make a decision rule so as to minimize such a cost,
This is the central task of decision theory.
![Page 36: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/36.jpg)
Cost of error
For example, if the fish packing company knows that:– Customers who buy salmon will object if they
see sea bass in their cans.– Customers who buy sea bass will not be
unhappy if they occasionally see some expensive salmon in their cans.
![Page 37: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/37.jpg)
Decision boundaries
We can perform better if we use more complex decision boundaries.
![Page 38: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/38.jpg)
Decision boundaries
There is a trade of between complexity of the decision rules and their performances to unknown samples.
Generalization: The ability of the classifier to produce correct results on novel patterns.
Simplify the decision boundary!
![Page 39: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/39.jpg)
The design cycle
![Page 40: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/40.jpg)
The design cycle
Collect data:– Collect train and test data
Choose features:– Domain dependence and prior information,– Computational cost and feasibility,– Discriminative features,– Invariant features with respect to translation, rotation and
scale,– Robust features with respect to occlusion, distortion,
deformation, and variations in environment.
![Page 41: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/41.jpg)
The design cycle
Choose model:– Types of models: templates, decision-theoretic
or statistical, syntactic or structural, neural, and hybrid.
Train classifier:– learn the rule from data,
Evaluate classifier:– estimate the performance– problems of overfitting and generalization.
![Page 42: OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d3a5503460f94a147a7/html5/thumbnails/42.jpg)
References
Srihari, S.N., Covindaraju, Pattern recognition, Chapman &Hall, London, 1034-1041, 1993, Sergios Theodoridis, Konstantinos Koutroumbas , pattern recognition , Pattern
Recognition ,Elsevier(USA)) ,1982 R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, New York: John Wiley, 2001,