csse463: image recognition day 11 lab 4 (shape) tomorrow: feel free to start in advance lab 4...
TRANSCRIPT
CSSE463: Image Recognition CSSE463: Image Recognition Day 11Day 11
Lab 4 (shape) tomorrow: feel free to start in Lab 4 (shape) tomorrow: feel free to start in advanceadvance
Test Monday through shape (no pattern rec.)Test Monday through shape (no pattern rec.) Start thinking about term project ideas.Start thinking about term project ideas. Questions?Questions?
Next 1.5 weeks: Pattern recognitionNext 1.5 weeks: Pattern recognition Concepts, error types (today)Concepts, error types (today) Basic theory and how to use classifiers in MATLAB:Basic theory and how to use classifiers in MATLAB:
Support vector machines (SVM).Support vector machines (SVM). Neural networksNeural networks
Pattern recognitionPattern recognition Making a decision from Making a decision from
datadata A classification problem: A classification problem:
assign a single class label assign a single class label to a datum pointto a datum point
Can include a special class, Can include a special class, rejectreject, ,
if a sample (a single datum if a sample (a single datum point) appears not to belong to point) appears not to belong to any known classany known class
If it is on the boundary If it is on the boundary between classesbetween classes
Else Else forced classificationforced classification Boundaries between Boundaries between
classes-how?classes-how? There’s tons of theory, There’s tons of theory,
can be applied to many can be applied to many areas. We focus on small areas. We focus on small subset of those used for subset of those used for visionvision Q1
Baseline: Hand-tuned decision Baseline: Hand-tuned decision boundariesboundaries
You did this based on observations for fruit You did this based on observations for fruit classificationclassification
You’ll do the same thing in Lab 4 for You’ll do the same thing in Lab 4 for shapesshapes
But what if the features were much more But what if the features were much more complex?complex?We now discuss classifiers that learn class We now discuss classifiers that learn class
boundaries based on boundaries based on exemplarsexemplars (e.g., labeled (e.g., labeled training examples)training examples)
Ex: Nearest neighbor classifierEx: Nearest neighbor classifier Assumes we have a feature Assumes we have a feature
vector for each imagevector for each image Calculate distance from Calculate distance from new new
test sampletest sample to to each labeled each labeled training sampletraining sample. .
Assign label as closest training Assign label as closest training samplesample
Generalize by assigning same Generalize by assigning same label as the majority of the k label as the majority of the k nearest neighbors. No majority?nearest neighbors. No majority?
d
i
ipipppdDIn
ypypxpxpppDIn
1
22121
221
22121
))()((,
))()(())()((,2
Nearest class meanNearest class mean
Find class means and Find class means and calculate distance to calculate distance to each meaneach mean Pro?Pro? Con?Con?
Partial solution: Partial solution: clusteringclustering Learning vector Learning vector
quantization (LVQ): quantization (LVQ): tries to find optimal tries to find optimal clustersclusters
LVQ
Test point
Q2
Common model of learning Common model of learning machinesmachines
StatisticalLearning
Labeled TrainingImages
Extract Features(color, texture)
TestImage
Summary
LabelClassifierExtract Features
(color, texture)
Focus on testingFocus on testingLet m = the number of possible class labelsLet m = the number of possible class labelsConsider m==2. Consider m==2. Example: Calculate distance to cluster Example: Calculate distance to cluster
means for 2 classes.means for 2 classes.
Dist/prob1
Dist/prob2
TestImage
Extract Features(color, texture)
Decide Class:1, 2
How good is your classifier?How good is your classifier?
Example from medicine: Example from medicine: Disease detectionDisease detection
Consider costs of false Consider costs of false neg. vs. false pos.neg. vs. false pos.
Lots Lots of different error of different error measuresmeasures Accuracy = 10500/10800 = Accuracy = 10500/10800 =
97%. Is 97% accuracy OK?97%. Is 97% accuracy OK? Recall (or true positive Recall (or true positive
rate) = 500/600=83%rate) = 500/600=83% Precision = 500/700=71%Precision = 500/700=71% False pos rate = 200/10200 False pos rate = 200/10200
= 2%= 2%
DetectedDetected
TrueTrue
YesYes NoNo
YesYes 500500
(true (true pos.)pos.)
100100
(false (false neg.)neg.)
NoNo 200200
(false (false pos.)pos.)
1000010000
(true (true neg.)neg.)
10200Total actualnegative
600Total actualpositive
700Total det. as pos.
10100Total det. as neg.
How good is your classifier?How good is your classifier?
Write out definitions of Write out definitions of each measure noweach measure now
ExamplesExamples Accuracy = 10500/10800 = Accuracy = 10500/10800 =
97%. 97%. Recall (or true positive Recall (or true positive
rate) = 500/600=83%rate) = 500/600=83% Precision = 500/700=71%Precision = 500/700=71% False pos rate = 200/10200 False pos rate = 200/10200
= 2%= 2%
Detected:Detected:
Has:Has:
YesYes NoNo
YesYes 500500
(true (true pos.)pos.)
100100
(false (false neg.)neg.)
NoNo 200200
(false (false pos.)pos.)
1000010000
(true (true neg.)neg.)
Q3a-d
Thresholding real-valued output allows you to Thresholding real-valued output allows you to tradeoff TPR and FPRtradeoff TPR and FPR
If t == 0: TPR = ___, FPR = ___If t == 0: TPR = ___, FPR = ___
9/129/12 2/82/8
If t == 1: TPR = ___, FPR = ___If t == 1: TPR = ___, FPR = ___
Repeat for many values of tRepeat for many values of t
-3 -2 -1 0 1 2 3
NN N P N P N N P N PP N PP PP PPP
Simple example: Classes P = positive, N = negative, and single Simple example: Classes P = positive, N = negative, and single real-valued output.real-valued output.
False Pos Rate
Tru
e P
os R
ate
True class:
Output:
Threshold output to get class. label = output > t ? P : NChoice of threshold a
ROC curve ROC curve
Receiver-operating Receiver-operating characteristiccharacteristic
Useful when you Useful when you can change a can change a threshold to get threshold to get different true and different true and false positive ratesfalse positive rates
Consider extremesConsider extremes Much more Much more
information information recorded here!recorded here!
Q3
Multiclass problems Multiclass problems
Consider m>2. Consider m>2. Example: Calculate distance to cluster Example: Calculate distance to cluster
means for 10 classes.means for 10 classes.
Dist/prob1
Dist/prob2
Dist/probN
TestImage
Extract Features(color, texture)
Decide Class:1, 2,…N
Confusion matrices for m>2 Confusion matrices for m>2 (outdoor image example)(outdoor image example)
Beach recall: 169/(169+0+2+3+12+14)=84.5%Beach recall: 169/(169+0+2+3+12+14)=84.5% Note confusion between mountain and urban Note confusion between mountain and urban
classes due to featuresclasses due to features Similar colors and spatial layoutSimilar colors and spatial layout
Bch Sun FF Fld Mtn UrbBch 169 0 2 3 12 14Sun 2 183 5 0 5 5FF 3 6 176 6 4 5Fld 15 0 1 173 11 0Mtn 11 0 2 21 142 24Urb 16 4 8 5 27 140
Detected
Tru
e
Q4
Why do we need separate training Why do we need separate training and test sets?and test sets?
Exam analogyExam analogy
But working on practice questions is helpful…get But working on practice questions is helpful…get the analogy? the analogy? We hope our ability to do well on We hope our ability to do well on practice questions helps us on the actual exampractice questions helps us on the actual exam
Application to nearest-neighbor classifiersApplication to nearest-neighbor classifiers
Often reserve a 3Often reserve a 3rdrd set for validation as well (to set for validation as well (to tune parameters of training set)tune parameters of training set)
Q5-8
Look at this to understand Look at this to understand nearest neighbor nearest neighbor
http://ai6034.mit.edu/fall12/index.php?title=Demonstrations
Shows Voronai diagrams for nearest Shows Voronai diagrams for nearest neighbor classifiersneighbor classifiers
Nice intro to SVMs alsoNice intro to SVMs also