recognition – pca and templates. recognition suppose you want to find a face in an imagesuppose...
Post on 21-Dec-2015
216 views
TRANSCRIPT
![Page 1: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/1.jpg)
Recognition –Recognition –PCA and TemplatesPCA and Templates
![Page 2: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/2.jpg)
RecognitionRecognition
• Suppose you want to find a face in an Suppose you want to find a face in an imageimage
• One possibility: look for something One possibility: look for something that looks sort of like a face (oval, dark that looks sort of like a face (oval, dark band near top, dark band near bottom)band near top, dark band near bottom)
• Another possibility: look for pieces of Another possibility: look for pieces of faces (eyes, mouth, etc.) in a specific faces (eyes, mouth, etc.) in a specific arrangementarrangement
![Page 3: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/3.jpg)
RecognitionRecognition
• Suppose you want to recognize aSuppose you want to recognize aparticularparticular face face
• How does How does thisthis face differ from face differ from average faceaverage face
![Page 4: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/4.jpg)
TemplatesTemplates
• Model of a “generic” or “average” Model of a “generic” or “average” faceface– Learn templates from example dataLearn templates from example data
• For each location in image, look for For each location in image, look for template at that locationtemplate at that location– Optionally also search over scale, Optionally also search over scale,
orientationorientation
![Page 5: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/5.jpg)
TemplatesTemplates
• In the simplest case, based on intensityIn the simplest case, based on intensity– Template is average of all faces in Template is average of all faces in
training settraining set– Comparison based on e.g. SSDComparison based on e.g. SSD
• More complex templatesMore complex templates– Outputs of feature detectorsOutputs of feature detectors– Color histogramsColor histograms– Often combine position and frequency Often combine position and frequency
information (wavelets)information (wavelets)
![Page 6: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/6.jpg)
Face Detection ResultsFace Detection Results
Sample ImagesSample Images
WaveletWaveletHistogramHistogramTemplateTemplate
Detection ofDetection offrontal / profilefrontal / profile
facesfaces
![Page 7: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/7.jpg)
More Face Detection ResultsMore Face Detection Results
Schneiderman and KanadeSchneiderman and Kanade
![Page 8: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/8.jpg)
How to Recognize Specific How to Recognize Specific People?People?
• Consider variation from average faceConsider variation from average face
• Not all variations equally importantNot all variations equally important– Variation in a single pixel relatively Variation in a single pixel relatively
unimportantunimportant
• If image is high-dimensional vector, If image is high-dimensional vector, want to find directions in this space want to find directions in this space along which variation is highalong which variation is high
![Page 9: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/9.jpg)
Principal Components Principal Components AnalaysisAnalaysis
• Principal Components Analysis (PCA): Principal Components Analysis (PCA): approximating a high-dimensional approximating a high-dimensional data set with a lower-dimensional data set with a lower-dimensional subspacesubspace
Original axesOriginal axes
****
******
**** **
**
********
**
**
****** **
**** ******
Data pointsData points
First principal componentFirst principal componentSecond principal componentSecond principal component
![Page 10: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/10.jpg)
Principal ComponentsPrincipal Components
• Computing PCA:Computing PCA:– Subtract out mean (“whitening”)Subtract out mean (“whitening”)– Find eigenvalues of covariance matrixFind eigenvalues of covariance matrix– Equivalently, compute SVD of data Equivalently, compute SVD of data
matrixmatrix
![Page 11: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/11.jpg)
PCA on Faces: “Eigenfaces”PCA on Faces: “Eigenfaces”
AverageAveragefaceface
First principal componentFirst principal component
OtherOthercomponentscomponents
For all except average,For all except average,“gray” = 0,“gray” = 0,
“white” > 0,“white” > 0,““black” < 0black” < 0
![Page 12: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/12.jpg)
Using PCA for RecognitionUsing PCA for Recognition
• Store each person as coefficients of Store each person as coefficients of projection onto first few principal projection onto first few principal componentscomponents
• Compute projections of target image, Compute projections of target image, compare to databasecompare to database
max
0iEigenfaceimage
i
iia
max
0iEigenfaceimage
i
iia
![Page 13: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/13.jpg)
Recognition UsingRecognition UsingRelations Between TemplatesRelations Between Templates
• Often easier to recognize a small featureOften easier to recognize a small feature– e.g., lips easier to recognize than facese.g., lips easier to recognize than faces– For articulated objects (e.g. people), For articulated objects (e.g. people),
template for whole class usually template for whole class usually complicatedcomplicated
• So, identify small pieces and look for So, identify small pieces and look for spatial arrangementsspatial arrangements– Many false positives from identifying Many false positives from identifying
piecespieces
![Page 14: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/14.jpg)
Graph MatchingGraph Matching
HeadHead
LegLegLegLeg
ArmArm
ArmArm
BodyBody
HeadHead
HeadHead
BodyBody
BodyBody
ArmArm
ArmArm LegLeg
LegLeg LegLeg
LegLeg
ModelModel Feature detection resultsFeature detection results
![Page 15: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/15.jpg)
Graph MatchingGraph Matching
HeadHead
LegLegLegLeg
ArmArm
ArmArm
BodyBody
ConstraintsConstraints
Next to, right of
Next to, right ofNext to
, below
Next to, below
![Page 16: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/16.jpg)
Graph MatchingGraph Matching
HeadHead
LegLegLegLeg
ArmArm
ArmArm
BodyBody
HeadHead
HeadHead
BodyBody
BodyBody
ArmArm
ArmArm LegLeg
LegLeg LegLeg
LegLeg
Combinatorial searchCombinatorial search
![Page 17: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/17.jpg)
Graph MatchingGraph Matching
HeadHead
LegLegLegLeg
ArmArm
ArmArm
BodyBody
HeadHead
HeadHead
BodyBody
BodyBody
ArmArm
ArmArm LegLeg
LegLeg LegLeg
LegLeg
Combinatorial searchCombinatorial search
OKOK
![Page 18: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/18.jpg)
Graph MatchingGraph Matching
HeadHead
LegLegLegLeg
ArmArm
ArmArm
BodyBody
HeadHead
HeadHead
BodyBody
BodyBody
ArmArm
ArmArm LegLeg
LegLeg LegLeg
LegLeg
Combinatorial searchCombinatorial search
Already assignedAlready assigned
![Page 19: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/19.jpg)
Graph MatchingGraph Matching
HeadHead
LegLegLegLeg
ArmArm
ArmArm
BodyBody
HeadHead
HeadHead
BodyBody
BodyBody
ArmArm
ArmArm LegLeg
LegLeg LegLeg
LegLeg
Combinatorial searchCombinatorial search
ViolatesViolatesconstraintconstraint
![Page 20: Recognition – PCA and Templates. Recognition Suppose you want to find a face in an imageSuppose you want to find a face in an image One possibility: look](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b0ff/html5/thumbnails/20.jpg)
Graph MatchingGraph Matching
• Large search spaceLarge search space– Heuristics for pruningHeuristics for pruning
• Missing featuresMissing features– Look for maximal consistent Look for maximal consistent
assignmentassignment
• Noise, spurious featuresNoise, spurious features
• Incomplete constraintsIncomplete constraints– Verification step at endVerification step at end