deciphering the faceclopinet.com/isabelle/projects/cvpr2011/slides/aleix.pdfaleix m. martinez...
TRANSCRIPT
![Page 1: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/1.jpg)
Deciphering the FaceDeciphering the Face
Aleix M. MartinezComputational BiologyComputational Biology
and Cognitive Science Labl i @ [email protected]
![Page 2: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/2.jpg)
Human-ComputerPoliticsInteraction
HumanHumanfacefaceArt
Sign Language
faceface
Language
CognitiveCognitiveScience Computer Vision
![Page 3: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/3.jpg)
Models of Face Perception
• Features: Shape vs. texture. ……
• 2D vs. 3D
• Form of the computational space:p p
Continuous vs. Categorical
![Page 4: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/4.jpg)
What we are going to show
• What is the form of the computational space in human face perception? Hybrid approach:in human face perception? Hybrid approach: Linear combination of continuous representations of categoriesrepresentations of categories.
+ c2c1 + … + cn
• What are the dimensions? Mostly configural.
21 n
• In computer vision we need precise detailed detection of faces and facial features.detect o o aces a d ac a eatu es.
![Page 5: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/5.jpg)
Identity
Same or different?
![Page 6: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/6.jpg)
Identity
Same or different?
![Page 7: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/7.jpg)
Identity
Same or different?Identity, expression, gender, etc.
![Page 8: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/8.jpg)
Dimensions of the Face Space
Same or different?Configural processing
![Page 9: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/9.jpg)
Form of the Computational Face SpaceComputational Face Space
Exemplar-based modelExemplar based model
Exemplar cells …
Norm-based modelMid-level
cells
vision
Low-level vision
![Page 10: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/10.jpg)
Facial Expressions of Emotion
![Page 11: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/11.jpg)
Muscle Positions Model
![Page 12: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/12.jpg)
Muscle Positions Model
• Global shape (bone structure)determines identity – configural.y g
• But ONLY muscles are responsiblefor expression interactionfor expression, interaction …
![Page 13: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/13.jpg)
![Page 14: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/14.jpg)
Configural Processing
Emotion perception inl femotionless faces
NeutralNeutralAngry
Sad
Neth & Martinez, JOV, 2009.
![Page 15: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/15.jpg)
Stimuli
25%
50% 100%
75%
Neth & Martinez, JOV, 2009.
![Page 16: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/16.jpg)
ExperimentExperiment
Less, same, more.
![Page 17: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/17.jpg)
Configural Processing
Sad
*
**
** *
*80
90
**
50
60
70
LessSame
*
* ** * *
*
20
30
40More
0
10
-100% -75% -50% -25% 0% 25% 50% 75% 100%
Neth & Martinez, JOV, 2009.
![Page 18: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/18.jpg)
Configural Processing
Angry
* **
* *80
90
*
**
*
50
60
70
LessSame
***
* *
***20
30
40More
0
10
-100% -75% -50% -25% 0% 25% 50% 75% 100%
Neth & Martinez, JOV, 2009.
![Page 19: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/19.jpg)
Norm-based Face SpaceSadness
MultidimensionalS 75%
100%
Face Space
- density+ density
50%
75%
+ density 25%
Easier
+ density- density
MEAN
density100%
More difficult
AngerNeth & Martinez, JOV, 2009.
![Page 20: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/20.jpg)
Configural Processing
Neth & Martinez, JOV, 2009.
![Page 21: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/21.jpg)
Computational Space
Neth & Martinez, Vision Research, 2010
![Page 22: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/22.jpg)
Computational Space
Thinner faceThinner face
Wider face
Neth & Martinez, Vision Research, 2010
![Page 23: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/23.jpg)
American Gothic Illusion
Neth & Martinez, Vision Research, 2010
![Page 24: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/24.jpg)
Why Configural Features?
![Page 25: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/25.jpg)
![Page 26: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/26.jpg)
15 x 10 pixels
![Page 27: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/27.jpg)
Why Configural cues?
sad neutral angry
Neth & Martinez, Vision Research, 2010; Du & Martinez, 2011
![Page 28: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/28.jpg)
Proposed Hybrid Model:Recognizing other emotion labelsRecognizing other emotion labels
+ cc + + c+ c2c1 + … + cn
Happily Angrily surprised
g ysurprised
Martinez, CVPR, 2011
![Page 29: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/29.jpg)
Configural Processing = Precise detection of facial featuresdetection of facial features
4 2 pixels3,930
images
4.2 pixelserror
(1.5%)(1.5%)
Ding & Martinez, PAMI, 2010
![Page 30: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/30.jpg)
Face Detection
![Page 31: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/31.jpg)
Features VS contextObservation: Most detections are near the correct location – they are not incorrect, they are imprecise.location they are not incorrect, they are imprecise.
Key idea: Use context information to train where nott d t t f d f i l f t
Ding & Martinez, CVPR, 2008; PAMI, 2010
to detect faces and facial features.
![Page 32: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/32.jpg)
Features VS contextObservation: Most detections are near the correct location – they are not incorrect, they are imprecise.location they are not incorrect, they are imprecise.
Key idea: Use context information to train where nott d t t f d f i l f tto detect faces and facial features.
Ding & Martinez, CVPR, 2008; PAMI, 2010
![Page 33: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/33.jpg)
Features VS contextObservation: Most detections are near the correct location – they are not incorrect, they are imprecise.location they are not incorrect, they are imprecise.
Key idea: Use context information to train where nott d t t f d f i l f tto detect faces and facial features.
Ding & Martinez, CVPR, 2008; PAMI, 2010
![Page 34: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/34.jpg)
Subclass Discriminant Analysisy
Between subclassBetween-subclass scatter matrix:
( ) ( )∑∑C H
Ti
Σ ( ) ( ).1 1∑∑= =
−−=i j
ijT
ijijB p μμμμΣ
Basis vectors:
.Λ= VΣVΣ XB
Basis vectors:
How many subclasses (H):Minimize the conflict, K.
Zhu & Martinez, PAMI, 2006
![Page 35: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/35.jpg)
Precise Detailed Detection
E 6 2 i l (2%) M l 4 2 (1 5%)Error: 6.2 pixels (2%) vs Manual: 4.2 (1.5%)
Ding & Martinez, CVPR, 2008; PAMI, 2010
![Page 36: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/36.jpg)
Detection + non-rigid SfM
Gotardo & Martinez, PAMI, 2011; Gotardo & Martinez, CVPR, 2011.
36
![Page 37: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/37.jpg)
Take Home Messages
• What is the form of the computational space in human face perception? Linear combination of known categories.
+ c2c1 + … + cn
Wh t th di i ? M tl fi l
21 n
• What are the dimensions? Mostly configural.• Precise detection of facial features.
![Page 38: Deciphering the Faceclopinet.com/isabelle/Projects/CVPR2011/slides/aleix.pdfAleix M. Martinez Computational BiologyComputational Biology and Cognitive Science Lab ali@ dleix@ece.osu.edu](https://reader034.vdocuments.us/reader034/viewer/2022052009/601f1b917fd68358bd2195d6/html5/thumbnails/38.jpg)
CBCSL
Paulo Gotardo, Shichuan Du, Don Neth, Liya Ding, OnurPaulo Gotardo, Shichuan Du, Don Neth, Liya Ding, Onur Hamsici, Samuel Rivera, Fabian Benitez, Hongjun Jia, Di You.
National Institutes of HealthNational Institutes of HealthNational Science Foundation