face detection using the spectral histogram representation by: christopher waring, xiuwen liu...
TRANSCRIPT
![Page 1: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/1.jpg)
Face Detection using the Spectral Histogram
representation
By: Christopher Waring, Xiuwen LiuDepartment of Computer Science
Florida State University
Presented by:
Tal Blum [email protected]
![Page 2: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/2.jpg)
Sources
• The presentation is based on a few resources by the authors:– Exploration of the Spectral Histogram for Face
Detection – M.Sc thesis by Christopher Waring (2002)– Spectral Histogram Based Face Detection – IEEE
(2003)– Rotation Invariant Face Detection Using Spectral
Histograms & SVM – CVPR submission– Independent Spectral Representation of images for
Recognition – Optical Society of America (2003)
![Page 3: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/3.jpg)
Overview• Spectral Histogram
– Overview of Gibbs Sampling + Simulated annealing
• Method for Lighting Normalization
• Data used
• 3 Algorithms
– SH + Neural Networks
– SH + SVM
– Rotation Invariant SH +SVM
• Experimental Results
• Conclusions & Discussions
![Page 4: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/4.jpg)
Two Approaches to Object Detection
• Curse of dimensionality– Features should be: (Vasconcelos)
• Independent • have low Bayes Error
• 2 main Approaches in Object Detection:– Complicated Features with many interactions
• Require many data points• Use syntactic variations that mimic the real variations• Estimation Error might be high• Assuming Model or Parameter structure
– Small set of features or small number of values• This is the case for Spectral Histograms• The Bayes Error might be high (Vasconcelos)• Estimation Error is low
![Page 5: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/5.jpg)
Why Spectral Histograms?
• Translation Invariant– Therefore insensitive to incorrect alignment.
• (surprisingly) seem to be able to separate Objects from Non-Objects well.
• Good performance with a very small feature set.• Good performance with a large rotation
invariance.• Don’t rely at all on any global spatial information • Combining of variant and invariant features• Will play a more Important role
![Page 6: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/6.jpg)
What is Spectral Histogram
),...,3,2,1:(
))((||
1)(
)()()(*)(
)(
)()(
KHH
vIzI
zH
uvIuFvIFvI
II
vI
u
![Page 7: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/7.jpg)
Types of Filters• 3 types of filters:
– Gradient Filters
– Gabor Filters
– Laplasian of Gaussians Filters
Ty
x
D
D
]1,1,0[*
]1,1,0[*
The exact composition of the filters is different for each algorithm.
Tyy
xx
D
D
]1,2,1[*
]1,2,1[*
2
22
)(*),,( 222 T
yx
eTyxTyxLoG
))sin()((2)))sin()cos(())sin()cos((4(
2
1 222
*),|,(
yx
Tiyxyx
T eeTyxG
![Page 8: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/8.jpg)
Gibbs Sampling+ Simulated Annealing
• We want to sample from• We can use the induced Gibbs Distribution
• Algorithm:• Repeat
– Randomly pick a location– Change the pixel value according to q
• Until for every filter
} )R(IR(I)| I { obs
})(
exp{1
)(
I
ZIq
),(obssyn II HHD
))(),(()( obsIRIRDI
![Page 9: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/9.jpg)
Face Synthesis usingGibbs Sampling + Simulated
Annealing
•A measure of the quality of the Representation
![Page 10: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/10.jpg)
Comparison - PCA vs. Spectral Histogram
Original Image Reconstructed Images
![Page 11: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/11.jpg)
Reconstruction vs. Sampling
Reconstruction sampling
![Page 12: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/12.jpg)
Spectral Histograms of several images
![Page 13: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/13.jpg)
Lighting correction
• They use a 21x21 sized images
• Minimal brightness plane of 3x3 is computed from each 7x7 block
• A 21x21 correction plane is computed by bi-linear interpolation
• Histogram Normalization is applied
![Page 14: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/14.jpg)
Lighting correction
![Page 15: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/15.jpg)
Detection & Post Processing
• Detection is don on 3 scaled Gaussian pyramid, each scale down sampled by1.1
• detections within 3 pixels are merged
• A detection is marked as final if it is found at at least two concurrent levels
• A detection counts as correct if at least half of the face lies within the detection window
![Page 16: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/16.jpg)
Adaptive Threshold
![Page 17: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/17.jpg)
Algorithm Iusing a Neural Network
• Neural Network was used as a classifier– Training with back propagation
• Data Processing– 1500 Face images & 8000 Non-Face images– Bootstrapping was used to limit the # non faces
(Sung Poggio) leaving 800 Non-Faces
• Use 8 filters with 80 bins in each
![Page 18: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/18.jpg)
Alg. I - Filter Selection• 7 LoG filters with • 4 Difference of gradient: Dx Dy Dxx Dyy• 70 Gabor filters with:
– T = 2,4,6,8,10,12,14– = 0,40,80,120,160,200,280,320
• Selected Filters (8 out of 81)• 4 LoG filters with:• 3 Difference of Gradiant: Dx Dxx & Dyy• 1 Gabor filter with T=2 and
6,5,4,3,2,1,2
2T
5,3,2,2
2T
320
![Page 19: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/19.jpg)
Spectral Histograms of several images
![Page 20: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/20.jpg)
Algorithm I – Resultson CMU test set I
Method Detection Rate
False Detections
Waring & Liu 93.8% 94Yang, Ahuja & Kreigman 93.6% 74
Yang, Ahuja & Kreigman 92.3% 82
Yang Roth & Ahuja 94.2% 84Rowley, Baluja & Kanade 92.5% 862
Schneiderman 93.0% 88Colmenarz & Huang 98.0% 12758
![Page 21: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/21.jpg)
Algorithm I – Resultson CMU test set II
Method Detection Rate
False Detections
Waring & Liu 89.4% 29Sung & Poggio 81.9 13
Rowley, Baluja & Kanade 90.3% 42
Yang, Ahuja & Kreigman 91.5% 1Yang, Ahuja & Kreigman 89.4% 3
Schneiderman 91.2% 12Yang Roth & Ahuja 93.6% 3
![Page 22: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/22.jpg)
Algorithm IIusing a SVM
• SVM instead of a Neural Network
• They use more filters– 34 filters (instead of 7)– 359 bins (instead of 80)
• 4500 randomly rotated Face images & 8000 Non-Face images from before
![Page 23: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/23.jpg)
Algorithm II (SVM)Filters
• The filters were hand picked• Filters:
– The Intensity filter– 4 Difference of Gradient filters
Dx,Dy,Dxx &Dyy– 5 LoG filgers– 24 gabor filters with
• Local & Global Constraints• Using Histograms as features
16,12,5,2T 150,120,90,60,30,0
![Page 24: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/24.jpg)
Spectral Histograms of several images
![Page 25: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/25.jpg)
Algorithm II (SVM) Results
![Page 26: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/26.jpg)
Old Results
![Page 27: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/27.jpg)
![Page 28: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/28.jpg)
![Page 29: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/29.jpg)
![Page 30: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/30.jpg)
![Page 31: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/31.jpg)
![Page 32: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/32.jpg)
![Page 33: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/33.jpg)
Algorithm IIIusing SVM +
rotation invariant features
• Same features as in Alg. II
• The Features enable 180 degrees of rotation invariance
• Rotate the image 180 degrees and switch Histograms achieving 360 degrees invariance
![Page 34: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/34.jpg)
Rotating 180 degrees
![Page 35: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/35.jpg)
Combining the two classifiers
![Page 36: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/36.jpg)
ResultsUpright test sets
![Page 37: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/37.jpg)
ResultsRotated test sets
![Page 38: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/38.jpg)
![Page 39: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/39.jpg)
![Page 40: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/40.jpg)
Rotation Invariation Results
![Page 41: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/41.jpg)
More pictures
![Page 42: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/42.jpg)
![Page 43: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/43.jpg)
![Page 44: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/44.jpg)
Conclusions
• A system which is rotation & translation invariant
• Achieves very high accuracy for frontal faces and rotated frontal faces
• The system is not real time, but is possible to implement convolution in hardware
• Uses limited amount of data
• Accuracy as a function of efficiency
![Page 45: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/45.jpg)
Conclusions (2)
• Faces are identifiable through local spatial dependencies where the global ones can be globally modeled as histograms
• The problem with spatial methods is the estimation of the parameters
• The SH representation is independent of classifier choice
• SVM outperforms Neural Networks• The Problems and the Errors of this system are
considerably different than of other systems
![Page 46: Face Detection using the Spectral Histogram representation By: Christopher Waring, Xiuwen Liu Department of Computer Science Florida State University Presented](https://reader030.vdocuments.us/reader030/viewer/2022033106/5697bfe71a28abf838cb5e09/html5/thumbnails/46.jpg)
Conclusions (3)
• Localization in Space and Scale is not as good as other methods
• Translation Invariant features can enable a coarser sampling the image
• Use adaptive thresholding
• Use several scales to improve performance
• SH can be used for sampling of objects