fusion of face and iris biometrics from a stand-off video sensor ryan connaughton kevin w. bowyer...
TRANSCRIPT
![Page 1: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/1.jpg)
Fusion of Face and Iris Biometrics from a Stand-Off
Video Sensor
Ryan ConnaughtonKevin W. Bowyer
Patrick Flynn
April 16, 2011Computer Vision Research Lab
Department of Computer Science & EngineeringUniversity of Notre Dame
![Page 2: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/2.jpg)
Biometrics and Multi-Biometrics
BiometricTrait
Sensor MatcherBiometricSample Output
Multi-Modal Multi-Sensor
Multi-Sample
Multi-Algorithm
2
Redundancy at any stage is referred to as multi-biometrics
![Page 3: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/3.jpg)
Fusion in Multi-Biometrics
Fusion: Combining information from multiple sources
Types of fusion:
– Signal Level
– Feature Level
– Score Level
– Rank Level
– Decision Level
33
![Page 4: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/4.jpg)
Advantages and Disadvantages
Potential advantages of multi-biometrics:
– Increased recognition accuracy
– Wider population coverage & lower failure-to-acquire rates
– More difficult to spoof
Potential disadvantages:
– Increased computation time
– Increased acquisition time
– Increased sensor cost
4
![Page 5: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/5.jpg)
Project Goal
Investigate the feasibility of multi-biometrics based on a single sensor
Specifically, combine multi-sample and multi-modal elements to create a system based on face and iris biometrics
Compare performance of multi-biometric approach to single biometric approach
5
![Page 6: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/6.jpg)
Sensors – Iris on the Move (IOM)
Developed by Sarnoff Corp. [1]
Designed for Iris recognition
Stand-off and on-the-move
Array of 3 frontal video cameras
– Each frame is 2048 x 2048 px
– Average iris diameter is ~120 px
Synchronized NIR illumination
Image from K. W. Bowyer, K. Hollingsworth, and P. J. Flynn. Image understanding for iris biometrics: A survey. In Computer Vision and Image Understanding, volume 110, pages 281-307. 2008.
6
![Page 7: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/7.jpg)
IOM Frame Example
7
![Page 8: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/8.jpg)
Sensors – LG IrisAccess 4000 (LG-4000)
Developed by LG Iris [2]
High-quality iris sensor
Short-range, stationary subjects
Average iris diameter is ~250 px
Image from LG Iris Products and Solutions, 2010. URL http://www.lgiris.com/ps/products/irisaccess4000.htm
8
![Page 9: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/9.jpg)
LG-4000 Image Example
9
![Page 10: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/10.jpg)
Diagram of Approach
10
![Page 11: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/11.jpg)
Preprocessing
Stitch and perform histogram matching between corresponding frames
Use template matching to determine translation required to align frames
11
![Page 12: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/12.jpg)
Face Detection
Performed on stitched frames
OpenCV version Viola-Jones face detector used [3],[4]
– Trained on whole faces
Faces are cropped according to face detector's estimation of size and location
12
![Page 13: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/13.jpg)
Eye Detection
Used for iris biometrics and for alignment during face matching
Performed in two phases
– Phase 1: Detect eyes in upper quadrants of previously detected faces
– Phase 2: Detect eyes in frames where no faces were found
Both phases use template matching approach to search for specular highlights
13
![Page 14: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/14.jpg)
Face and Iris Matcher
Face Matcher
– Colorado State University's implementation of eigenface [5],[6]
– Mahalanobis Cosine: -1 to 1, -1 is perfect match
Iris Matcher
– Modified version of Daugman's algorithm [7]
– Normalized Hamming Distance: 0 to 1.0, 0 is perfect match
14
![Page 15: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/15.jpg)
Fusion Summary
Multi-modal and multi-sample scenario
Test and compare multiple fusion approaches
– Score-level
– Rank-level
Three approaches:
– Min rule
– Borda count
– Sum rule
15
![Page 16: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/16.jpg)
Min Fusion
Multi-sample, uni-modal, score-level fusion
MinIris = Min{ Ii,j | i=1...n, j=1...G }
MinFace = Min { Fi,j | i=1...m, j=1...G }
Ii,j = HD between i-th probe iris and j-th gallery iris
Fi,j = Mahalanobis distance between i-th probe face and j-th gallery face
n,m = number of irises and faces detected
G = number of gallery subjects
16
![Page 17: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/17.jpg)
Borda Fusion
Multi-sample, multi-modal or uni-modal, rank-level fusion
For each probe biometric sample
– Sort gallery subjects by match score (best to worst)
– Cast votes for the top v-ranked gallery subjects
• BordaLinear: VoteWeightn = v + 2 – n
• BordaExp: VoteWeightn = 2v-n
Gallery subject with the most votes is the best match for that probe video
Three variations: BordaIris, BordaFace, and BordaBoth
17
![Page 18: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/18.jpg)
Sum Fusion
Multi-sample, multi-modal or uni-modal, score-level fusion
Ii,k = HD between i-th probe iris and k-th gallery iris
FNormi,k = Normalized Mahalanobis distance between i-th probe face and k-th gallery face
n,m = number of irises and faces detected
α,β = weights assigned to face and iris modalities
18
![Page 19: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/19.jpg)
Dataset
Collected 1,886 IOM video sets, spanning 363 subjects
– Ranged from 1 to 15 probe videos per subject
Iris gallery consisted of one left eye and one right eye for each subject
– Acquired with an LG-4000
Face gallery consisted of one full face image for each subject
– Manually selected and annotated from stitched IOM frames
– Earliest IOM video with full face available was used to generate gallery image
– Videos used to generate gallery images were not included in probe set
19
![Page 20: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/20.jpg)
Detection Results
20
![Page 21: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/21.jpg)
Face Matching Results
Mean match score:
-0.281 (σ = 0.213)
Mean non-match score:
0.000 (σ = 0.676)
Independent rank-one:
51.6% (5073/9833)
21
![Page 22: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/22.jpg)
Iris Matching Results
Mean match score:
0.398 (σ = 0.053)
Mean non-match score:
0.449 (σ = 0.013)
Independent rank-one:
46.6% (13556/29112)
22
![Page 23: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/23.jpg)
Rank-One Recognition Rates
23
![Page 24: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/24.jpg)
Comparison Summary
24
![Page 25: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/25.jpg)
Conclusions
Investigated fusion of face and iris biometrics from a single sensor
Conducted multi-modal experiments on a genuine dataset of 1886 videos of 363 subjects
Combined multi-modal and multi-sample biometrics, as well as score-level and rank-level fusion
Implemented the proposed multi-biometric workflow on a stand-off and on-the-move sensor
Thus far, the best tested multi-modal approach yielded an increase of 5.4% in rank-one recognition over uni-modal approach
25
![Page 26: Fusion of Face and Iris Biometrics from a Stand-Off Video Sensor Ryan Connaughton Kevin W. Bowyer Patrick Flynn April 16, 2011 Computer Vision Research](https://reader036.vdocuments.us/reader036/viewer/2022070411/56649f525503460f94c75a80/html5/thumbnails/26.jpg)
Acknowledgments & Questions
Datasets used in this work were acquired under funding from the National Science Foundation under grant CNS01-30839, by the Central Intelligence Agency, and by the Technical Support Working Group under US Army Contract W91CRB-08-C-0093.
Current funding is provided by a grant from the Intelligence Advanced Research Projects Activity.
26
[1] J. Matey, O. Naroditsky, K. Hanna, R. Kolczynski, D. LoIacono, S. Mangru, M. Tinker, T. Zappia, and W. Zhao. Iris on the Move: Acquisition of Images for Iris Recognition in Less Constrained Environments. In Proceedings of the IEEE, volume 94, pages 1936-1947. November 2006.
[2] LG Iris. LG Iris Products and Solutions, 2010. URL http://www.lgiris.com/ps/products/irisaccess4000.htm.
[3] G. Bradski and A. Kaehler. Learning OpenCV. O'Reilly Media, Inc., 2008.
[4] P. Viola and M. Jones. Rapid Object Detection Using a Boosted Cascade of Simple Features. In 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), volume 1, pages 511-518, 2001.
[5] Colorado State University. Evaluation of Face and Recognition Algorithms, 2010. URL http://www.cs.colostate.edu/evalfacerec/algorithms6.html.
[6] M. Turk and A. Pentland. Face Recognition Using Eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 1991), volume 1, pages 586-591, June 1991.
[7] J. Daugman. How Iris Recognition Works. In 2002 International Conference on Image Processing, volume 1, pages 33-36, 2002.