bxgrid: a data repository and computing grid for biometrics research hoang bui university of notre...
TRANSCRIPT
BXGrid: A Data Repository and Computing Grid for Biometrics
ResearchHoang Bui
University of Notre Dame
1
Biometric Research
• Now what?– I have collected 100,000 irises.– I have an algorithm to compare 2 irises
– I want evaluate my algorithm by comparing only brown irises
– First, I need to convert raw iris images to iris codes
– But I need to find all brown irises
6
BXGrid
How do I search for brown irises fast?
Where do I store iris images?
How do I evaluate my algorithm?
DBMS
Relational Database (2x)
Active Storage Cluster (16x)
CPU
Relational Database
CPU CPU CPU
CPU CPU CPU CPU
Condor Pool (500x)
7
Workflow Abstractions
B1
B2
B3
A1 A2 A3
F F F
F
F F
F F
F
L brown
L blue
R brown
R brown
S1
S2
S3
eye color
F
F
F
ROCCurve
S = Select( color=“brown” )
B = Transform( S,F )
M = AllPairs( A, B, F )
Bui, Thomas, Kelly, Lyon, Flynn, ThainBXGrid: A Repository and Experimental Abstraction… poster at IEEE eScience 200813
Transform Abstraction• B = Transform( S,F )• Transform set S into set B using function F
• Single PC and 100,000 iris images– Core 2 Duo 1.8Ghz 1GB RAM PC– 6 seconds/transform 170 hours– Storage: 30GB• Let’s use Condor• You want to:– Do it faster– Manage resource properly
14
Transform Summary
• Use up to 1GB local storage• Transform 10,000 irises– Single PC: 60,000 seconds– Condor: 1400 seconds
• Speedup: ~43 times
19
AllPairs AbstractionAllPairs( set A, set B, function F )
returns matrix M whereM[i][j] = F( A[i], B[j] ) for all i,j
B1
B2
B3
A1 A2 A3
F F F
A1A1An
B1B1Bn
F
AllPairs(A,B,F)F
F F
F F
F20
AllPairs Result
• 10,000 irises vs. 10,000 irises• Condor pool: 32 nodes• AllPairs took 150 minutes to complete 100,000,000 comparisons
• Speedup: ~ 7 times
21
Workflow Summary
23
Transform AllPairsB1
B2
B3
A1 A2 A3
F F F
F
F F
F F
F
Condor Condor
Iris Iris Code
Result Matrix
Storage Cluster
Future Works
• Run bigger Transform & All-Pairs experiments• Using Condor to perform Automated Validation
• Extend the repository for other types of data
24
Acknowledgments
• Cooperative Computing Lab– http://www.cse.nd.edu/~ccl
• BXGrid– http://bxgrid.cse.nd.edu
Grad StudentsGrad Students– Chris MorettiChris Moretti– Li YuLi Yu– Deborah ThomasDeborah Thomas– Karen HollingswortKaren Hollingswort– Tanya PetersTanya Peters
Faculty:Faculty:– Douglas ThainDouglas Thain– Patrick FlynnPatrick Flynn
Undergrads & StaffUndergrads & Staff– Mike KellyMike Kelly– Rory CarmichaelRory Carmichael– Mark PasquierMark Pasquier– Christopher LyonChristopher Lyon– Diane WrightDiane Wright
26