![Page 1: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/1.jpg)
BIOMETRICS LABBiometric Standards, Performance and Assurance LaboratoryDepartment of Technology, Leadership and Innovation
STEPHEN ELLIOTTHFES PRESENTATION – MARCH 29. 2012
EVOLUTION OF THE HUMAN BIOMETRICS SENSOR INTERACTION MODEL
![Page 2: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/2.jpg)
CONTRIBUTORS TO THE PRESENTATION
• Michael Brockly• Kevin O’Connor• Carl Dunkelberger• Tyler Veegh• Thomas Cimino• Rob Pingry• Brent Shuler
• Jacob Hasslegren• Rob Larsen• Chris Clouser• Craig Hebda• Weng Kwong
Chan
![Page 3: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/3.jpg)
AGENDA
• What are biometrics?• The missing pieces of biometric testing• The HBSI model• Evolution of the model • Summer 2012 meetings
![Page 4: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/4.jpg)
BIOMETRICS LABBiometric Standards, Performance and Assurance LaboratoryDepartment of Technology, Leadership and Innovation
WHAT IS MISSING IN BIOMETRIC TESTING
![Page 5: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/5.jpg)
BIOMETRIC TESTING
• Traditional biometric testing– Algorithm testing
• Well established metrics• Well understood testing methodologies
• Operational testing– Harder to do
• Access to environments• Test methodologies dependent in some cases on
the test
![Page 6: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/6.jpg)
TESTING AND EVALUATION
• Essentially trying to understand how a system performs– Maybe more fundamentally – who or what is
causing the errors
![Page 7: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/7.jpg)
GAPS IN BIOMETRIC TESTING
• There have been several papers on the contribution of individual error on performance– What causes these errors?
• There are some papers that examine meta-data and the contribution of variables (age) and examines the training of algorithms
![Page 8: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/8.jpg)
GAPS IN BIOMETRIC TESTING
• Training– How do users get accustomed to devices– Can they remember how to use them– How do we provide good training to the
users that has a consistent message
![Page 9: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/9.jpg)
GAPS IN BIOMETRIC TESTING
• Accessibility– How many people know people (!) who have
problems with interacting with a biometric systems
– How do we deal with accessibility and usability issues
• Hearing and sight issues
![Page 10: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/10.jpg)
GAPS IN BIOMETRIC TESTING
• Human Factors– Testing and evaluating biometric systems by
looking at how the users interact with the system
• Are performance results different in an operational environment than collect in a lab?
• Are these performance results due to the environment?
![Page 11: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/11.jpg)
GAPS IN BIOMETRIC TESTING
• It the error always subject centric?– The role of the device?– The role of the operator?
![Page 12: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/12.jpg)
BIOMETRICS LABBiometric Standards, Performance and Assurance LaboratoryDepartment of Technology, Leadership and Innovation
OVERVIEW OF HBSI MODEL
![Page 13: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/13.jpg)
DEVELOPMENT OF THE MODEL
• The HBSI model is concerned with the data collection portion of the biometric model– Consistent and repeatable presentation to
the sensor
![Page 14: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/14.jpg)
HBSI MODEL
Conceptual model for HBSI
Human
Biometric SystemSensor
Ergonomics
Usability
Image Quality
Human-BiometricSensor Interaction (HBSI)
![Page 15: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/15.jpg)
UNDERLYING MODEL
![Page 16: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/16.jpg)
MODALITY TESTING AND HBSI
Year Hand Finger Iris Face DSV
2004 Age Mobile iris Illumination Different devices
2005 Co-Rec
2006 Height /Placement
2007 Habituation Force
2008 Gender
2009 Initial HBSI Calc Force Training
2010 Fixed iris
2011 Gender Device (different sensors)
2012 Hand alignment ForceFinger interactions /
Kinect
HBSITraining /
Kinect
Detractors Forgery
2012 Interaction Age Interaction Age Interaction Age Interaction Age Interaction Age
![Page 17: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/17.jpg)
MODEL DEVELOPMENT - V1
![Page 18: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/18.jpg)
MODEL DEVELOPMENT - V2
![Page 19: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/19.jpg)
INCLUSION OF OTHER MODELS
• General Biometric Model• Operation Times Model (Lazarick,
Kukula, et.al)
![Page 20: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/20.jpg)
Fuji, et. al (2011)
![Page 21: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/21.jpg)
HBSI METRICS V2
Metrics created and validated for:• Iris• Fingerprint (different sensors)• Signature Verification
Record the environment (video
and sometimes audio) from different
angles in order to watch the subject
and to classify their presentation
-typically 3 video angles and operator
screen
![Page 22: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/22.jpg)
BIOMETRICS LABBiometric Standards, Performance and Assurance LaboratoryDepartment of Technology, Leadership and Innovation
HBSI MODEL 3.0
![Page 23: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/23.jpg)
UNDERLYING MODEL EXAMPLES
More actors:• Subject (typically
the biometric donor)• Operator • Other people in the
environment
![Page 24: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/24.jpg)
UNDERLYING MODEL EXAMPLES
Additional variables: Subject conditions• moisture, • elasticity, • oilinessCollected conditions such as:•
temperature • humidity
![Page 25: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/25.jpg)
DETERMINATION OF ERRORS V1
• Process:– Recorded in real time as the study is
underway– Interactions are coded– Metrics of the evaluation model are
completed– Interaction errors are classified as HBSI
terms
![Page 26: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/26.jpg)
BIOMETRICS LABBiometric Standards, Performance and Assurance LaboratoryDepartment of Technology, Leadership and Innovation
KINECT AND THE HBSI MODEL V.3
CRAIG HEBDA | ROB PINGRY | WENG KWONG CHAN |BRENT SHULER
![Page 27: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/27.jpg)
MOTIVATION
• Video coding is time consuming• Inter-rater reliability
– Requires good robust definitions
![Page 28: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/28.jpg)
DEFINING MOVEMENTJake Hasselgren (2011)
Slouched: Subject is not standing up straight during fingerprint scan.
Head Movement: Subject’s head is not still during fingerprint scan.
Body Movement: Subject’s body is not still during fingerprint scan.
Upright: Subject is standing up straight during fingerprint scan.
Labored Walking: Subject has bag or other item on shoulder when approaching device
Pivoting Palm: Subject’s hand pivots on edge of device
Rocking Fingers: Subjects fingers rock from one finger to the next when hand is placed on device
Slapping Hand: Subject slaps hand on to the device
Angled Fingers: Subjects fingers are at an angle other then 90 degrees from edge of the device
![Page 29: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/29.jpg)
MICROSOFT ® SDK INTERFACE
![Page 30: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/30.jpg)
MICROSOFT ® KINECT™ SKELETAL TRACKING SYSTEM
Source: http://msdn.microsoft.com/en-us/library/hh438998.aspx
Source: http://www.genbetadev.com/herramientas/disponible-el-sdk-de-kinect-para-desarrollar-nuestras-propias-aplicaciones-usando-los-sensores
![Page 31: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/31.jpg)
SLOUCHING
• Dictionary Definition• Slouching-A gait or posture characterized by an
ungainly stooping of the head and shoulders or excessive relaxation of body muscles.
• Points of interest:• -Shoulders• -Head• -Spine• -Hips
Source:http://www.merriam-webster.com/dictionary/slouch
![Page 32: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/32.jpg)
SLOUCHING
• Tracking Points to be used:• Shoulder_Right• Shoulder_Left• Shoulder_Center• Head• Spine• Hip_Center• Hip_Right• Hip_Left
![Page 33: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/33.jpg)
SLOUCHING
• Left Slouching:• Left shoulder will be lower then
the right shoulder. • All points on left arm will be
lower then base image.• Head will be tilted to left.• Left hip will be lower then right
hip.• Spine point will move slightly
up and right.
Highlight what is slouchingBreak it into left right
=Movement Up =Movement Down
![Page 34: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/34.jpg)
SOLUTIONS TO THE CHALLENGES OF DEFINING
• A multi point approach can help solve the majority of the problems when describing what is slouching.
• Use a combination of how much each point moves to determine if the subject is slouching or just moving one part of their body.
![Page 35: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/35.jpg)
HEAD DISPLACEMENT
• Definition of Head Movement:• Voluntary or involuntary motion of head
that may be relative to or independent of body.
• http://www.medical-dictionary.cc/what-does/head-movement-mean
![Page 36: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/36.jpg)
HEAD DISPLACEMENT
• Critical Tracking Points (TPs):
1. Head (H)
2. Shoulder_Center (SC)
• Associated Tracking Points (TPs):
1. Shoulder_Right (SR)
2. Shoulder_Left (SL)
![Page 37: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/37.jpg)
DEFINITION OF HEAD MOVEMENT BASED ON TRACKING POINTSHead Movements Tracking Points Definition Changes in Coordinates
Lowering head H approaches SC X, Y, maybe Z too
Nodding H moves back and forth from SC repeatedly X, Y, maybe Z too
Head turning Turn to the left: H moved to the left X, Y, Z
Turn to the right: H moved to the right
Head tilted to one side Tilt to the left: H moved to the left X, Y
Tilt to the right: H moved to the right
Head bobbing H moves in random direction with minimal distance X, Y, Z
Head sliding forward H moves forward Z
Head wagging H moves in left and right rapidly X, Y
Source: http://www.thefreedictionary.com/Head+Movements
![Page 38: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/38.jpg)
HOW WILL THIS WORK?A QUICK ROADMAP
Observation of error
Automatic identification
and classification of
error
Feedback to the user,
customized to their interaction
error
![Page 39: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/39.jpg)
ACTIVITIES
• Link behavior and interaction to the image
• Understand the basic performance characteristics
• Relay back whether the interaction (or change in interaction) affects performance
![Page 40: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/40.jpg)
The benefit is to examine the information associated with the sample, but also the video interaction of the image.
HBSI V3 has (will have):• video and audio
interaction– Watch the interaction– Understand who is
contributing the error– Replay the interaction in
real time as it was collected
• Metadata collected and searchable
![Page 41: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/41.jpg)
QUESTIONS?
• Other actors• Contribution of the operator to the error• Contribution of the test administrator to the error
• HBSI workflow• Semi-automatic coding of the model using Kinect
• New work • Accessibility study – hearing and sight impaired (started Jan 2012)
• Contribution of cost to the model (started Jan 2011)• Examining the role of the impostor (thinking …. As this model only
has been rested in a “genuine” environment)• Development of products that can help improve interactions
Get Involved in shaping these projects – contact [email protected] to participate in the development of the model
Teleconferences over the summer 2012 period
![Page 42: (2012) Human Factors and Ergonomics Society Presentation at Purdue University](https://reader035.vdocuments.us/reader035/viewer/2022062514/558b06c2d8b42a850f8b474e/html5/thumbnails/42.jpg)
KEEP UPDATED
• Scan the QR code with your phone or follow this link for more information:
• http://eepurl.com/kxG-r