skin detection for shrp2 face video - virginia tech

Skin Detection for SHRP2 Face Video

ABHIJIT SARKAR A. LYNN ABBOTT ZACHARY DOERZAPH

9/19/2016 1

Motivation

o Remote measurement of heart rate from face video

Wu et al. SIGGRAPH 2012

9/19/2016 2

Motivation

o Driver attributes– Soft biometrics– Wearables

o Driver behavior analysis– Head pose– Emotion analysis

o Face Segmentation and trackingo Limb detection – Activity

recognitiono Face de-identification

9/19/2016 3

Motivationo Traditional skin detectors depends on color cues.o SHRP2 data does not have color information.o NextGen - Night time NIR video is grayscale.

9/19/2016 4

Motivation

o Presence of color

o Appearance of human skin varies for different intrinsic and extrinsic factors

o Human observers are adept at using textural and contextual cues

o We need a universal skin detector

o Image based skin detection

o Use contextual information

9/19/2016 6

Grayscale skin detection

o Learn local statistics to understand global skin characteristics.

9/19/2016 7

Face detection of grayscale image

Skin MASK Non-skin map Distance based probability map

9/19/2016 8

Grayscale skin detection

( ) ( )|s

n IP I s

N=

( ) ( ) ( )

( )

| |

face

gs dist

P s I P I s P s

W W s

α

αλ

∝

∝ >

( )|gsW P I s∝( )dist faceW P s∝

( )|P s I

9/19/20169

Experiments and results

o We have used SFA dataset Dataset comprises of images from

different age, sex, skin tone and facial hair and accessories.

Tested on 1000 images Face detector succeeded for 890 images Original color images are converted to

grayscale

Face skin detection: Indoor

9/19/2016 10

Face skin detection: Drivers

o We have tested for different head pose

9/19/2016 11

Face skin detection: Drivers

9/19/2016 12

Region Growingo What about face segmentation and limb detection?o We use a region growing algorithm that uses belief

propagation from one Superpixel to its neighbor.

9/19/2016 13

9/19/2016 14

Challenges

9/19/2016 15

Challengeso Confusion with

background, hair coloro Failure of face detectoro Illumination gradient

Conclusion

9/19/2016 16

Contributionso We have developed a standalone, universal skin

detection algorithmo First to our knowledge for grayscale images.o Useful for driver monitoring and attribute

detection

Future Directiono Interactive skin segmentationo Iterative method to improve face detection

Acknowledgements

9/19/2016 17

o Review team – NDRS 2016o Surface Transport Safety Center for Excellence

(STSCE)o Virginia Tech Transportation Institute

Image source - http://i.telegraph.co.uk/multimedia/archive/01398/young-driver_1398460c.jpg

9/19/2016 18

Questions ?