Region-based Skin Color Detection

Rudra P K Poudel (Presenter), Hammadi Nait-Charif, Jian Jun ZhangMedia School, Bournemouth University, UK

David LiuSiemens Corporate Research, USA

Outline of the talk

1. Introduction- skin color detection

2. Literature Review

3. Current Problems

4. Region-Based Technique

5. Proposed Region-Based Technique

6. Experimental Results

7. Conclusions

1. Introduction

• Task: separate skin and non-skin

regions (not pixels)

• Motivation: invariant of rotation,

scaling and occlusion

• Problems: illumination, ethnicity

background, make-up, hairstyle,

eyeglasses, background color,

shadows, motion illumination, skin

look like colors, etc.

Source: Harry Potter movie

1.1 Applications

• Hand tracking, face detection, pornography detection, person tracking

• Skin color detection module equally applicable for other color editing, detection etc applications

• Color is used as primary clue in many image processing and computer vision applications

2. Literature Review

2.1 Color space• RGB• HSV• YCbCr• Perceptually uniform color systems (CILLAB, CIELUV,

LAB)• Normalized RGB

2.2 Skin color classifier• Nonparametric methods: histogram, Bayes classifier,

self-organizing map• Parametric methods: single Gaussian, mixture of

Gaussian• Others: neural network

2. Literature ReviewSummary:• Color space: RGB and HSV are two widely used techniques• Classification method: Naïve Bayes classifier and mixture of

Gaussian are widely used techniques

• Gaussian model: need few training data, difficulties on parameter tuning, need less memory space, processing/detection slow

• Bayes theorem: need for larger training data, easy for learning, need more memory space, processing/detection fast

State-of-the-art method:• Jones, M. J. and Rehg, J. M. (2002). Statistical color models with

application to skin detection. International Journal of Computer Vision, 46(1):81–96.

3. Current Problem

• Probability accumulation for higher level vision task- as probability for skin/non-skin vary highly even for adjacent pixels

Naturally skin is continuous region

4 Region Based Approach

• Yang and Ahuja (1998) and Kruppa et al. (2002)- search elliptical regions for face detection

• Sebe Sebe et al.(2004)- 3x3 fixed size patches to train Bayesian network

• Our approach treat skin as region with varying sizes, which is purely based on image evidence

Proposed Technique/Framework

1. Region extraction- quick shift image

segmentation, also know as “superpixels”

2. Region classification- pixel/region based

3. Smoothing- Conditional Random Field (CRF)

5. Proposed Region-Based technique

5.1 Region Extraction

Region/Superpixel extraction – quick shift image segmentation using RGB color and positional (XY) coordinate

5.1 Region Extraction- Region extraction is purely evidence based i.e. based on RGB

color and spatial location (xy-coordinate) of the image

- Regions have different size and shape, which is depend upon complexity of the image

- No explicit concept of boundary

- Quick shift preserve the boundary of the objects, hence we could get very accurate object segmentation

- We could set importance on color difference vs spatial distance

5.2 Region Classification

5.2.1 Basic Skin Color Classifer

Naïve Bayes: posterior likelihood * prior∝

However, we could use any suitable/best method for skin classification

)(

)()/()/(

cp

spscpcsp =

)(

)()/()/(

cp

nspnscpcnsp =

Θ>)/(

)/(

cnsp

csp

Θ>)()/(

)()/(

nspnscp

spscp

1)/(

)/( >nscp

scp Where, c = color, s = skin and ns = non-skin

5.2 Region Classification

• Average the skin probability (s) of all color pixels (c) belongs to the given superpixel (sp)

• Average the non-skin probability (ns) of all color pixels (c) belongs to the given superpixel (sp)

∑=N

iicsPN

spsP )|(1

)|(

∑=N

iicnsP

NspnsP )|(

1)|(

5.3 Smoothing with CRF

• Conditional Random Field (CRF) optimization equation

• Color potential

• Edge and boundary potential

∑∑∈∈

Φ+Ψ−=−Ess

jijiSs

ii

jii

ssccslSLP),(

),|,()|());|(log( ωω

))|(log()|( iiii slPsl =ψ

[ ]jiji

jijiji cc

ss

ssLsscc ≠

−+

=Φ ,||||1

),(),|,(

5.4 Training

First Phase (training histogram):• Train 2 histograms for skin and non-skin separately

Second phase (training CRF): learning :

si sj …px1(s|c)px1(ns|c)

px1(s|c)px1(ns|c)

color difference +

boundary length

∑∑∈∈

Φ+Ψ−=−Ess

jijiSs

ii

jii

ssccslSLP),(

),|,()|());|(log( ωω

ω

6 Experimental Results

• Dataset content 14 thousands images collected freely from

the web (Compaq dataset)

• 4,700 are skin and 9,000 non-skin images

• Approximately 1 billion pixels are manually labeled

• 50% is use for training and 50% for testing

Method True Positive

False Positive

Jones and Rehg (2002) 90% 14.2%

Our (Superpixel only) 91.44% 13.73%

Our (Superpixel and CRF) 91.17% 13.12%

6 Experimental Results

Our proposed new region-based technique outperform current state-of-the–art technique

6 Experimental Results

Applying CRF is always not good !

6 Experimental Results

However, in aggregate CRF performs better!

7. Conclusions• Region-based technique performs better than pixel-based

• Region-based technique could easily incorporate texture info

and other type of features to improve the result

• Aggregation of pixels into region help to reduce local

redundancy.

• Region-based technique extracts larger smooth regions,

which is very helpful for higher-level vision task

The message to take home:

It is better/natural to treat skin as regions instead of

individual pixels!

Thank you !

Questions ???