Histograms of Oriented Gradients forHuman Detection

Navneet Dalal and Bill Triggs

INRIA Rhone-Alpes

Grenoble, France

Funding: aceMedia, LAVA, Pascal Network

Histograms of Oriented Gradients for Human Detection – p. 1/13

Introduction

Detect & localize upright peoplein static images

ChallengesWide variety of articulated poses

Variable appearance/clothing

Complex backgrounds

Unconstrained illumination

Occlusions, different scales

ApplicationsPedestrian detection for smart cars

Film & media analysis

Visual surveillance

Histograms of Oriented Gradients for Human Detection – p. 2/13

Approach & Data Set

We focus on building robust feature sets

Classifier is just linear SVM on normalizedimage windows, is reliable & fast

Moving window based detector withnon-maximum suppression over scale-space

Data set availablehttp://pascal.inrialpes.fr/data/human/

Dat

aS

et

Train Test

614 positive images 288 positive images

1218 negative images 453 negative images

1208 positive windows 566 positive windows

Overall 1774 human annotations + reflections

Histograms of Oriented Gradients for Human Detection – p. 3/13

Feature Sets

Haar Wavelets + SVM: Papageorgiou & Poggio (2000), Mohan et al(2001), DePoortere et al (2002)

Rectangular differential features + adaBoost: Viola & Jones (2001)

Parts based binary orientation position histograms + adaBoost:Mikolajczyk et al (2004)

Edge templates + nearest neighbor: Gavrila & Philomen (1999)

Dynamic programming: Felzenszwalb & Huttenlocher (2000), Ioffe &Forsyth (1999)

Orientation histograms: c.f. Freeman et al (1996), Lowe (1999)

Other descriptors:- Shape contexts: Belongie et al (2002)- PCA-SIFT: Ke and Sukthankar (2004)

Histograms of Oriented Gradients for Human Detection – p. 4/13

Processing Chain

Orientation Voting

Overlapping Blocks

Local NormalizationInput Image Gradient Image

gradientsCompute into spatial &

Weighted vote

orientation cellscolour

Normalizegamma &

Inputimage

over detectionwindow

Collect HOG’s

SVMLinear

spatial blocksover overlappingContrast normalize

non−personPerson /

classification

Histograms of Oriented Gradients for Human Detection – p. 5/13

HOG Descriptors

Parameters Schemes

Gradient scale

Orientation bins

Percentage of block overlap

RGB or Lab, color/gray-space

Block normalization,L2-norm, v → v/

√

‖v‖2

2+ ε2

orL1-norm, v →

√

v/(‖v‖1 + ε)

R-HOG/SIFT C-HOG

Blo

ck

Cell

Blo

ck

Center Bin

Radial Bins, Angular Bins

Histograms of Oriented Gradients for Human Detection – p. 6/13

Performance

MIT pedestrian database INRIA person database

10−6

10−5

10−4

10−3

10−2

10−1

0.010.02

0.05

0.1

0.2DET − different descriptors on MIT database

false positives per window (FPPW)

mis

s ra

te

Lin. R−HOGLin. C−HOGLin. EC−HOGWaveletPCA−SIFTLin. G−ShaceCLin. E−ShaceCMIT best (part)MIT baseline

10−6

10−5

10−4

10−3

10−2

10−1

0.01

0.02

0.05

0.1

0.2

0.5DET − different descriptors on INRIA database

false positives per window (FPPW)m

iss

rate

Ker. R−HOGLin. R2−HOGLin. R−HOGLin. C−HOGLin. EC−HOGWaveletPCA−SIFTLin. G−ShapeCLin. E−ShapeC

- R/C-HOG give near perfect seperation on MIT database- Have 1-2 orders of magnitude lower false positives than otherdescriptors

Histograms of Oriented Gradients for Human Detection – p. 7/13

Gradient Smoothening & Orientation Bins

Gradient scale, σ Orientation bins, β

10−6

10−5

10−4

10−3

10−2

10−1

0.01

0.02

0.05

0.1

0.2

0.5DET − effect of gradient scale σ

false positives per window (FPPW)

mis

s ra

te

σ=0σ=0.5σ=1σ=2σ=3σ=0, c−cor

10−6

10−5

10−4

10−3

10−2

10−1

0.01

0.02

0.05

0.1

0.2

0.5DET − effect of number of orientation bins β

false positives per window (FPPW)m

iss

rate

bin= 9 (0−180)bin= 6 (0−180)bin= 4 (0−180)bin= 3 (0−180)bin=18 (0−360)bin=12 (0−360)bin= 8 (0−360)bin= 6 (0−360)

Using simple smoothed gradients & many orientations helps!

Reducing gradient scale from 3 to 0 decreases false positives by10 times

Increasing orientation bins from 4 to 9 decreases false positives by10 times

Histograms of Oriented Gradients for Human Detection – p. 8/13

Normalization Method & Block Overlap

Normalization method Block overlap

10−5

10−4

10−3

10−2

0.02

0.05

0.1

0.2DET − effect of normalization methods

false positives per window (FPPW)

mis

s ra

te

L2−HysL2−normL1−SqrtL1−normNo normWindow norm

10−6

10−5

10−4

10−3

10−2

10−1

0.01

0.02

0.05

0.1

0.2

0.5DET − effect of overlap (cell size=8, num cell = 2x2, wt=0)

false positives per window (FPPW)

mis

s ra

te

overlap = 3/4, stride = 4overlap = 1/2, stride = 8overlap = 0, stride =16

Strong local normalization isessential

Overlapping blocks improvesperformance, but descriptor sizeincreases

Histograms of Oriented Gradients for Human Detection – p. 9/13

Effect of Block & Cell Size

4x46x6

8x810x10

12x12

Cell size (pixels)

1x12x2

3x34x4 Block size (Cells)

0

5

10

15

20

Mis

s R

ate

(%)

Trade off between need for local spatial invarianceand need for finer spatial resolution

Histograms of Oriented Gradients for Human Detection – p. 10/13

Descriptor Cues

input image weightedpos wts

weightedneg wts

avg. grad outside in block

The most important cuesare head, shoulder, legsilhouettes

Vertical gradients insidethe person count asnegative

Overlapping blocks thosejust outside the contourare the most important

Histograms of Oriented Gradients for Human Detection – p. 11/13

Conclusions

Fine grained features improve performanceNo gradient smoothning, [−1, 0, 1] derivative mask

Use gradient magnitude (no thresholding)

Orientation voting into fine bins

Spatial voting into coarser bins

Strong local normalization

Overlapping normalization blocks

- A general object classifier- Also works well for other classes- Linear SVM is reliable & fast, but not optimal- Human detection: 90% at 10−4 false positives per window

Histograms of Oriented Gradients for Human Detection – p. 12/13

Demo

No temporal smoothning

Histograms of Oriented Gradients for Human Detection – p. 13/13