jiu xu, axel beaugendre and satoshi goto computer sciences and convergence information technology...

Real-time Human Tracking by Detection based on HOG and Particle Filter

1

REAL-TIME HUMAN TRACKING BY

DETECTION BASED ON HOG AND

PARTICLE FILTER

Jiu XU, Axel BEAUGENDRE and Satoshi GOTOComputer Sciences and Convergence Information

Technology (ICCIT), 2011 6th International Conference on


2

Outline Introduction Proposed Method

Human DetectionMoving object feature, Hierarchical search algorithm

Human TrackingColor-EdgeTexture histogram, Occlusion Handling

Strategy

Experimental Results Conclusion


3

Introduction The real-time foreground objects

tracking and detection is the most critical and fundamental step in video surveillance systems.

Different from vehicles, it is much more difficult to locate and track the human body out of the background.


4

Introduction

Pedestrian detection by Histogram of Oriented Gradient (HOG) put forward by Dalal [1] [2].

The main idea of this method is to use gradient direction histogram of small pieces to describe the image.

[1] N. Dalal, "Finding People in Images and Videos," PhD thesis, Institut National Polytechnique de Grenoble, 2006.[2] N. Dalal and B. Triggs, "Histogram of oriented gradient for human detection," in CVPR, 2005.

Detection


5

Introduction Feature extraction by adjusting the

parameters, the HOG feature can effectively describe the body’s shape information.

Moreover, it also has the ability of invariance when the small local area occurs dithering and rotation.


6

Introduction However, the computational complexity

of feature extraction makes the method very slow, and difficult to meet the needs of the practical application of the system.


7

Introduction

Some common methods perform tracking by pre-initialized trackers based on Kalman filter [6] or particle filter [7].

Tracking

[6] Gutman, P., Velger, M. “Tracking Targets Using Adaptive Kalman Filtering”, IEEE Transactions on Aerospace and Electronic Systems Vol. 26, No. 5: pp. 691-699 1990.[7] B.Ristic, “Beyond the Kalman Filter: Particle Filters for Tracking Applications”. Arthech House, 2004.


8

Introduction Particle filter is based on the Bayes

principle and is a sequential Monte-Carlo simulation method indicated by probability density of particles.


9

Introduction one of the intractable problems is that

the target is usually occluded by other objects frequently and instantaneously.

To select powerful features to avoid hijacking problems when tracking similar objects.

If several humans walk together as a group, we cannot separate them individually.


10

Introduction The target of this paper is to build a

powerful tracking system for real-time surveillance system.

Combining human detection and tracking and do tracking by detection in order to achieve high accuracy and low time consumption together with occlusion solutions and group segmentations.


11




Strategy



12

Proposed Method

a Tracking by Detection System using human detection together with human tracking


13

Human Detection Histogram of Oriented Gradient (HOG)

is quite a popular method of detecting people in static image.


14

Human Detection The image is first divided into blocks

while these blocks overlap with each other. Each block contains four cells.


15

Human Detection For each pixel I(x,y), the

orientation Θ(x,y) and the magnitude m (x, y) of the gradient


16

Human Detection A histogram is calculated for each cell,

and the volume of each bin is the sum of magnitude of the pixels whose orientations are in the corresponding angle interval.


17

Human Detection However, HOG can’t be used into real-

time system due to its high time consumption.

Improvements Moving object feature Hierarchical search algorithm


18

Human Detection Moving object feature Kim [10] presented a new kind of non-

parametric algorithm for background subtraction.

For each pixel, it builds a codebook consisting of one or more codewords.

[10] K. Kim, T. H. Chalidabhonse, D. Harwood, and L. Davis. “Real-time foreground-background segmentation using codebook model”. Elsevier Real-Time Imaging, vol. 11, no.3, 167–256, June 2005.


19

Human Detection In our previous work [13], a block-based

codebook method has been proposed to make it more adoptable for human tracking.

The block feature heighten the pixel-based method to block level, thus we could take advantage of the relationship between neighboring pixels.

[13] Jiu Xu, Ning Jiang, Satoshi Goto, “Block-based Codebook Model with Oriented-Gradient Feature for Real-time Foreground Detection”, IEEE 13th International Workshop on Multimedia Signal Processing(MMSP), 2011.


20

Human Detection

morphological operation(opening, closing)


21

Human Detection Hierarchical search algorithm The expected target might appear at

any position and the size would also keep changing


22

Human Detection From our experiments, we find that the

curve of the number of detected pedestrians under various scaling satisfied the curve of normal distribution probability density function.

We could estimate the minimum interval [a, b].

Image scale levels only from a to b should be detected, reduce the searching levels and the amount of computation as well.


23

Human Tracking


24

Human Tracking Particle tracker initialization


25

Transition model p()- how objects move between frames. Observation model p()- specifies the likelihood of an object being in a specific state. Initial state p()- describes initial distribution

Human Tracking


26

Human Tracking Color-EdgeTexture histogram We proposed a Color-EdgeTexture

Histogram to generate the weight for the observation models.

We choose the HSV color. The brightness variations while in HSV color space we could better separate the brightness with others.HSV = Hue, Saturation, Value


27

Human Tracking Moreover, we add the edge local binary

pattern to describe the shape texture of the moving objects.

Local binary pattern (LBP) [14] is an effective texture description operator, which could be used to measure and extract texture information from the local neighborhood in a gray image.

[14] T. Ojala, M. Pietikainen, and D. Harwood, “A Comparative Study of Texture Measures with Classification Based on Feature Distributions”, Pattern Recognition, vol. 29, pp. 51-59. 1996


28

Human Tracking Local binary pattern (LBP)Consider a pixel (, )


Human Tracking LBP value together with H-S color

information improve the performance Two problems :1. time consumption. Since LBP is a pixel-

wise coding, if we calculate all the LBP value inside the whole regions of predicted position of the particles, the calculation is really quite huge, thus it will extremely decrease the real-time ability.


Human Tracking2. the LBP value for background part of

the regions within the trackers are also calculated, it will greatly reduce the tracking rate when the size of the object is changing, and the portion of the background becomes larger and larger since the weight of the background is increased.


Human Tracking To solve these problems, we use a

concept of edge LBP and the edge LBP only focuses on the edge points of the foreground objects.

canny edge detector


Human Tracking we use a kind of H-S-

ForegroundEdgeLBP histogram in which the size is 8x8x8 for each component.

H-S-LBP histogram


Human Tracking Particles are weighted according to the

similarity between the target histogram distribution q(u) and the histogram distributions p(u) given by particles.


Human Tracking the weight of the i-th particle is defined

as

color histogram of the target at time k histogram of the i-th particle


35

Human Tracking Occlusion Handling Strategy In traditional particle filter tracking, if

the object meets some partial or total occlusions, the observation model will turn to the occluder and will not track the previous objects any longer.


36

Human Tracking After color-edgetextue histogram, we

define a threshold to this weight.

① If the tracker moves out of the margin of the frame

② If the tracker is still inside the frame and the max weight is great than the threshold

③ If the tracker is still inside the frame and the max weight is less than the threshold

Delete

Update

Keep increase the number of the particles, searching range


37




Strategy



38

Experimental Results

Other method


39


Other method


40


Other method


41


Separate group


42


The time consumption of our method is much less than [8][9][11].

(400 frames long)


43

[8] L.M.Fuentes and S.A.Velastin, “People tracking in surveillance applications”, Image and Vision Computing, pp.1165-1171, 2006

[9] Tao Yang, Quan Pan, Jing Li and Li, S.Z. “Real-time Multiple Objects Tracking with Occlusion Handling in Dynamic Scene” In CVPR, Vol.1. pp. 970-975, 2005

[11] R. Hess and A. Fern, “Discriminatively Trained Particle Filters for Complex Multi-Object Tracking”. In CVPR, 2009.



44

Uses a combination strategy of HOG human detection method and particle filter human tracking algorithm in surveillance system.

Original HOG feature is not be so efficient in real-time system

In order to enhance the performance in color-based particle tracking

The proposed method has a good robustness in all kinds of situations together with low time consumption.

Conclusion


45

Thanks for listening !!

jiu xu, axel beaugendre and satoshi goto computer sciences and convergence information technology...

Documents

tracking targets

tracking applications

hog feature

powerful tracking system

human body

particle filters

particle filterintroductionone

realtime surveillance