a distributed outdoor video surveillance system for detection of abnormal people trajectories

http://imagelab.ing.unimore.it

ICDSC 2007 – Vienna, Austria – 25-28 Sept. 2007

1

A Distributed Outdoor Video Surveillance System for

Detection of Abnormal People Trajectories

Simone Calderara, Rita Cucchiara, Andrea Prati

Imagelab laboratoryUniversity of Modena and Reggio Emilia, Italy



2

Agenda• Motivations• The IMAGELAB distributed video

surveillance system• Motion detection and tracking• Multi-camera consistent labeling• Probabilistic people trajectory

classification• Conclusions

• Motivations



3

Motivations• Video surveillance: why?

– Increasing security level of public places– Crime Prevention …

• Why automated video surveillance?– Manual monitoring of large areas is difficult– Humans typically focus their attention on particular

spots (some others may not be observed)– Interesting information may be extracted and stored

for subsequent analysis (posterity logging for forensic off-line analysis)



4

Motivations (2)• Distributed multiple cameras mean:

– Wider coverage of the scene– Redundant data (improved accuracy)– Different viewpoints disambiguate groups, help with

occlusions• Distributed multiple cameras - disadvantages:

– System complexity can seriously rise with many cameras

– Using multiple cameras is almost useless if data from different cameras are not correlated

– Need for camera communication/coordination



5

Motivations (3)• Previous generations of automatic surveillance

systems are focused on robustly performing low-level tasks (motion detection, segmentation, perimeter control,…)

• Next generations will need to operate at a higher level of abstraction (infer or learn behavioral patterns, detect specific behaviors, understand what is happening in the scene,…)



6




• Motivations• The IMAGELAB distributed video surveillance

system



IMAGELAB VS system

Sakbot

Ad hoc

Hecol

PPC

Projects and prototypes since 1998



8

Park experimentation

(LAICA Project Reggio Emilia)



9



classification• Experiments• Conclusions


system• Motion detection and tracking



10

Motion detection• SAKBOT based on temporal median + selective

knowledge-based updating• Suitable modifications of SAKBOT [1] system to

deal with outdoor requirements:– Bootstrapping: initial bkg model created using single

difference at block-level (16x16)– Adaptive bkg differencing: hysteresis (local and pixel-

varying) thresholding of the bkg difference

[1] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts and shadows in video streams,” IEEE Trans. on PAMI, vol. 25, no. 10, pp. 1337–1342, Oct. 2003.



11

Motion detection (2)• SAKBOT’s modifications (cont’d):

– Fast “ghost” suppression: similar to bootstrapping (based on single difference), but at region-level: valid object only if a sufficient number of pixels are moving

– Others: object validation, shadow suppression, …



12

Object tracking• Apperance-based tracking approaches keep

track not only of the state but also of the shape at pixel-level, necessary for gait or posture analysis

• AD-HOC (Appearance Driven Human tracking with Occlusion Classification) [2] based not only on object’s position and speed, but also on its appearance map and probability mask

[2] R. Vezzani, C. Grana, R. Cucchiara, “AD-HOC: Appearance Driven Human tracking with Occlusion Classification”, under review in Pattern Recognition, 2007



13





system• Motion detection and tracking• Multi-camera consistent labeling



[3] S. CALDERARA, R. CUCCHIARA, PRATI A. Bayesian-competitive Consistent Labeling for People Surveillance. In press on IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE

Multi-camera consistent labeling• Consistent labeling allows to assign the same

label to different instances of the same person in different cameras

• Consistent labeling can be exploited not only for people tracking on wide scenes, but also for posterity logging; multiple views of the same person exploited to improve retrieval for post-analysis

• HECOL (Homography and Epipolar-based Consistent Labeling): pure geometrical approach [3]



15

The HECOL system



The HECOL system (2)• Off-line process automatically computes

ground-plane homography and epipolar constraints

correct correspondence

• On-line process employs Bayesian-competitive approach with warping of vertical axis and a two-contributions check

a2j

a1jhomography

homography

epipolar geometry

<a1j ,a2

j>FG

• MAP label assignment:



17





system• Motion detection and tracking• Multi-camera consistent labeling• Probabilistic people trajectory classification



Trajectory acquisition



Path modeling• Each trajectory is encoded as a sequence of

directions: • Each trajectory is modeled as a Von Mises

distribution

where I0 is modified zero-order Bessel function of the first kind.

• The Von Mises parameters are learnt through ML estimator:

jjjjT ,3,2,1 ,,

)cos(

00

0

)(21),|(

me

mImp

1

,1

0, 1

,1

sinarctan

cos

j

j

n

i jML ij n

i ji

1

0

( )( )

( )

MLjML

j MLj

I mA m

I m



Learning phase• A training set of trajectories has been collected: 88

trajectories to train the classifier and model the concept of normality

• Aiming at clustering similar trajectories, not similar directions

• Parameters’ space clustered with k-medoids, because it is not Euclidean; specific distance metric based on Bhattacharyya distance

• Analytical and closed-form expression derived for two Von Mises distributions:

2 20 0, 0,

0 0

1( , ) 1 2 cos ( )( ) ( ) p q p q p q

p q

B p q I m m m mI m I m



Clustering trajectories



Clustering trajectories (2)• The medoids constitute the model of normal

behavior and are used to build a multimodal mixture distribution of “normality”:– The final distribution is a mixture of Von Mises

distribution– The components are the Von Mises pdf of each

medoid.– The weights of each component is proportional to the

number of training trajectories that fall into a specific cluster

– The influence of abnormal trajectories acquired during training is smoothed by clustering and mixture component weighting coefficients



Testing phase• Classification of a new path Tj is performed by a

two-steps approach that:– first selects the best candidate model among the

available medoids – and subsequently tests its fitness with the observed

data

First step Second step



Testing phase (2)• Model selection exploiting MAP framework to

maximize the model’s parameters posterior over the observed path directions (a discrete hidden 1-of-K variable Z is introduced)

• After the evaluation, the desired model parameters are selected according to the Zi value that maximizes the posterior:



Testing phase (3)• After the selection of the model, the path is

verified against the model using a first-order Bayesian network:

• Right side decoupled in two contributions, the first coming by the fitness of the variable against the selected model and the second coming from the range of variability with respect to the previous observed value:



Results for path classification• Testing: more than two hours of logging, two

sets (of 121 and 135 trajectories)• Ground truths based on judges of experts. The

experts divided the trajectories into 95 abnormal and 161 normal.

• The classification rate is 100% for abnormal and 97.5% for normal.

• It is important to observe that the system correctly detects all the abnormal trajectories generating only false warnings in the case of normal behavior erroneously classified.



Conclusions• Complete system for distributed video

surveillance presented• Single steps commented and results

shown• Details on high-level trajectory

classification for abnormal path detection• The overall system has been deeply

tested on campus-based controlled scenario

• Preliminary setup on park (uncontrolled) scenario shows promising results



28

Thank you!!



29

a distributed outdoor video surveillance system for detection of abnormal people trajectories

Documents

automated video surveillance

motivationsvideo surveillance

tracking http

camerasusing multiple

system complexity

different cameras

bkg difference1

initial bkg model