Abandoned Object Detection for Indoor Public Surveillance Video

Dept. of Computer ScienceNational Tsing Hua University

Outline Introduction

Common Surveillance Scenarios and Schemes Scenario of Few Pedestrians Scenario of Normal Case Scenario of Rush Hours

Proposed Abandoned Object Detection Scheme

Experimental Results

Conclusions and Future Works

Applications of Video Surveillance Systems

Security Surveillance of housing, public area Detecting or tracking suspicious objects

Behavior analysis Segmentation of the human body Classify the behavior of the human

Scenario Types

Few Pedestrians (Lib)

Normal Case (DingXi Station)

Rush Hours (Taipei Main Station)

Object Presence Frequently

Object Presence Occasionally

Object Presence Rarely

Scenarios of Environments

Scenarios Types

Object Presence Object Detection

Few Few PedestriansPedestrians FrequentFrequent Background Background

SubtractionSubtraction

Normal CaseSomewhat Frequently

Simple Motion Filter

Rush Hours RareAdvanced

Motion Filter

Scenario of Few Pedestrians – Background Subtraction

The reference background Current frame

Scenarios of Environments

Scenarios Types

Object Presence Object Detection

Few Pedestrians

FrequentBackground Subtraction

Normal CaseNormal Case Somewhat Somewhat FrequentlyFrequently

Simple Motion Simple Motion FilterFilter

Rush Hours RareAdvanced

Motion Filter

Scenario of Normal Case- Background Subtraction

The reference background Current frame

Scenario of Normal Case- Most Frequent Intensity

X

Frame Counter

Pixel Intensity

0255Background or

Stationary Objects

Most Frequent Intensity !!

Scenario of Normal Case- Most Frequent Intensity

The reference background The Most Frequent IntensityMost Frequent Intensity Picture

Scenarios of Environments

Scenarios Types

Object Presence Object Detection

Few Pedestrians

FrequentBackground Subtraction

Normal CaseSomewhat Frequently

Simple Motion Filter

Rush HoursRush Hours RareRareAdvanced Advanced

Motion FilterMotion Filter

Proposed Abandoned Object Detection Scheme for Scenario of Rush Hours Pixel-based

MoG

Advanced Motion Filter for Scenario of Rush Hours Using Vertical Scan Line Eliminate the Sparse Background Clutter Extracting the Complete Shape of an Abandoned Object Tracing Through Vertical Scan Lines Controllable System Alarm Response Time Grouping Abandoned Pixels to Objects

A Multi-model Background Modeling Algorithm － Mixture of Gaussian (MoG)

1

frame #

weight

0

x

Background distribution

Observations from vertical scan line

h1

h2

Observations from Vertical Scan Line

h1

h2

Proposed Motion Filter using Vertical Scan Line

Proposed Motion Filter -Eliminate the Sparse Background Clutter

The referenced background Current frame

Proposed Motion Filter -Eliminate the Sparse Background Clutter

Proposed Motion Filter-Extracting the Complete Shape of the Abandoned Object

The referenced background Current frame

Proposed Motion Filter -Extracting the Complete Shape of the Abandoned Object

First foreground point Complete Shape

Proposed Motion Filter -Tracing Through Vertical Scan Lines

x

Stop at first foreground section

Tracing through the next foreground sectionCurrent frame

Proposed Motion Filter-Controllable System Alarm Response Time

Different reasonable response time for different applications

Avoid to issue the alarm for temporally still pedestrians

Proposed Motion Filter -Grouping Abandoned Pixels to Objects

Background Pixel

Abandoned Object Pixel

Constraint: Object size ≥ 4 pixels

One Alarm

Experimental Results-Test Sequences and

Parameters

Sequence Name Total Frames

The Amount of Pedestrians

Abandoned Object is Shot First

Taipei Station Metro

1200 Rush Hours In the 99th Frame

DingXi Metro 1000 Normal Case In the 1st Frame

NTHU Library 1000 Few In the 1st Frame

Experimental Results-Evaluating Parameters

Application-depended Thresholds Eliminate the Sparse Background Clutter

Te Size of an Abandoned Object

Ts Controllable System Alarm Responding Time

Tr Performance Evaluation

Response Time (<25s) Alarms for Abandoned Objects / Total Alarms

↑

Eliminate the Sparse Background Clutter (Taipei Station)

475

685

861

0 200 400 600 800 1000

Te=1

Te=10

Te=20

Number of Frames

Response Time

2

14 14

5

3

7

4

1

3

0

2

4

6

8

10

12

14

16

Abandoned Objects Still Pedestrians False Alarms

Num

ber o

f Ala

rms Te=1

Te=10

Te=20

Alarms Count

<25s 2/(2+14+14)=1/15 5/(5+3+7)=1/3

Size of an Abandoned Object (Taipei Station)

641

685

845

0 100 200 300 400 500 600 700 800 900

Ts=50

Ts=100

Ts=200

Number of Frames

Response Time

78

15

5

3

7

3

0

2

0

2

4

6

8

10

12

14

16

Abandoned Objects Still Pedestrians False AlarmsN

umbe

r of A

larm

s

Ts=50

Ts=100

Ts=200

Alarms Count

7/(7+8+15)=7/30 5/(5+3+7)=1/3<25s

Controllable System Alarm Responding Time (Lib)

78

328

578

0 100 200 300 400 500 600 700

Tr=1

Tr=250

Tr=500

Number of Frames

Response Time

14

4

0

13

0 0

13

0 00

2

4

6

8

10

12

14

16

Abandoned Objects Still Pedestrians False AlarmsN

umbe

r of A

larm

s

Tr=1

Tr=250

Tr=500

Alarms Count

<25s

Experimental Results-Comparisons with Related Works

1

31

76

2

14

118

5 3 7

0

20

40

60

80

100

120

140

Abandoned Objects Still Pedestrians False Alarms

Num

ber o

f Alar

ms

Change DetectQ[11]

Multi- Background[12]

Proposed

[11]

[12]Demo

Experimental Results-Time Complexity Analysis

Process Time

0

40

80

120

160

Complete

Area

Reduce

Area

Skip Pixels Reudce

Area +Skip

Pixels

Mill

iSec

ond

Process Time47.7

Conclusions & Future Works An Abandoned Object Detection

Scheme to Deal with all of the Scenarios of Few Pedestrians, Normal case and Rush hours

Define different method for new scenarios

Object Detection Scheme for adaptive environment (Light changes , outdoor)

Define new interested events