ibm smart surveillance system - wocc yingli tian.pdf · ibm smart surveillance system yingli tian...

IBM Research

© IBM Corporation

IBM Smart Surveillance System

Yingli Tian

Exploratory Computer Vision Group

IBM T. J. Watson Research Center

Yorktown Heights, NY

http://www.research.ibm.com/peoplevision

IBM Research

IBM Smart Surveillance System © IBM Corporation

IBM Smart Surveillance System (S3-R1)The world’s first, event based, distributed smart surveillance systemwww.research.ibm.com/peoplevision

Video camera based surveillance systems are key to acquiring intelligence in urban combat zones. Current systems provide raw image data leaving the intelligence gathering task to humans. Human monitoring of video is known to be ineffectual from an intelligence gathering perspective. The first generation of smart surveillance systems were limited to providing real time alarms to “pre-programmed” events. While real-time alarms are useful, the ability to search through event data and understand patterns of activity enables the adoption of new “security strategies & postures” in challenging urban environments. The IBM S3-R1 system is a platform for event-based surveillance. S3-R1 provides the following capabilitiesReal Time Alerts including Motion Detection, Directional Motion, Abandoned Object, Object Removal & Camera Move/Blind: .Event Search using Object Type, Size, Speed, Location, ColorAutomatic Face Capture & Statistical Event Tracking:

Rea

l Tim

e A

lert,

Eve

nt S

earc

h&

Eve

nt S

tatis

tics

Ser

vice

s

Real-Time Alerts

XML Metadata

Video StreamsEvent SearchEvent Statistics

Video Analytics

IBM DB2Event Database

Video Encoder

Cameras

Command Center

IBM Smart Surveillance System (S3-R1)

VideoArchiveManager

Contact: Dr. Arun HampapurIBM T. J. Watson Research Center19 Skyline Drive, Hawthorne, NY 10532 (914) 784-7440, [email protected]

Application Scenarios While being able to provide real time alarms to “pre-programmed” events is useful, the ability to search through event data and understand patterns enables new “security strategies”.

Force Protection: Understanding long term patterns of activity and the ability to search through events can help identify potential vulnerabilities.

Border Monitoring: While real-time alarms are useful, reacting to a large number of real time alarms is infeasible. Understanding patterns of violations for effective deployment ofpatrols is critical.

Large Scale Investigations: In large scale investigations like the Washington Sniper incident, the ability to sift through large numbers of surveillance video tapes to identify commonality is essential.

��

��

��

� �� !" ��#

$��

" ��%�& �'

��

��

��

� �� !" ��#

$��

" ��%�& �'

$��

" ��%�& �'

IBM Research


S3-R1 Pilot at IBM Research Facility in Hawthorne, NY

$��

%��(��

��$��

��

��

��

��

��

��!��"��#��

��!��"��

IBM Research


Real-Time Alerts

� Person / vehicle in area� Wrong direction motion � Crowd / loiter detection� Abandoned object� Vehicle arrival / departure� High speed objects� Camera move / blind� Removed object� Virtual Perimeter Monitoring

IBM Research


S3-R1 Query Result: Find People Events

IBM Research


S3-R1 Query Result: Find Car Events

IBM Research


S3-R1 Query Result with linked Video Player

IBM Research


Face Images Captured by SSE Face Engine

IBM Research


Long Term Monitoring Results – for 24 hours

Office

200 sq. ft.

Office

1875 sq. ft.

Stairs Downto Building

PedestrianXing

Parking Spots

Parking Spots

Stairs toUpper

Parking Lot

Building

MainEntrance

Cameramounted on

roof ofbuilding

CameraCoverageArea

Road Road

Lamps

TypicalPedestrianPath

TypicalVehiclePath

Hawthorne1 Deployment Scenario for Camera #2

Arrival & Departure Distribution of people into Hawthorne 1

Paths of Large Objects over a 24 hour period

Tracks of all objects over a 24 hours

0

10

20

30

40

50

60

70

80

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76

Series1

Arrival Peak:

~ 8.30 am

Departure Peak: ~ 3.30 pm & 5.30pm

IBM Research


Applications of Smart Surveillance

� Security Applications�Airports/Critical Facility

�Borders.

�Embassies & Bases

�Enterprise Security

�Schools� Retail Applications

�Banking

�Loss Prevention

�CRM – people counting, demographics, shopping patterns

� Manufacturing�Safety Applications.

�Operations – Time & Motion.

� Transportation�Safety – railways.

�Congestion monitoring� Health Care.

�Elder Care facilities.

�Home Monitoring.� Sports

�Player tracking.� Gaming

�Casinos

IBM Research


Casino GamingApplication

Casino Gaming

Homeland Security

Application

Homeland Security

Facility ProtectionApplication

Critical Facility Protection

Biz IntelligenceApplication

Business Intelligence

Alert Archive Retrieval Intg. Mgr

Smart Surveillance System-Release 1 (S3-R1) Middleware/Framework

SSE Ingest

DB2-CM, Websphere

GPFS, IBM Blade Servers

Smart Surveillance Middleware

Core Video Analytics(Detection, Tracking, Classifications, Activity Analysis)

IBM Research


Camera

Database Server

Video Manager

ApplicationServer

Video Encoder

MILS

APP2

(Websphere)

(IBM DB2)

(IBM DB2 CM)APP1

tracksSSE

(Smart clips)

start/stop

MP

EG

4

Window

s Media

Verint E

ncoder

Object D

etection

Object Tracking

Object C

lassification

Activity A

nalysis

Speeding A

lert

Vehicle A

lert

Loitering Alert

Other A

lerts

videos

Open Architecture for Smart Surveillance

IBM Research


Distributed Smart SurveillanceS

SE

SS

ES

SE

SS

ES

SE

SS

ES

SE

SS

E

Metadata

Video Server

Application S

erver

Network

SSESSE SSESSE SSESSE SSESSE

Metadata Video Server

Application Server

Network

WebWeb

Centralized Management

Monitoring & Queries

New York Ports

LA Ports

IBM Research


Open Architecture for Sensor Analytics COTS Data

ManagementTechnology

(IBM’s RAMMP)

Ope

n S

enso

r-E

vent

Dat

aIn

gest

Mid

dlew

areOpen Platform for

Sensor Analytics

Ana

lytic

s #

1

Ana

lytic

s #

2

Ana

lytic

s #

N

ProprietaryAnalytics

Application

Ope

n C

omm

and

and

Con

trol

Mid

dlew

are

ProprietaryControl

Application

ProprietaryControl

Application

Sensor EventMeta-data

Store

MediaStore

SystemMeta-data

Store

Pro

prie

tary

Use

r A

pplic

atio

n #2

Pro

prie

tary

Use

r A

pplic

atio

n #1

Sea

rch

and

Ret

rieva

l Ser

vice

sR

eal-t

ime

Ale

rtS

ervi

ces

Com

man

d an

dC

ontr

ol S

ervi

ces

Open InterfacesXML /

Webservices

IBM Research


System Requirements for S3-R1

� Analytics Server– handles 4 video streams (320x240)� Single Processor 2.8Ghz+ Intel Pentium

� 1 GB Ram, 160GB Hard disk

� 4 Channel Frame Grabber Card—Osprey � Database / Analytics Server – handles up to 24 cameras

� Dual Processor System 2.8GHz+ Pentium

� 2GB Ram, 160GB Hard disk� Video Encoding and Storage.

� Partner Systems.� Ongoing Scalability Work

� Re-architect SSE and port to DSP’s

� Backend scalability testing

IBM Research


Competitive Analysis

-+Advanced Analytics

--++Backend Scalability

==Performance (Speed)

==Accuracy

--++System Architecture

++--User Interfaces

+++

=

IBM Research

--

=

Other Vendors

Event Data Management

Video Analytics

Dimension

+ Advantage

= Similar Capabilities

- Behind

ibm smart surveillance system - wocc yingli tian.pdf · ibm smart surveillance system yingli tian...

Documents