www.kingston.ac.uk/dirc/
Vigilant Real-time storage and
intelligent retrieval of visual surveillance data
Dr Graeme A. Jones
www.kingston.ac.uk/dirc/
Vigilant aims
• to design a database that provides real-time efficient storage of events occurring within a monitored scene.
• to enable untrained security operators to generate human-centric queries for video data i.e. queries based on content.
www.kingston.ac.uk/dirc/
System constraints
• Handle video stream in real-time
• Compress the terabyte of digital video generated per camera per day onto an swappable one gigabyte disk
• Maximum of one high spec PC per camera
• Intuitive button-press query builder
www.kingston.ac.uk/dirc/
System architecture
dB
Event detectionand tracking
Event detectionand tracking
OfflineAnnotation
ColourColour
TrajectoryTrajectory
ClassificationClassification
Fuzzy SearchFuzzy Search
www.kingston.ac.uk/dirc/
System architecture
dB
Event detectionand tracking
Event detectionand tracking
www.kingston.ac.uk/dirc/
Fast Robust Event Detection (and tracking) Algorithm
• Model-based approach exploiting expected projected object size, with shadow/reflection
iiB
iiW
AiiH
)(
)(min
)(
i
H
B
W
www.kingston.ac.uk/dirc/
Sum
Gate
Sum
Z-1
Z-1
Gate
Temporal updating of reference images with soft gating
Blob detection from pixel comparison with mean and variance image
CurrentFrame
EventMask
DifferenceFrame
Threshold
SubtractReference
Map
VarianceMap
FREDA- Low-level
www.kingston.ac.uk/dirc/
FREDA - High LevelRegion detection from
connected components
Unsupported regions generate new objects
EventMask
CC`
RegionList
`Objects`New
Objects
HypothesizeObjects
`Region
List
+
CurrentObjects
ValidateObjects
Z-1
Gate
Current objects validated from supporting regions
www.kingston.ac.uk/dirc/
Example Results
Image Acquisition 11%Blob Detection 32%Updating Background 23%Connected Component 20%Object Validation 12%
100%Total Time (secs) 0.127Frame Rate 8 Hz
www.kingston.ac.uk/dirc/
Information Balance Sheet
Input data
• [800x600] x (1+0.5+0.5) x 25 x (60x60x24)
2Tbytes/day
Output knowledge
• Periodic background + DCT updates
• Subimage sequences (pixels and contour) per temporal event
www.kingston.ac.uk/dirc/
The Intelligent Camera
• Boundary between PC and intelligent camera depends on issues of frame rate, bandwidth and computational resources
Pix
el C
ompa
rsio
n
Con
nect
ed
Com
pone
nts
Hyp
othe
sis
Gen
erat
ion
Obj
ect V
alid
atio
n
Cha
ract
eris
atio
n
www.kingston.ac.uk/dirc/
System architecture
dB
Event detectionand tracking
Event detectionand tracking
OfflineAnnotation
ColourColour
TrajectoryTrajectory
ClassificationClassification
www.kingston.ac.uk/dirc/
Object Classification• Object event may be classified into Person,
Vehicle, Large Vehicle classes based on history of depth-compensated dimensions and speed.
www.kingston.ac.uk/dirc/
Colour Annotation
• Dominant colour(s) of object extracted as modes from colour histogram generated from pixels of temporal event.
Munsell Space. Semantic classification in HSV colour space
HUE
Val
ue
www.kingston.ac.uk/dirc/
3D Trajectory
• Ground plane calibration (learnt) enables 3D speed to be computed, and hence velocity behaviours derived e.g. car maneouvring, person running, etc.
• Trajectory commentary derived from areas of interest previously assigned labels by operator e.g. gate, bikeshed, disabled parking.
www.kingston.ac.uk/dirc/
Fasset Road
Fasset Lane
Zone 1
Zone 2
Zone 3
West Carpark Approach
REGIONS
GATE
CARPARK
GATE
08:34:56 Vehicle 3434 enters gate08:35:03 Vehicle 3434 enters F.Lane08:35:12 Vehicle 3434 enters carpark08:35:31 Vehicle 3434 enters zone308:36:05 Vehicle 3434 stops in zone3
15:55:23 Vehicle 3434 leaves zone3
www.kingston.ac.uk/dirc/
Behaviour analysis
• Hidden Markov Models based on states derived from clusters of positions along training sets of trajectories used to classify object and its behaviour
•Car Entering•Car Leaving•Person Entering•Person Leaving
S1 S2 S3
0.8 0.7
0.05
0.15 0.25
www.kingston.ac.uk/dirc/
System architecture
dB
Event detectionand tracking
Event detectionand tracking
OfflineAnnotation
ColourColour
TrajectoryTrajectory
ClassificationClassification
Fuzzy SearchFuzzy Search
www.kingston.ac.uk/dirc/
Conclusions
• Its easy to derive atomic units of useful user-oriented knowledge
• Requirement for human-oriented query specification tools employing fuzzy matching
• Plug ‘n play characteristics e.g. camera calibration
• Distribution of computer intelligence