Suspicious Behavior in Outdoor Video Analysis - Challenges & Complexities

• Air Force Institute of Technology/ROME Air Force Research Lab

• Unclassified IED test sequences showing dropped package from vehicle (DPV)• Combination of motion analysis and change detection

– Homogenous regions and aperture problem and for optic flow approaches– Learning appropriate background for change (ghost objects appear due to slow or fast

learning)– Global camera motion/jitter

• Occlusion and Camouflage• Environmental problems

– Dust and smoke– Wind –local object motion (swaying of branches, shadows) – Precipitation –rain, slow etc. – Clutter (background model)

• Illumination problems– Shadows (static and moving cast shadow) - missed objects or false detections – Glare – false detections, object shape and trajectory distortions– Sudden illumination changes (cloud movements) – false detections – Low contrast or color saturation

Dropped package from vehicle (DPV) sequences - Logitech Orbit 20ft Vertical Run 2

Frame 150: Event of interest marked

Detecting Occluded EventSequence - Logitech Orbit 20ft Vertical Run 3

Frame 121: Event of interest marked

Glare and ShadowsSequence - Logitech Orbit 10ft Vertical Run 2

Frame 43: False detection due to glare Frame 141: Event of interest marked

Dust and ShadowsSequence - Logitech Orbit 10ft Vertical Run 3

Frames 72&170: False detection due to dust

Frame 159: Event of interest marked

Dust and ShadowsSequence - Logitech QuickCamPro 5000 10ft Vertical Run 3

Frame 150: Event of interest missed due to shadow and insufficient contrast

Frame 91: False detections due to dust

Sequence - Logitech QuickCamPro 5000 20ft Vertical Run 2

No event of interest

Sequence - Logitech QuickCamPro 5000 20ft

Vertical Run 3

Frame 104: Event of interest marked

Effect of Learning Rate in Background Modeling Sequence - Logitech Orbit 10ft Vertical Run 2

Frame 86: Correct Detection when in motion

Frame 134: Object that stops for a while blends into the background

Frame 210: Ghost object left behind (due to slow background learning using Mixture ofGaussians) when the car starts to move again

Problems with Flow-based Approaches

Sequence - Logitech QuickCamPro 5000 20ft Vertical Run 1

Frame 24: Aperture problem, motion of homogeneous regions is not detected

Frame 35: Non-moving objects not detected

Frame 83: Larger temporal window results in false detections and larger object boundaries

Suspicious Behavior in Outdoor Video Analysis - Challenges & Complexities

• Combination of motion analysis and change detection– Homogenous regions and aperture problem and for optic flow approaches– Learning appropriate background for change (ghost objects appear due to slow

or fast learning)– Global camera motion/jitter

• Occlusion and Camouflage• Environmental problems

– Dust and smoke– Wind –local object motion (swaying of branches, shadows) – Precipitation –rain, slow etc. – Clutter (background model)

• Illumination problems– Shadows (static and moving cast shadow) - missed objects or false detections – Glare – false detections, object shape and trajectory distortions– Sudden illumination changes (cloud movements) – false detections – Low contrast or color saturation

Moving Object Detection ApproachesOptical Flow Analysis: Characteristics of flow (velocity) vectors of moving

objects over time are used to detect changed regions. Advantage: can be used in the presence of camera motion.Disadvantage: usually computationally expensive & aperture problem.

Change DetectionBackground subtraction: Moving regions are detected through difference

between the current frame and a reference background image. | framei-Backgroundi |>Th

Advantage: provides the most complete feature data.Disadvantage: sensitive to dynamic scene changes due to lighting and

extraneous events and cannot handle global motion.Temporal differencing: Similar to background subtraction but the estimated

background is the previous frame. | framei-framei-1 |>Th

Advantage: very adaptive to dynamic environments.Disadvantage: has problems in extraction of all relevant feature pixels

(aperture problem).