geo-spatial aerial processing for scene understanding and object tracking
DESCRIPTION
Geo-Spatial Aerial Processing for Scene Understanding and Object Tracking. Jiangjian Xiao, Hui Cheng, Feng Han, Harpreet Sawhney. Problem. Given Aerial Video Understand the Scene Find buildings Trees Roads Cars Use understanding Object Detection Tracking Cool Idea - PowerPoint PPT PresentationTRANSCRIPT
Geo-Spatial Aerial Processing for Scene Understanding and Object Tracking
Jiangjian Xiao, Hui Cheng, Feng Han, Harpreet Sawhney
Problem Given Aerial Video Understand the Scene
Find buildings Trees Roads Cars
Use understanding Object Detection Tracking
Cool Idea Trees and buildings are in
3D
Related Work CVPR 2006
Hui Cheng, Darren Butler and Chumki Basu
ViTex: Video To Tex and Its Applications in Aerial Video Survellance.
Related Work CVPR2008
Jake Porway, Kristy Wang, Benjamin Yao, Song Chun ZhuA Hierarchical and Contextual Model for Aerial Image Understanding
System Overview
Input Frames
Geo-reference image
Initial camera location
Geo-registration Pose estimation
Depth estimation
Non-ground object detection
Planar + depth extension for structure detection
Road Detection
GIS
Scene segmentation output
Stage 1
Stage 2
Stage1
Input Frames
Geo-reference image
Initial camera location
Geo-registration Pose estimation
Depth estimation
Stage 1
GeoRegistration
Input Frames
Geo-reference image
Geo-registration
Meta Data
GPS
Aircraft Parameters
Camera Parameters
GeoRegistration
GPS
Aircraft Parameters
Camera Parameters
Frame To Frame transformations
Bundle Adjustment
SIFT matching
Stage1
Input Frames
Geo-reference image
Initial camera location
Geo-registration Pose estimation
Depth estimation
Stage 1
Adjusting camera position Metadata Gives camera position
Along with many other parameters Metadata has error
In all parameters Georegistration overcomes error
Returns a 3x3 homography matrix Want to figure out the exact camera position
Adjusting camera position
Ground Point
Image Point
Project Ground point to image
Adjusting camera position
Alternatively the point can be projected using homography obtained from georegistration
Get rid of translation parameters
Adjusting camera position
Extract rotation and calibration parameters using SVD
smooth
and Using Kalman filter
Use refined
and to estimate translation parameters
Stage1
Input Frames
Geo-reference image
Initial camera location
Geo-registration Pose estimation
Depth estimation
Stage 1
Depth Estimation Use graphcuts to estimate depth
A difficult task due to poor image quality, and unconstrained motion
Solution Fuse depthmaps
Project several depthmaps unto the DOQ Take their average Smooth out the average map
Depth is quantized along Z direction
Depth Estimation
Stage 2
Non-ground object detection
Planar + depth extension for structure detection
Road Detection
GIS
Scene segmentation output
Stage 2
Detect Non-Ground Regions
Threshold Depth Map
Stage 2
Non-ground object detection
Planar + depth extension for structure detection
Road Detection
GIS
Scene segmentation output
Stage 2
Detect Roofs
Threshold Depth Map Fit Plane
Remove Trees
“Roof” Refinement Fit a plane to the detected “roofs”. We have a set of x,y,z points Want to fit
“Roof” refinement
Z
Y
Z
z
u
v
Depth Along u
Must be invariant
Building Detection
Extend Roof To GroundGives Building height
Tree Detector Classify each pixel as tree non-tree 9D Gaussian Mixture
Color, Depth, Texture Supervised offline training
Stage 2
Non-ground object detection
Planar + depth extension for structure detection
Road Detection
GIS
Scene segmentation output
Stage 2
GIS constrained Road Detection
Road Information Provided by GIS
Want to determine
Precise road center
Road Width
Training Sample Patches along roads Align patches along road direction Extract Features
Color Gradient
Feature Vector = histogram of color and gradients
Model: Gaussian Mixture model Offline Training
DetectionAlign the Road
Extract patches
Feed patches into MOG model
Response of the modelGives Road
center
Gradient Histogram
Peaks Give Road bounds
Road Detection
Object Detection Stabilization Optical flow warping Depth warping
Tracking with/without depth
without depth
with depth
Tracking with/without depth
without depth
with depth
Quantitative Results
False acceptance count
False rejection count
False identity switches
Ground truth object count
Multiple object racking accuracy
Quantitative Results
MOTA improvement: 0.740 to 0.851 (15% improvement)
FAR improvement: 0.190 to 0.072 (62% improvement)
More Results
More Results
More Results