path-based constraints for accurate scene reconstruction from aerial video
DESCRIPTION
Path-Based Constraints for Accurate Scene Reconstruction from Aerial Video. Mauricio Hess-Flores 1 , Mark A. Duchaineau 2 , Kenneth I. Joy 3. 1,3 Institute for Data Analysis and Visualization, University of California, Davis, USA. 2 Lawrence Livermore National Laboratory, Livermore, CA, USA**. - PowerPoint PPT PresentationTRANSCRIPT
Path-Based Constraints for Accurate Scene Path-Based Constraints for Accurate Scene Reconstruction from Aerial VideoReconstruction from Aerial Video
Mauricio Hess-Flores1, Mark A. Duchaineau2, Kenneth I. Joy3
Abstract - This paper discusses the constraints imposed by the path of a moving camera in multi-view sequential scene reconstruction scenarios such as in aerial video, which allow for an efficient detection and correction of inaccuracies in the feature tracking and structure computation processes. The main insight is that for short, planar segments of a continuous camera trajectory, parallax movement corresponding to a viewed scene point should ideally form a scaled and translated version of this trajectory when projected onto a parallel plane. Two constraints arise, which allow for the detection and correction of inaccurate feature tracks and scene structure, which differ from classical approaches such as factorization and RANSAC. Results are shown for real and synthetic aerial video and turntable sequences, where the use of such constraints was proven to correct outlier tracks, detect and correct tracking drift, and allow for a novel improvement of scene structure, while additionally resulting in an improved convergence for bundle adjustment optimization.
1,3Institute for Data Analysis and Visualization, University of California, Davis, USA
Introduction Algorithm (continued) Results
Future work
• Accurate models developed from aerial video can form a base for large-scale multi-sensor networks that support activities in detection, surveillance, tracking, registration, terrain modeling and ultimately semantic scene analysis.
• Due to varying lighting conditions, occlusions, repetitive patterns and other issues, feature tracks may not be perfect and this skews subsequent calibration and structure estimation.
• For short, planar segments of a continuous camera trajectory, parallax movement corresponding to a viewed scene point should ideally form a scaled and translated version of this trajectory, or a parallax path, when projected onto a parallel plane. This introduces two strong constraints, which differ from classical factorization and RANSAC, that can be used to detect and correct inaccurate feature tracks, while allowing for a very simple structure computation.
[email protected], [email protected], [email protected]
• Track auto-completion, making use of track scale values.• Accurate, dense feature tracking despite occlusions.• Accurate matching over texture-less regions. • Data compression, achieved by storing only scale-related information.• Multi-view tensor based on the proposed constraints.
This work was supported in part by the Department of Energy, National Nuclear Security Agency through Contract No. DE-GG52-09NA29355. This work was performed in part under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
2Lawrence Livermore National Laboratory, Livermore, CA, USA**
• Each path on the reconstruction plane, computed for a given track, is placed in a position-invariant reference, where ideally each differs only by scale:
1) Bundle adjustment convergence. Total reprojection error ε in pixels, processing time t in seconds and iterations I of Levenberg-Marquardt, for bundle adjustment applied using the output of the proposed algorithm (PPBA) versus bundle adjustment applied using the original feature tracks and structure (TBA), along with number of scene points NSP:
Input images
Algorithm flowchart
Consensus path Locus lineScaled paths
2) Drift detection and track correction results (Dinosaur dataset):
Ray equation:
C = camera center, P = projection matrix, plane = (A,B,C,D), x = pixel feature track position, X = 3D position
Position-invariant reference
Algorithm
**This author is now at Google, Inc.
Inaccurate dense
reconstruction Parallax paths
Ray-plane intersection:
Initial parallax path calculation (assuming known cameras)
Original parallax pathsAt position-invariant reference,
where paths only differ by scale s
Inter and intra-camera constraints• In this reference, inter-camera consensus path and intra-camera line constraints are defined, whose intersections (perfect grid) predict how inaccurate tracks should be corrected:
Replicas
Structure computation after constraint-based track correction
Original tracks Corrected tracks Detected drift
3) Improvement in scene structure (Stockton aerial dataset):
Position-invariant reference (top), constrained paths (bottom)
Path differences from perfect grid
Original (top) versus corrected structure (bottom)