yuping lin and gérard medioni. introduction method register uav streams to a global reference...
TRANSCRIPT
Yuping Lin and Gérard Medioni
Introduction Method
Register UAV streams to a global reference image▪ Consecutive UAV image registration▪ UAV to Map registration▪ Interleaving image to image and image to map▪ Partial local mosaic
Synchronization of multiple video streams Conclusion
Input: Multiple UAV video streams Position of moving objects in each video
stream
Goal: Synchronize using a common moving object
Register UAV streams to a global reference image (a map), then
Synchronize the streams using the unique path of a common moving object on the map
Introduction Method
Register UAV streams to a global reference image▪ Consecutive UAV image registration▪ UAV to Map registration▪ Interleaving image to image and image to map▪ Partial local mosaic
Synchronization of multiple video streams Conclusion
Input: Global reference image (Map) UAV stream The homography of the first frame of the
UAV stream to the map
ISSUES
UAV images and the map are different in terms of viewpoints, sensors and time of capture
Direct matching is difficult
APPROACH
Given the homography of the first UAV frame to the map,
Two step registration Consecutive UAV image
registration, then UAV to Map registration
Introduction Method
Register UAV streams to a global reference image▪ Consecutive UAV image registration▪ UAV to Map registration▪ Interleaving image to image and image to map▪ Partial local mosaic
Synchronization of multiple video streams Conclusion
Method: extract features in each frame Establish feature correspondences
between consecutive images estimate the transformation
ISSUES
Features should be descriptive for matching and sufficient to give good transform estimation
Feature matching
Transform estimation
APPROACH
SIFT feature extraction 128 dimension feature
descriptor Avg. 2000 features in each
image Nearest neighbor
matching Avg. 1000 matches in each
pair of images RANSAC
Avg. 600 inliers in each pair of images
Illustration
Illustration
Illustration
Illustration
MiH ,1
Illustration
MiH ,1 1,,1', iiMiMi HHH
Problem: error is accumulated
i
kkkMiiMiMi HHHHH
1,1,01,,1,
Introduction Method
Register UAV streams to a global reference image▪ Consecutive UAV image registration▪ UAV to Map registration▪ Interleaving image to image and image to map▪ Partial local mosaic
Synchronization of multiple video streams Conclusion
Method: Perform local search for
correspondences between the UAV image and the map
ISSUES
UAV images are very different from the map, SIFT features cannot always match
APPROACHES
Sample points in the map
For each point, locally search for the most similar image patch in the UAV image
Use Mutual Information as similarity measurement
Illustration
Illustration
',MiH
Illustration
',MiH
H
Illustration
Illustration
',, MiMi HHH
Introduction Method
Register UAV streams to a global reference image▪ Consecutive UAV image registration▪ UAV to Map registration▪ Interleaving image to image and image to map▪ Partial local mosaic
Synchronization of multiple video streams Conclusion
Method: Perform consecutive UAV image
registration and UAV to Map registration iteratively▪ Consecutive UAV image
registration produce good initials for UAV to Map registration▪ Register the partial local
mosaic to the map
ISSUES
Correspondences in a single frame are not enoughRegistration is unstable
APPROACH
Multiple frames in a time window forms a partial local mosaic which spans a larger region and provides more correspondences More robust Smooth transition
ISSUES
Correspondences in a single frame are not enoughRegistration is unstable
APPROACH
Multiple frames in a time window forms a partial local mosaic which spans a larger region and provides more correspondences More robust Smooth transition
Result
Register single frame
Register partial local mosaic
Illustration
Illustration
Illustration
Illustration
Result
Introduction Method
Register UAV streams to a global reference image▪ Consecutive UAV image registration▪ UAV to Map registration▪ Interleaving image to image and image to map▪ Partial local mosaic
Synchronization of multiple video streams Conclusion
Input: UAV image sequences of different views, different frame rates, but capture the same area and overlap in time
An moving object on the ground plane which serves as a “clock” to synchronize the sequences
The moving object should generate a single path on the map
Use sequence alignment algorithm to synchronize the UAV streams
Two steps to register an UAV image to the map Register each frame to its previous frame
to derive an initial estimate Register UAV image to the map to derive
Limitations Initial estimate should be given Unable to recover from a bad estimate
MH ,0',MiH
',MiH
MiH ,