nikolas engelhard 1, felix endres 1, jürgen hess 1, jürgen sturm 2, wolfram burgard 1 1 university...
TRANSCRIPT
![Page 1: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/1.jpg)
Nikolas Engelhard1, Felix Endres1,
Jürgen Hess1, Jürgen Sturm2,
Wolfram Burgard1
1University of Freiburg, Germany
2Technical University Munich, Germany
Real-Time 3D Visual SLAM with a Hand-Held RGB-D Camera
![Page 2: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/2.jpg)
Motivation
Our goal: Learn 3D models of (indoor) scenes
Applications:– Architecture– Gaming– Archaeology
![Page 3: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/3.jpg)
Related Work
Visual SLAM
[Konolige, IROS 09], [Klein, ISMAR 09]
Laser scanners [Wurm, ICRA 10]
RGB-D [Pitzer, ICRA 10], [Henry, ISER 10]
![Page 4: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/4.jpg)
RGB-D Sensor
Principle: structured light– IR projector + IR camera– RGB camera
Dense depth images Full video frame rate
![Page 5: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/5.jpg)
Schematic Overview
![Page 6: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/6.jpg)
Feature Extraction and Matching SURF: scale and rotation invariant descriptor
Runtime– 600-800 features– OpenCV: 1 sec– SurfGPU: 40 ms
Matching– FLANN– < 15 ms
![Page 7: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/7.jpg)
Features in 3D
Associate features with 3D points Problem: missing data (glass, occlusion)
![Page 8: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/8.jpg)
Estimate Relative Pose
RANSAC: good for estimating models in the presence of outliers
Algorithm:– Find correspondences– Repeatedly sample three correspondences,
estimate pose, count inliers (and optimize)– Return pose with most inliers
Runtime: < 5 ms
![Page 9: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/9.jpg)
Correspondences in 2D
Image at time t Image at time t+1
![Page 10: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/10.jpg)
Pose Refinement
Iterative Closest Point Generalized ICP [Segal, 2009]
– Plane-to-Plane metric
Local optimization (prone to local maxima) Needs good initialization
Runtime: ~500 ms on a sampled subset
![Page 11: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/11.jpg)
Pose Refinement (2)
Without ICP With ICP
![Page 12: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/12.jpg)
Loop Closing
Summation of incremental error → drift Detect loops and average errors Model as pose graph Pose graph optimization [Grisetti, 2010]
System is over-determined Needs to find the best globally consistent
alignment
Corrected poses
![Page 13: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/13.jpg)
Example Pose Graph
Axes = estimated camera poses White edges = relative transformations
![Page 14: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/14.jpg)
Video (1/3)
[http://www.youtube.com/watch?v=XejNctt2Fcs]
![Page 15: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/15.jpg)
Video 2/3
[http://www.youtube.com/watch?v=5qrBEPfEPaY]
![Page 16: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/16.jpg)
Video 3/3
[http://www.youtube.com/watch?v=NR-ycTNcQu0]
![Page 17: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/17.jpg)
Conclusions
Visual SLAM system– SURF feature matching– RANSAC pose estimation– ICP pose refinement– Pose graph optimization
Real-time Open-source (in ROS) + Tutorial available:
http://www.ros.org/wiki/openni/Contests/ROS 3D/RGBD-6DSLAM
Live demo after the coffee break!
![Page 18: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/18.jpg)
Future Work Ground truth evaluation using a Motion
Capturing system (in cooperation with ETH Zurich)
Improve speed of ICP Parallelization
![Page 19: Nikolas Engelhard 1, Felix Endres 1, Jürgen Hess 1, Jürgen Sturm 2, Wolfram Burgard 1 1 University of Freiburg, Germany 2 Technical University Munich,](https://reader035.vdocuments.us/reader035/viewer/2022062515/56649cf85503460f949c8f66/html5/thumbnails/19.jpg)
Thank you!