motion from image and inertial measurements (additional slides)
DESCRIPTION
Motion from image and inertial measurements (additional slides). Dennis Strelow Carnegie Mellon University. Outline. Robust image feature tracking (in detail) Lucas-Kanade and real sequences The “smalls” tracker Motion from omnidirectional images. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/1.jpg)
Motion from image and inertial measurements
(additional slides)
Dennis Strelow
Carnegie Mellon University
![Page 2: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/2.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 2
Outline
Robust image feature tracking (in detail)
Lucas-Kanade and real sequences
The “smalls” tracker
Motion from omnidirectional images
![Page 3: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/3.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 3
Robust image feature tracking: Lucas-Kanade and real sequences (1)
Combining image and inertial measurements improves our situation, but…
we still need accurate feature tracking tracking
some sequences do not come with inertial measurements
![Page 4: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/4.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 4
Robust image feature tracking: Lucas-Kanade and real sequences (2)
better feature tracking for improved 6 DOF motion estimation
remaining results will be image-only
![Page 5: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/5.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 5
Robust image feature tracking: Lucas-Kanade and real sequences (3)
Lucas-Kanade has been the go-to feature tracker for shape-from-motion
minimizes a correlation-like matching error
using general minimization
evaluates the matching error at only a few locations
subpixel resolution
![Page 6: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/6.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 6
Robust image feature tracking: Lucas-Kanade and real sequences (4)
Additional heuristics used to apply Lucas-Kanade to shape-from-motion:
task: heuristic:
choose features to track high image texture
identify mistracked, occluded, no-longer-visible
convergence, matching error
handle large motions image pyramid
![Page 7: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/7.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 7
Robust image feature tracking: Lucas-Kanade and real sequences (5)
But Lucas-Kanade performs poorly on many real sequences…
![Page 8: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/8.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 8
Robust image feature tracking: the “smalls” tracker (1)
smalls is a new feature tracker targeted at 6 DOF motion estimation
exploits the rigid scene assumption
eliminates the heuristics normally used with Lucas-Kanade
SIFT is an enabling technology here
![Page 9: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/9.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 9
Robust image feature tracking: the “smalls” tracker (2)
First step: epipolar geometry estimation
use SIFT to establish matches between the two images
get the 6 DOF camera motion between the two images
get the epipolar geometry relating the two images
![Page 10: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/10.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 10
Robust image feature tracking: the “smalls” tracker (3)
![Page 11: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/11.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 11
Robust image feature tracking: the “smalls” tracker (4)
![Page 12: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/12.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 12
Robust image feature tracking: the “smalls” tracker (5)
Second step: track along epipolar lines
use nearby SIFT matches to get initial position on epipolar line
exploits the rigid scene assumption
eliminates heuristic: pyramid
![Page 13: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/13.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 13
Robust image feature tracking: the “smalls” tracker (6)
Third step: prune features
geometrically inconsistent features are marked as mistracked and removed
clumped features are pruned
eliminates heuristic: detecting mistracked features based on convergence, error
![Page 14: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/14.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 14
Robust image feature tracking: the “smalls” tracker (7)
Fourth step: extract new features
spatial image coverage is the main criterion
required texture is minimal when tracking is restricted to the epipolar lines
eliminates heuristic: extracting only textured features
![Page 15: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/15.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 15
Robust image feature tracking: the “smalls” tracker (8)
![Page 16: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/16.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 16
Robust image feature tracking: the “smalls” tracker (9)
left: odometry only right: images only
average error: 1.74 m
maximum error: 5.14 m
total distance: 230 m
![Page 17: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/17.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 17
Robust image feature tracking: the “smalls” tracker (10)
Recap:
exploits the rigid scene and eliminates heuristics
allows hands-free tracking for real sequences
can still be defeated by textureless areas or repetitive texture
![Page 18: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/18.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 18
Outline
Robust image feature tracking (in detail)
Motion from omnidirectional images
![Page 19: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/19.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 19
Motion from omnidirectional images (1)
![Page 20: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/20.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 20
Motion from omnidirectional images (2)
![Page 21: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/21.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 21
Motion from omnidirectional images (3)
![Page 22: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/22.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 22
Motion from omnidirectional images (4)
![Page 23: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/23.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 23
Motion from omnidirectional images (5)
left: non-rigid camera right: rigid camera
squares: ground truth points solid: image-only estimates
dash-dotted: image-and-inertial estimates
![Page 24: Motion from image and inertial measurements (additional slides)](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813e68550346895da879a3/html5/thumbnails/24.jpg)
Dennis Strelow -- Motion estimation from image and inertial measurements – January 6, 2005 24
Motion from omnidirectional images (6)
In this experiment:
omni images
conventional images + inertial
have roughly the same advantages
But in general:
inertial has some advantages that omni images alone can’t produce
omni images can be harder to use