vision-based registration for ar
DESCRIPTION
Vision-based Registration for AR. Presented by Diem Vu Nov 20, 2003. Markerless Tracking using Planar Structure in the Scene . G. Simon, A.W. Fitzgibbon and A. Zisserman, 2000. Calibration-Free Augmented Reality . K.N Kutulakos and J.R. Vallino , 1998. Planar-surface tracking. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/1.jpg)
Vision-based Registration for AR
Presented by Diem VuNov 20, 2003
![Page 2: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/2.jpg)
Markerless Tracking using Planar Structure in the Scene. G. Simon, A.W. Fitzgibbon and A. Zisserman, 2000.
Calibration-Free Augmented Reality. K.N Kutulakos and J.R. Vallino, 1998.
![Page 3: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/3.jpg)
Planar-surface tracking.Camera position can be recovered from
planar homography.Planar structure is common in almost all
scenarios.
![Page 4: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/4.jpg)
y
x
z
Hw
World to image homography
jiH
Image to image homography
![Page 5: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/5.jpg)
World to image homographyConsider our tracking plane is the plane
Z=0
y
x
z
Hw
1H
1w YX
yx
![Page 6: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/6.jpg)
Projection matrix
trrr 321KP
![Page 7: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/7.jpg)
trrrr 2121 KP
Projection matrix
1yx
10YX
y
x
z
P
10
K1
2
YX
yx
trrr 31
![Page 8: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/8.jpg)
trrrr 2121 KP
Projection matrix
1yx
10YX
y
x
z
P
1K
1YX
yx
trr 21
![Page 9: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/9.jpg)
If K and Hw are known, then r1, r2 and t can be recovered, thus P.
Question: How to compute Hw?Direct.Indirect.
trr 21KHw
![Page 10: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/10.jpg)
Direct measurement of Hw
Select 4 points {xk} on a rectangle in the scene.
Compute H which maps the unit square to {xk}.
(0,0)
(0,1) (1,1)
((1,0))
![Page 11: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/11.jpg)
Direct measurement of Hw
Select 4 points {xk} on a rectangle in the scene.
Compute H which maps the unit square to {xk}.
s,1)(1,diagH H w
trr 21 s HK -1
s,1)(1,diagK H trr 21
Compute Hw=Hdiag(1,1/s,1)
(0,0)
(0,s) (1,s)
((1,0))
![Page 12: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/12.jpg)
Indirect measurement of Hw
iwH
jiH
? H jw
y
x
z
![Page 13: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/13.jpg)
Indirect measurement of Hw
iwH
jiH
iw
ji
jw HHH
y
x
z
![Page 14: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/14.jpg)
Algorithm summaryCompute (direct measure).For each frame i, compute frame to frame
homography (RANSAC)Compute by:
0wH
1-iw
i1-i
iw HHH
i1-iH
iwH
![Page 15: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/15.jpg)
Other …Using only 2 points in direct method ??Matching the frame i with frame 0 in order
to reduce error.Estimate intrinsic parameters K Hand-off mechanism.
![Page 16: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/16.jpg)
![Page 17: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/17.jpg)
![Page 18: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/18.jpg)
![Page 19: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/19.jpg)
![Page 20: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/20.jpg)
Possible problems?Homography is only up-to-scale?Plain surface (no texture) or moving
objects in the foreground ?Depth order, occlusion ?Speed ?
![Page 21: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/21.jpg)
Affine virtual object representation
Represent virtual objects so that their projection can be computed as a linear combination of the projection of the fiducial points.
![Page 22: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/22.jpg)
![Page 23: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/23.jpg)
Project a point from its affine coordinates
![Page 24: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/24.jpg)
Compute affine coordinates from projection along two viewing
direction
![Page 25: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/25.jpg)
Algorithm Setup the affine basis
![Page 26: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/26.jpg)
Algorithm Setup the affine basis Locate the object in 2 frames.
![Page 27: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/27.jpg)
Algorithm Setup the affine basis Locate the object in 2 frames. Compute the affine coordinates
for each point.
![Page 28: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/28.jpg)
Algorithm Setup the affine basis Locate the object in 2 frames. Compute the affine coordinates
for each point. Compute projection of the object
and render the object in each frame.
![Page 29: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/29.jpg)
![Page 30: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/30.jpg)
Camera viewing direction and are the first and second row of
2x3.The camera viewing direction expressed in
the coordinate frame of the affine basis points: =
![Page 31: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/31.jpg)
Depth order
w is the z-value of point p (x,y,z).
![Page 32: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/32.jpg)
![Page 33: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/33.jpg)
AdvantagesNo need any metric information.Able to use with the existing hardware to
accelerate graphics operations.Can be used to improve tracking.
![Page 34: Vision-based Registration for AR](https://reader036.vdocuments.us/reader036/viewer/2022062315/56815d88550346895dcb92a2/html5/thumbnails/34.jpg)
LimitationAffine constraints.Lost of metric information.