vision-based registration for ar presented by diem vu nov 20, 2003
Post on 20-Dec-2015
219 views
TRANSCRIPT
Vision-based Registration for AR
Presented by Diem Vu
Nov 20, 2003
Markerless Tracking using Planar Structure in the Scene. G. Simon, A.W. Fitzgibbon and A. Zisserman, 2000.
Calibration-Free Augmented Reality. K.N Kutulakos and J.R. Vallino, 1998.
Planar-surface tracking.
Camera position can be recovered from planar homography.
Planar structure is common in almost all scenarios.
y
x
z
Hw
World to image homography
jiH
Image to image homography
World to image homography
Consider our tracking plane is the plane Z=0
y
x
z
Hw
1
H
1w Y
X
y
x
Projection matrix
trrr 321KP
trrrr 2121 KP
Projection matrix
1
y
x
1
0
Y
X
y
x
z
P
1
0K
12
Y
X
y
x
trrr 31
trrrr 2121 KP
Projection matrix
1
y
x
1
0
Y
X
y
x
z
P
1
K
1
Y
X
y
x
trr 21
If K and Hw are known, then r1, r2 and t can be recovered, thus P.
Question: How to compute Hw?Direct.Indirect.
trr 21KHw
Direct measurement of Hw
Select 4 points {xk} on a rectangle in the scene.
Compute H which maps the unit square to {xk}.
(0,0)
(0,1) (1,1)
((1,0))
Direct measurement of Hw
Select 4 points {xk} on a rectangle in the scene.
Compute H which maps the unit square to {xk}.
s,1)(1,diagH H w
trr 21 s HK -1
s,1)(1,diagK H trr 21
Compute Hw=Hdiag(1,1/s,1)
(0,0)
(0,s) (1,s)
((1,0))
Indirect measurement of Hw
iwH
jiH
? H jw
y
x
z
Indirect measurement of Hw
iwH
jiH
iw
ji
jw HHH
y
x
z
Algorithm summary
Compute (direct measure).For each frame i, compute frame to frame
homography (RANSAC)Compute by:
0wH
1-iw
i1-i
iw HHH
i1-iH
iwH
Other …
Using only 2 points in direct method ??Matching the frame i with frame 0 in order
to reduce error.Estimate intrinsic parameters K Hand-off mechanism.
Possible problems?
Homography is only up-to-scale?Plain surface (no texture) or moving
objects in the foreground ?Depth order, occlusion ?Speed ?
Affine virtual object representation
Represent virtual objects so that their projection can be computed as a linear combination of the projection of the fiducial points.
Project a point from its affine coordinates
Compute affine coordinates from projection along two viewing
direction
Algorithm
Setup the affine basis
Algorithm
Setup the affine basis Locate the object in 2 frames.
Algorithm
Setup the affine basis Locate the object in 2 frames. Compute the affine coordinates
for each point.
Algorithm
Setup the affine basis Locate the object in 2 frames. Compute the affine coordinates
for each point. Compute projection of the object
and render the object in each frame.
Camera viewing direction
and are the first and second row of 2x3.
The camera viewing direction expressed in the coordinate frame of the affine basis points: =
Depth order
w is the z-value of point p (x,y,z).
Advantages
No need any metric information.Able to use with the existing hardware to
accelerate graphics operations.Can be used to improve tracking.
Limitation
Affine constraints.Lost of metric information.