where am i? using vanishing pointshspark/csci5980/lec4_localization... · jj jj ªº «» «»...
TRANSCRIPT
![Page 1: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/1.jpg)
Where am I? Using Vanishing Points
![Page 2: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/2.jpg)
Vanishing point
Vanishing point
What can vanishing line tell us about me? • Horizon • Camera pitch angle (looking down) • Camera roll angle (tilted toward right)
Vanishing line for horizon
![Page 3: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/3.jpg)
Ground plane
Where am I w.r.t. Ground Plane?
3D world
Camera
Xu
Yv
ZK r r r tC
1 2 3 W
11
How to compute?
3D world
r1
r2
r3
![Page 4: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/4.jpg)
What can a Vanishing Point tell us about? Camera
3D world
r1
r2
r3
v
Ground plane
![Page 5: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/5.jpg)
Ground plane
What can a Vanishing Point tell us about? Camera
3D world
r1
r2
r3
v
Z
0
0
1
0
![Page 6: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/6.jpg)
Ground plane
What can a Vanishing Point tell us about? Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
C1 2 3 W
0
0
1
0
v K r r r t
![Page 7: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/7.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
C1 2 3 W
0
0
1
0
v K r r r t
-13
K v r
3v Kr
![Page 8: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/8.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
C1 2 3 W
0
0
1
0
v K r r r t
-13
K v r
3v Kr
K v
rK v
-1
3 -1 because r3 is a unit vector.
Z vanishing point tells us about the surface normal of the ground plane
![Page 9: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/9.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
C1 2 3 W
0
0
1
0
v K r r r t
-13
K v r
3v Kr
K v
rK v
-1
3 -1 because r3 is a unit vector.
Z vanishing point tells us about the surface normal of the ground plane
Ground plane
Rotation ambiguity
![Page 10: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/10.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
-13
K v r
K v
rK v
-1
3 -1
Roll and pitch angle can be computed by the ground plane.
![Page 11: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/11.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
-13
K v r
K v
rK v
-1
3 -1
Roll and pitch angle can be computed by the ground plane.
3D spatial rotation: a sequence of Euler angle rotation (roll/pitch/yaw).
![Page 12: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/12.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
-13
K v r
K v
rK v
-1
3 -1
Roll and pitch angle can be computed by the ground plane.
Yaw: rotation about z axis: yaw
cos -sin 0
sin cos 0
0 0 1
R
![Page 13: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/13.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
-13
K v r
K v
rK v
-1
3 -1
Roll and pitch angle can be computed by the ground plane.
Yaw: rotation about z axis: yaw
cos -sin 0
sin cos 0
0 0 1
R
Pitch: rotation about y axis: pitch
cos 0 sin
0 1 0
-sin 0 cos
R
![Page 14: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/14.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
-13
K v r
K v
rK v
-1
3 -1
Roll and pitch angle can be computed by the ground plane.
Yaw: rotation about z axis: yaw
cos -sin 0
sin cos 0
0 0 1
R
Pitch: rotation about y axis: pitch
cos 0 sin
0 1 0
-sin 0 cos
R
Roll: rotation about x axis: roll
1 0 0
0 cos -sin
0 sin cos
R
![Page 15: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/15.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
-13
K v r
K v
rK v
-1
3 -1
Roll and pitch angle can be computed by the ground plane.
Yaw: rotation about z axis: yaw
cos -sin 0
sin cos 0
0 0 1
R
Pitch: rotation about y axis: pitch
cos 0 sin
0 1 0
-sin 0 cos
R
Roll: rotation about x axis: roll
1 0 0
0 cos -sin
0 sin cos
R
T
1 2 3 yaw pitch roll
-sin
cos sin
cos cos
r r r R R R
![Page 16: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/16.jpg)
Ground plane
Single Vanishing Point Camera
3D world
r1
r2
r3
v
0 0
0 0
1
1 0
Z
-13
K v r
K v
rK v
-1
3 -1
Roll and pitch angle can be computed by the ground plane.
-1 31
2 232 33
-1 32
33
tan
tan
r
r r
r
r
T
1 2 3 yaw pitch roll
-sin
cos sin
cos cos
r r r R R R
Pitch:
Roll:
![Page 17: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/17.jpg)
Single Vanishing Point (Exercise)
f = 1224; K = [f 0 size(im,2)/2; 0 f size(im,1)/2; 0 0 1]; m1 = [2563;25;1]; m2 = [2439;545;1]; m3 = [571;25;1]; m4 = [723;498;1]; l1 = GetLineFromTwoPoints(m1,m2); l2 = GetLineFromTwoPoints(m3,m4); v1 = GetPointFromTwoLines(l1,l2); v1 = v1/v1(3); r3 = inv(K)*v1/norm(inv(K)*v1) pitch = atan(-r3(1)/norm(r3(2:3))) roll = atan(r3(2)/r3(3))
ComputeCameraUsingVanishingPoint.m
r3 = -0.0736 0.8959 0.4381 pitch = 0.0736 roll = 1.1160
![Page 18: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/18.jpg)
Ground plane
Two Vanishing Points
3D world
Camera
r1
r2
r3
Yv
vX
3D world
![Page 19: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/19.jpg)
Ground plane
Two Vanishing Points Camera
0
1
0
0
Y
X
1
0
0
0
3D world
r1
r2
r3
Yv
vX
3D world
![Page 20: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/20.jpg)
Ground plane
Two Vanishing Points Camera
X
1
0
0
0
v K r r r t X
Kr
CX 1 2 3 W
1
0
1
0
0
Y
3D world
r1
r2
r3
Yv
vX
3D world
![Page 21: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/21.jpg)
Ground plane
Two Vanishing Points Camera
X
1
0
0
0
CY 1 2 3 W
2
v K r r r t Y
Kr
v K r r r t X
Kr
CX 1 2 3 W
1
0
1
0
0
Y
3D world
r1
r2
r3
Yv
vX
3D world
![Page 22: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/22.jpg)
Ground plane
Two Vanishing Points Camera
X
1
0
0
0
-1 -1
X Y1 2-1 -1
X Y
,
K v K v
r rK v K v
3 1 2 r r r : Orthogonality constraint
CY 1 2 3 W
2
v K r r r t Y
Kr0
1
0
0
Y
3D world
r1
r2
r3
v K r r r t X
Kr
C
X 1 2 3 W
1
Yv
vX
3D world
![Page 23: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/23.jpg)
Ground plane
Geometric Interpretation
3D world
r1
r2
r3
X
1
0
0
0
-1X
K v
![Page 24: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/24.jpg)
Ground plane
Geometric Interpretation
X
1
0
0
0
-1X
K v
0
1
0
0
Y-1Y
K v
3D world
r1
r2
r3
![Page 25: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/25.jpg)
Ground plane
Geometric Interpretation
X
1
0
0
0
-1X
K v
3D world
r1
r2
r3
-1Y
K v
0
1
0
0
Y
![Page 26: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/26.jpg)
Ground plane
Geometric Interpretation
X
1
0
0
0
-1X
K v
-1 -1X Y K v K v
3D world
r1
r2
r3
-1Y
K v
0
1
0
0
Y
![Page 27: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/27.jpg)
Ground plane
Geometric Interpretation (Translation Ambiguity)
![Page 28: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/28.jpg)
Ground plane
Geometric Interpretation (Translation Ambiguity)
![Page 29: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/29.jpg)
Ground plane
Geometric Interpretation (Translation Ambiguity)
![Page 30: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/30.jpg)
Ground plane
Two Vanishing Points
3D world
Camera
r1r2
r3
f = 4000; K = [f 0 size(im,2)/2; 0 f size(im,1)/2; 0 0 1]; l11 = GetLineFromTwoPoints(m11,m12); l12 = GetLineFromTwoPoints(m13,m14); l21 = GetLineFromTwoPoints(m21,m22); l22 = GetLineFromTwoPoints(m23,m24); v1 = GetPointFromTwoLines(l11,l12); v2 = GetPointFromTwoLines(l21,l22); r1 = inv(K)*v1/norm(inv(K)*v1); r2 = inv(K)*v2/norm(inv(K)*v2); r3 = Vec2Skew(r1)*r2;
R = 0.2448 -0.5178 0.0424 -0.1737 -0.1960 -0.6978 0.9539 0.8327 -0.1379 det(R) = 0.5077 R’*R = 0.3299 0.0294 -0.2036 0.0294 0.5555 -0.2327 -0.2036 -0.2327 1.6224
Not orthogonal matrix!
ComputeCameraUsingTwoVanishingPoints.m
Yv
vX
3D world
![Page 31: Where am I? Using Vanishing Pointshspark/CSci5980/Lec4_Localization... · JJ JJ ªº «» «» «»¬¼ R. Ground plane Single Vanishing Point Camera 3D world rr1 2 r3 v f 00 00 1](https://reader033.vdocuments.us/reader033/viewer/2022050523/5fa64ca5128bd2210f127209/html5/thumbnails/31.jpg)
Two Vanishing Points f = 1224; K = [f 0 size(im,2)/2; 0 f size(im,1)/2; 0 0 1]; l11 = GetLineFromTwoPoints(m11,m12); l12 = GetLineFromTwoPoints(m13,m14); l21 = GetLineFromTwoPoints(m21,m22); l22 = GetLineFromTwoPoints(m23,m24); v1 = GetPointFromTwoLines(l11,l12); v2 = GetPointFromTwoLines(l21,l22); r1 = inv(K)*v1/norm(inv(K)*v1); r2 = inv(K)*v2/norm(inv(K)*v2); r3 = Vec2Skew(r1)*r2;
ComputeCameraUsingTwoVanishingPoints.m
R = 0.5846 -0.8496 0.0508 -0.4149 -0.3216 -0.8367 0.6972 0.4180 -0.5405 det(R) = 0.9948 R’*R = 1.0662 -0.0118 0.0250 -0.0118 0.9757 0.0285 0.0250 0.0285 0.9530
Orthogonal matrix!
Change focal length