3d photography
DESCRIPTION
3D photography. Marc Pollefeys Fall 2007 http://www.inf.ethz.ch/personal/pomarc/courses/3dphoto/. 3D photography course schedule (tentative). Projective Geometry and Camera model Class 2. points, lines, planes conics and quadrics transformations camera model - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/1.jpg)
3D photography
Marc Pollefeys
Fall 2007
http://www.inf.ethz.ch/personal/pomarc/courses/3dphoto/
![Page 2: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/2.jpg)
3D photography course schedule(tentative)
Lecture Exercise
Sept 26 Introduction -
Oct. 3 Geometry & Camera model Camera calibration
Oct. 10 Single View Metrology Measuring in images
Oct. 17 Feature Tracking/matching(Friedrich Fraundorfer)
Correspondence computation
Oct. 24 Epipolar Geometry F-matrix computation
Oct. 31 Shape-from-Silhouettes(Li Guan)
Visual-hull computation
Nov. 7 Stereo matching Project proposals
Nov. 14 Structured light and active range sensing
Papers
Nov. 21 Structure from motion Papers
Nov. 28 Multi-view geometry and self-calibration
Papers
Dec. 5 Shape-from-X Papers
Dec. 12 3D modeling and registration Papers
Dec. 19 Appearance modeling and image-based rendering
Final project presentations
![Page 3: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/3.jpg)
Projective Geometry and Camera model
Class 2
points, lines, planesconics and quadrics
transformationscamera model
Read tutorial chapter 2 and 3.1http://www.unc.edu/courses/2004fall/comp/290/089/
![Page 4: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/4.jpg)
Homogeneous coordinates
0=++ cbyax ( ) ( ) 0=1x,y,a,b,c T
( ) ( ) 0≠∀,~ ka,b,cka,b,c TT
Homogeneous representation of 2D points and lines
equivalence class of vectors, any vector is representative
Set of all equivalence classes in R3(0,0,0)T forms P2
( ) ( ) 0≠∀,1,,~1,, kyxkyx TT
The point x lies on the line l if and only if
Homogeneous coordinates
Inhomogeneous coordinates ( )Tyx,
( )T321 ,, xxx but only 2DOF
Note that scale is unimportant for incidence relation
0=xlT
![Page 5: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/5.jpg)
Points from lines and vice-versa
l'×l=x
Intersections of lines
The intersection of two lines and is l l'
Line joining two points
The line through two points and is x'×x=lx x'
Example
1=x
1=y
1
)1-,1,0( y
x
1
)1-,0,1( y
x
Note:[ ] 'xx='x×x ×
with
0-
0-
-0
x
xy
xz
yz
![Page 6: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/6.jpg)
Ideal points and the line at infinity
T0,,l'l ab
Intersections of parallel lines
( ) ( )TTand ',,=l' ,,=l cbacba
Example
1=x 2=x
Ideal points ( )T0,, 21 xx
Line at infinity ( )T1,0,0=l∞
∞22 l∪= RP
tangent vector
normal direction
-ab,( )ba,
Note that in P2 there is no distinction
between ideal points and others
![Page 7: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/7.jpg)
3D points and planes
Homogeneous representation of 3D points and planes
The point X lies on the plane π if and only if
0=π+π+π+π 44332211 XXXX
0=XπT
The plane π goes through the point X if and only if
0=XπT
![Page 8: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/8.jpg)
Planes from points
0π
X
X
X
3
2
1
T
T
T
0=πX 0=πX 0,=πX π 321TTT andfromSolve
(solve as right nullspace of )π
T
T
T
3
2
1
X
X
X
![Page 9: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/9.jpg)
Points from planes
0X
π
π
π
3
2
1
T
T
T
x=X M 321 XXXM
0=π MT
0=Xπ 0=Xπ 0,=Xπ X 321TTT andfromSolve
(solve as right nullspace of )X
T
T
T
3
2
1
π
π
π
Representing a plane by its span
![Page 10: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/10.jpg)
Lines
T
T
B
AW μBλA
2×2** 0=WW=WWTT
0001
1000W
0010
0100W*
Example: X-axis
(4dof)Representing a line by its span
T
T
Q
PW* μQλP
Dual representation
(Alternative: Plücker representation, details see e.g. H&Z)
![Page 11: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/11.jpg)
Points, lines and planes
TX
WM 0=πM
Tπ
W*
M 0X M
W
X
*W
π
![Page 12: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/12.jpg)
Conics
Curve described by 2nd-degree equation in the plane
0=+++++ 22 feydxcybxyax
0=+++++ 233231
2221
21 fxxexxdxcxxbxax
3
2
3
1 , xxyx
xx or homogenized
0=xx CT
or in matrix form
fed
ecb
dba
2/2/
2/2/
2/2/
Cwith
{ }fedcba :::::5DOF:
![Page 13: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/13.jpg)
Five points define a conic
For each point the conic passes through
0=+++++ 22 feydxcyybxax iiiiii
or
( ) 0=.1,,,,, 22 ciiiiii yxyyxx ( )Tfedcba ,,,,,=c
0
1
1
1
1
1
552555
25
442444
24
332333
23
222222
22
112111
21
c
yxyyxx
yxyyxx
yxyyxx
yxyyxx
yxyyxx
stacking constraints yields
![Page 14: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/14.jpg)
Tangent lines to conics
The line l tangent to C at point x on C is given by l=Cx
lx
C
![Page 15: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/15.jpg)
Dual conics
0=ll *CTA line tangent to the conic C satisfies
Dual conics = line conics = conic envelopes
1-* CC In general (C full rank):
![Page 16: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/16.jpg)
Degenerate conics
A conic is degenerate if matrix C is not of full rank
TT ml+lm=C
e.g. two lines (rank 2)
e.g. repeated line (rank 1)
Tll=C
l
l
m
Degenerate line conics: 2 points (rank 2), double point (rank1)
( ) CC ≠**Note that for degenerate conics
![Page 17: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/17.jpg)
Quadrics and dual quadrics
(Q : 4x4 symmetric matrix)0=QXXT
• 9 d.o.f.• in general 9 points define quadric • det Q=0 ↔ degenerate quadric• tangent plane
Q
QX=π
0=πQπ *T
-1* Q=Q• relation to quadric (non-degenerate)
![Page 18: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/18.jpg)
2D projective transformations
A projectivity is an invertible mapping h from P2 to itself such that three points x1,x2,x3 lie on the same line if and
only if h(x1),h(x2),h(x3) do.
Definition:
A mapping h:P2P2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx
Theorem:
Definition: Projective transformation
3
2
1
333231
232221
131211
3
2
1
'
'
'
x
x
x
hhh
hhh
hhh
x
x
x
x=x' Hor
8DOF
projectivity=collineation=projective transformation=homography
![Page 19: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/19.jpg)
Transformation of 2D points, lines and conics
Transformation for lines
l=l' -TH
Transformation for conics-1-TCHHC ='
Transformation for dual conicsTHHCC ** ='
x=x' HFor a point transformation
![Page 20: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/20.jpg)
Fixed points and lines
e=e λH (eigenvectors H =fixed points)
l=l λTH (eigenvectors H-T =fixed lines)
(1=2 pointwise fixed line)
![Page 21: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/21.jpg)
Hierarchy of 2D transformations
1002221
1211
y
x
taa
taa
1002221
1211
y
x
tsrsr
tsrsr
333231
232221
131211
hhh
hhh
hhh
1002221
1211
y
x
trr
trr
Projective8dof
Affine6dof
Similarity4dof
Euclidean3dof
Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio
Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞
Ratios of lengths, angles.The circular points I,J
lengths, areas.
invariantstransformed squares
![Page 22: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/22.jpg)
The line at infinity
∞∞∞ l
1
0
0
1t
0ll
A
AH
TT
A
The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity
Note: not fixed pointwise
![Page 23: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/23.jpg)
Affine properties from images
projection rectification
APA
lll
HH
321
010
001
0≠,l 3321∞ llll T
![Page 24: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/24.jpg)
Affine rectificationv1 v2
l1
l2 l4
l3
l∞
21∞ vvl
211 llv 432 l×l=v
![Page 25: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/25.jpg)
The circular points
0
1
I i
0
1
J i
I
0
1
0
1
100
cossin
sincos
II
iseitss
tssi
y
x
S
H
The circular points I, J are fixed points under the projective transformation H iff H is a similarity
![Page 26: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/26.jpg)
The circular points
“circular points”
0=++++ 233231
22
21 fxxexxdxxx 0=+ 2
221 xx
l∞
T
T
0,-,1J
0,,1I
i
i
( ) ( )TT 0,1,0+0,0,1=I i
Algebraically, encodes orthogonal directions
03 x
![Page 27: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/27.jpg)
Conic dual to the circular points
TT JIIJ*∞ C
000
010
001*∞C
TSS HCHC *
∞*∞
The dual conic is fixed conic under the projective transformation H iff H is a similarity
*∞C
Note: has 4DOF
l∞ is the nullvector
*∞C
l∞
I
J
![Page 28: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/28.jpg)
Angles
( )( )22
21
22
21
2211
++
+=cos
mmll
mlmlθ
( )T321 ,,=l lll ( )T
321 ,,=m mmmEuclidean:
Projective: mmll
mlcos
**
*
CC
CTT
T
0=ml *∞CT (orthogonal)
![Page 29: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/29.jpg)
Transformation of 3D points, planes and quadrics
Transformation for lines
( )l=l' -TH
Transformation for conics
( )-1-TCHHC ='
Transformation for dual conics
( )THHCC ** ='
( )x=x' H
For a point transformation
X=X' H
π=π' -TH
-1-TQHH=Q'
THHQ=Q' **
(cfr. 2D equivalent)
![Page 30: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/30.jpg)
Hierarchy of 3D transformations
vTv
tAProjective15dof
Affine12dof
Similarity7dof
Euclidean6dof
Intersection and tangency
Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞
Angles, ratios of length
The absolute conic Ω∞
Volume
10
tAT
10
tRT
s
10
tRT
![Page 31: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/31.jpg)
The plane at infinity
∞∞∞ π
1
0
0
0
1t-
0ππ
A
AH
TT
A
The plane at infinity π is a fixed plane under a projective transformation H iff H is an affinity
1. canonical position2. contains directions 3. two planes are parallel line of intersection in π∞
4. line // line (or plane) point of intersection in π∞
( )T1,0,0,0=π∞
( )T0,,,=D 321 XXX
![Page 32: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/32.jpg)
The absolute conic
The absolute conic Ω∞ is a fixed conic under the projective transformation H iff H is a similarity
04
23
22
21
X
XXX
The absolute conic Ω∞ is a (point) conic on π.
In a metric frame:
T321321 ,,I,, XXXXXXor conic for directions:(with no real points)
1. Ω∞ is only fixed as a set2. Circle intersect Ω∞ in two circular points3. Spheres intersect π∞ in Ω∞
![Page 33: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/33.jpg)
The absolute dual quadric
00
0I*∞ T
The absolute dual quadric Ω*∞ is a fixed conic under
the projective transformation H iff H is a similarity
1. 8 dof2. plane at infinity π∞ is the nullvector of Ω∞
3. Angles:
( )( )2
*∞21
*∞1
2*∞1
πΩππΩπ
πΩπ=cos
TT
T
θ
![Page 34: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/34.jpg)
Camera model
Relation between pixels and rays in space
?
![Page 35: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/35.jpg)
Pinhole camera
![Page 36: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/36.jpg)
Pinhole camera model
TT ZfYZfXZYX )/,/(),,(
101
0
0
1
Z
Y
X
f
f
Z
fY
fX
Z
Y
X
linear projection in homogeneous coordinates!
![Page 37: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/37.jpg)
Pinhole camera model
101
0
0
Z
Y
X
f
f
Z
fY
fX
101
01
01
1Z
Y
X
f
f
Z
fY
fX
PXx
0|I)1,,(diagP ff
![Page 38: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/38.jpg)
Principal point offset
Tyx
T pZfYpZfXZYX )+/,+/(),,(
principal pointT
yx pp ),(
101
0
0
1
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
Z
Y
X
y
x
x
x
![Page 39: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/39.jpg)
Principal point offset
101
0
0
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
y
x
x
x
camX0|IKx
1y
x
pf
pf
K calibration matrix
![Page 40: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/40.jpg)
Camera rotation and translation
( )C~
-X~
R=X~
cam
X10
RC-R
1
10
C~
R-RXcam
Z
Y
X
camX0|IKx XC~
-|IKRx
t|RKP C~
R-t PX=x
~
![Page 41: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/41.jpg)
CCD camera
1yy
xx
p
p
K
11y
x
y
x
pf
pf
m
m
K
![Page 42: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/42.jpg)
General projective camera
1yx
xx
p
ps
K
1yx
xx
p
p
K
C~
|IKRP
non-singular
11 dof (5+3+3)
t|RKP
intrinsic camera parametersextrinsic camera parameters
![Page 43: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/43.jpg)
Radial distortion
• Due to spherical lenses (cheap)• Model:
R
y
xyxKyxKyx ...))()(1(),( 44
222
1R
http://foto.hut.fi/opetus/260/luennot/11/atkinson_6-11_radial_distortion_zoom_lenses.jpgstraight lines are not straight anymore
![Page 44: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/44.jpg)
Camera model
Relation between pixels and rays in space
?
![Page 45: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/45.jpg)
Projector model
Relation between pixels and rays in space(dual of camera)
(main geometric difference is vertical principal point offset to reduce keystone effect)
?
![Page 46: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/46.jpg)
Meydenbauer camera
vertical lens shiftto allow direct ortho-photographs
![Page 47: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/47.jpg)
Affine cameras
![Page 48: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/48.jpg)
Action of projective camera on points and lines
forward projection of line
( ) μb+a=μPB+PA=μB)+P(A=μX
back-projection of line
lP=Π T
PXlX TT ( )PX= x0;=xlT
PX=x
projection of point
![Page 49: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/49.jpg)
Action of projective camera on conics and quadricsback-projection to cone
CPP=Q Tco 0=CPXPX=Cxx TTT
( )PX=x
projection of quadric
TPPQC ** 0lPPQlQ T*T*T
( )lP=Π T
![Page 50: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/50.jpg)
Image of absolute conic
![Page 51: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/51.jpg)
A simple calibration device
(i) compute H for each square (corners (0,0),(1,0),(0,1),(1,1))
(ii) compute the imaged circular points H(1,±i,0)T
(iii) fit a conic to 6 circular points(iv) compute K from through cholesky factorization
(≈ Zhang’s calibration method)
![Page 52: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/52.jpg)
Exercises: Camera calibration
![Page 53: 3D photography](https://reader030.vdocuments.us/reader030/viewer/2022032612/56812d8b550346895d92a4b7/html5/thumbnails/53.jpg)
Next class:Single View Metrology
Antonio Criminisi