![Page 1: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/1.jpg)
![Page 2: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/2.jpg)
![Page 3: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/3.jpg)
Projective Geometry and Camera Models
Computer Vision
CS 143
Brown
James Hays
09/09/11
Slides from Derek Hoiem,
Alexei Efros, Steve Seitz, and
David Forsyth
![Page 4: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/4.jpg)
Administrative Stuff
• Textbook
• Matlab Tutorial
• Office hours
– James: Monday and Wednesday, 1pm to 2pm
– Geoff, Monday 7-9pm
– Paul, Tuesday 7-9pm
– Sam, Wednesday 7-9pm
– Evan, Thursday 7-9pm
• Project 1 is out
![Page 5: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/5.jpg)
Last class: intro
• Overview of vision, examples of state of art
• Computer Graphics: Models to Images
• Comp. Photography: Images to Images
• Computer Vision: Images to Models
![Page 6: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/6.jpg)
What do you need to make a camera from scratch?
![Page 7: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/7.jpg)
Today’s class
Mapping between image and world coordinates
– Pinhole camera model
– Projective geometry
• Vanishing points and lines
– Projection matrix
![Page 8: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/8.jpg)
Today’s class: Camera and World Geometry
How tall is this woman?
Which ball is closer?
How high is the camera?
What is the camera
rotation?
What is the focal length of
the camera?
![Page 9: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/9.jpg)
Image formation
Let’s design a camera – Idea 1: put a piece of film in front of an object – Do we get a reasonable image?
Slide source: Seitz
![Page 10: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/10.jpg)
Pinhole camera
Idea 2: add a barrier to block off most of the rays
– This reduces blurring
– The opening known as the aperture
Slide source: Seitz
![Page 11: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/11.jpg)
Pinhole camera
Figure from Forsyth
f
f = focal length
c = center of the camera
c
![Page 12: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/12.jpg)
Camera obscura: the pre-camera
• Known during classical period in China and Greece (e.g. Mo-Ti, China, 470BC to 390BC)
Illustration of Camera Obscura Freestanding camera obscura at UNC Chapel Hill
Photo by Seth Ilys
![Page 13: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/13.jpg)
Camera Obscura used for Tracing
Lens Based Camera Obscura, 1568
![Page 14: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/14.jpg)
First Photograph
Oldest surviving photograph
– Took 8 hours on pewter plate
Joseph Niepce, 1826
Photograph of the first photograph
Stored at UT Austin
Niepce later teamed up with Daguerre, who eventually created Daguerrotypes
![Page 15: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/15.jpg)
Point of observation
Figures © Stephen E. Palmer, 2002
Dimensionality Reduction Machine (3D to 2D)
3D world 2D image
![Page 16: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/16.jpg)
Projection can be tricky… Slide source: Seitz
![Page 17: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/17.jpg)
Projection can be tricky… Slide source: Seitz
![Page 18: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/18.jpg)
Projective Geometry
What is lost?
• Length
Which is closer?
Who is taller?
![Page 19: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/19.jpg)
Length is not preserved
Figure by David Forsyth
B’
C’
A’
![Page 20: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/20.jpg)
Projective Geometry
What is lost?
• Length
• Angles
Perpendicular?
Parallel?
![Page 21: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/21.jpg)
Projective Geometry
What is preserved?
• Straight lines are still straight
![Page 22: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/22.jpg)
Vanishing points and lines
Parallel lines in the world intersect in the image at a “vanishing point”
![Page 23: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/23.jpg)
Vanishing points and lines
o Vanishing Point o
Vanishing Point
Vanishing Line
![Page 24: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/24.jpg)
Vanishing points and lines
Vanishing point
Vanishing line
Vanishing point
Vertical vanishing point
(at infinity)
Slide from Efros, Photo from Criminisi
![Page 25: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/25.jpg)
Vanishing points and lines
Photo from online Tate collection
![Page 26: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/26.jpg)
Note on estimating vanishing points
![Page 27: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/27.jpg)
Projection: world coordinatesimage coordinates
Camera
Center
(tx, ty, tz)
Z
Y
X
P.
.
. f Z Y
v
up
.
Optical
Center
(u0, v0)
v
u
![Page 28: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/28.jpg)
Homogeneous coordinates
Conversion
Converting to homogeneous coordinates
homogeneous image
coordinates
homogeneous scene
coordinates
Converting from homogeneous coordinates
![Page 29: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/29.jpg)
Homogeneous coordinates
Invariant to scaling
Point in Cartesian is ray in Homogeneous
w
y
wx
kw
ky
kwkx
kw
ky
kx
w
y
x
k
Homogeneous Coordinates
Cartesian Coordinates
![Page 30: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/30.jpg)
Slide Credit: Saverese
Projection matrix
XtRKx x: Image Coordinates: (u,v,1)
K: Intrinsic Matrix (3x3)
R: Rotation (3x3)
t: Translation (3x1)
X: World Coordinates: (X,Y,Z,1)
Ow
iw
kw
jw
R,T
![Page 31: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/31.jpg)
Interlude: why does this matter?
![Page 32: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/32.jpg)
Object Recognition (CVPR 2006)
![Page 33: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/33.jpg)
Inserting photographed objects into images (SIGGRAPH 2007)
Original Created
![Page 34: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/34.jpg)
X0IKx
10100
000
000
1z
y
x
f
f
v
u
w
K
Slide Credit: Saverese
Projection matrix
Intrinsic Assumptions
• Unit aspect ratio
• Optical center at (0,0)
• No skew
Extrinsic Assumptions • No rotation
• Camera at (0,0,0)
![Page 35: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/35.jpg)
Remove assumption: known optical center
X0IKx
10100
00
00
1
0
0
z
y
x
vf
uf
v
u
w
Intrinsic Assumptions
• Unit aspect ratio
• No skew
Extrinsic Assumptions • No rotation
• Camera at (0,0,0)
![Page 36: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/36.jpg)
Remove assumption: square pixels
X0IKx
10100
00
00
1
0
0
z
y
x
v
u
v
u
w
Intrinsic Assumptions • No skew
Extrinsic Assumptions • No rotation
• Camera at (0,0,0)
![Page 37: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/37.jpg)
Remove assumption: non-skewed pixels
X0IKx
10100
00
0
1
0
0
z
y
x
v
us
v
u
w
Intrinsic Assumptions Extrinsic Assumptions • No rotation
• Camera at (0,0,0)
Note: different books use different notation for parameters
![Page 38: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/38.jpg)
Oriented and Translated Camera
Ow
iw
kw
jw
t
R
![Page 39: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/39.jpg)
Allow camera translation
XtIKx
1100
010
001
100
0
0
1
0
0
z
y
x
t
t
t
v
u
v
u
w
z
y
x
Intrinsic Assumptions Extrinsic Assumptions • No rotation
![Page 40: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/40.jpg)
3D Rotation of Points
Rotation around the coordinate axes, counter-clockwise:
100
0cossin
0sincos
)(
cos0sin
010
sin0cos
)(
cossin0
sincos0
001
)(
z
y
x
R
R
R
p
p’
y
z
Slide Credit: Saverese
![Page 41: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/41.jpg)
Allow camera rotation
XtRKx
1100
0
1 333231
232221
131211
0
0
z
y
x
trrr
trrr
trrr
v
us
v
u
w
z
y
x
![Page 42: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/42.jpg)
Degrees of freedom
XtRKx
1100
0
1 333231
232221
131211
0
0
z
y
x
trrr
trrr
trrr
v
us
v
u
w
z
y
x
5 6
![Page 43: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/43.jpg)
Orthographic Projection
• Special case of perspective projection
– Distance from the COP to the image plane is infinite
– Also called “parallel projection”
– What’s the projection matrix?
Image World
Slide by Steve Seitz
11000
0010
0001
1z
y
x
v
u
w
![Page 44: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/44.jpg)
Scaled Orthographic Projection
• Special case of perspective projection
– Object dimensions are small compared to distance to camera
– Also called “weak perspective”
– What’s the projection matrix?
Image World
Slide by Steve Seitz
1000
000
000
1z
y
x
s
f
f
v
u
w
![Page 45: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/45.jpg)
Field of View (Zoom)
![Page 46: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/46.jpg)
Suppose we have two 3D cubes on the ground facing the viewer, one near, one far.
1. What would they look like in perspective?
2. What would they look like in weak perspective?
Photo credit: GazetteLive.co.uk
![Page 47: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/47.jpg)
Beyond Pinholes: Radial Distortion
Image from Martin Habbecke
Corrected Barrel Distortion
![Page 48: Projective Geometry and - Brown University · Projective Geometry and Camera Models Computer Vision CS 143 Brown James Hays 09/09/11 Slides from Derek Hoiem, Alexei Efros, Steve Seitz,](https://reader033.vdocuments.us/reader033/viewer/2022042913/5f49febc91bb81620f6712a1/html5/thumbnails/48.jpg)
Things to remember
• Vanishing points and vanishing lines
• Pinhole camera model and camera projection matrix
• Homogeneous coordinates
Vanishing point
Vanishing line
Vanishing point
Vertical vanishing point
(at infinity)
XtRKx