lecture 13: cameras and geometry - cs department - homemtappen/cap5415/lecs/lec12.pdf• each set of...
TRANSCRIPT
![Page 1: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/1.jpg)
Lecture 12: Cameras and Geometry
CAP 5415Fall 2010
![Page 2: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/2.jpg)
The midterm
• What does the response of a derivative filter tell me about whether there is an edge or not?
![Page 3: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/3.jpg)
Things aren't working
• Did you look at the filters?• Why not?• Normalize the filters
![Page 4: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/4.jpg)
How do we see the world?
Let’s design a camera– Idea 1: put a piece of film in front of an
object– Do we get a reasonable image?
Slide by Steve Seitz
![Page 5: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/5.jpg)
Pinhole camera
Add a barrier to block off most of the rays– This reduces blurring– The opening known as the aperture– How does this transform the image?
Slide by Steve Seitz
![Page 6: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/6.jpg)
Pinhole camera model
Pinhole model:– Captures pencil of rays – all rays through a single point– The point is called Center of Projection (COP)– The image is formed on the Image Plane– Effective focal length f is distance from COP to Image
PlaneSlide by Steve Seitz
![Page 7: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/7.jpg)
A little bit of history on building cameras
![Page 8: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/8.jpg)
Camera Obscura• Latin for “Dark Box”• Dark room with a pinhole in wall• Projects image onto wall• Allows artists to get perspective right
Image from Wikipedia
![Page 9: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/9.jpg)
Camera Obscura
The first camera– Known to Aristotle– Depth of the room is the effective focal length
Camera Obscura, Gemma Frisius, 1558
![Page 10: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/10.jpg)
Camera Obscura
• Can also be a box
![Page 11: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/11.jpg)
We’ll use the pinhole camera model to describe image formation
(Image from Slides by Forsyth)
Notice how the image is inverted
![Page 12: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/12.jpg)
Projection Effects
• Height of objects depends on the distance from the pinhole (O)
(Image from Slides by Forsyth)
Pinhole
![Page 13: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/13.jpg)
Projection Effects: Horizon Line• Consider two parallel lines that lie in a plane (Π)• Will converge to a point on the horizon line(H)
(Image from Slides by Forsyth)
Pinhole
![Page 14: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/14.jpg)
• Observe this next time you are driving on a flat road
![Page 15: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/15.jpg)
Vanishing points• Each set of parallel
lines (=direction) meets at a different point– The vanishing point for
this direction
• Sets of parallel lines on the same plane lead to collinear vanishing points. – The line is called the
horizon for that plane
• Good ways to spot faked images– scale and perspective
don’t work– vanishing points
behave badly– supermarket tabloids
are a great source.
(From Slides by Forsyth)
![Page 16: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/16.jpg)
The equation of projection
(Image from Slides by Forsyth)
![Page 17: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/17.jpg)
The equation of projection
(Image from Slides by Forsyth)
We know:
so
![Page 18: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/18.jpg)
Lenses
• Why Lenses?• For an ideal pinhole, only one ray of light
reaches each point– Very Dim Image
• Why not make pinhole bigger?
![Page 19: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/19.jpg)
Why not make pinhole bigger?• Only one point can generate rays that strike
a particular point on the image plane
![Page 20: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/20.jpg)
Why not make pinhole bigger?• Now add an aperture
![Page 21: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/21.jpg)
Pinhole too big - many directions are averaged, blurring the image
Pinhole too small- diffraction effects blur the image
Generally, pinhole cameras are dark, becausea very small set of raysfrom a particular pointhits the screen.
(From Slides by Forsyth)
![Page 22: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/22.jpg)
Lenses
• The lens focuses multiple rays coming from the same point
(Image from Slides by Forsyth)
![Page 23: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/23.jpg)
Thin Lens Equation
![Page 24: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/24.jpg)
Focus and Defocus
A lens focuses light onto the film– There is a specific distance at which objects are “in
focus”• other points project to a “circle of confusion” in the
image– How can we change focus distance?
“circle of confusion”
Slide by Steve Seitz
![Page 25: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/25.jpg)
More on Lenses
Canon EF 28-135mm f/3.5-5.6 IS USM Standard Zoom Lens for Canon SLR Cameras
![Page 26: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/26.jpg)
28-135mm is the focal length
i o
P
P’
f
Diagram by Shree Nayar
![Page 27: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/27.jpg)
What's f/3.5-5.6?
Canon EF 28-135mm f/3.5-5.6 IS USM Standard Zoom Lens for Canon SLR Cameras
![Page 28: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/28.jpg)
f-number
• f is the focal length• D is the diameter of the pupil or aperture
• f/2 is the same as N=2• f/16 is the same as N=16• Which has the bigger aperture?
![Page 29: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/29.jpg)
What's f/3.5-5.6?
• This is the widest possible aperture
Canon EF 28-135mm f/3.5-5.6 IS USM Standard Zoom Lens for Canon SLR Cameras
![Page 30: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/30.jpg)
Why should I adjust the aperture?
• Big aperture means more light, shorter exposure time
• Also affects sharpness and depth of field
![Page 31: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/31.jpg)
Here, the rays are focused on the image plane
![Page 32: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/32.jpg)
Now, look at a point that is farther way
Circle of Confusion
![Page 33: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/33.jpg)
It grows as you move farther away
Circle of Confusion
![Page 34: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/34.jpg)
Circle of Confusion• Spot caused by a point that is not in focus
• You decide the tolerable limits(Diagram from Wikipedia)
![Page 35: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/35.jpg)
Aperture also causes blurring• Go back to pinhole camera model• Only one point can generate rays that strike
a particular point on the image plane
![Page 36: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/36.jpg)
Aperture also causes blurring• Now add an aperture
![Page 37: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/37.jpg)
Depth of Field• Increasing the aperture diameter increases
the size of the circle of confusion
f/22 f/5.6
![Page 38: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/38.jpg)
Diffraction• When light passes through a small aperture
the rays begin to interfere with each other• For a perfectly circular aperture this leads
to the airy disc pattern
Image from http://www.cambridgeincolour.com/tutorials/diffraction-photography.htm#
![Page 39: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/39.jpg)
This leads to a loss of sharpness
From http://www.cambridgeincolour.com/tutorials/diffraction-photography.htm#
f/8 f/11 f/16
f/22
![Page 40: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/40.jpg)
After Light Strikes the sensor
• Engineering problem:– I have sensor that records the amount of light
at different pixels– How do I get a color image instead of a black
and white image?
![Page 41: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/41.jpg)
Solutions
• Three sensors• One sensor with a color mask
– Each pixel records one wavelength
• A common pattern for the mask is the Bayer pattern:
![Page 42: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/42.jpg)
Mosaicing
• So, if I took a picture of this edge
• My sensor would record this image
![Page 43: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/43.jpg)
Demosaicing
• I have 1 color at each pixel• I need three• Easy solution: Interpolate
+
![Page 44: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/44.jpg)
Problem! This smooths across the edge
• Because the different pixels are used to red and green, the smoothing may be different
+
![Page 45: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/45.jpg)
Result: Color Fringing
![Page 46: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/46.jpg)
Color Fringing
(Results from Brainard et al)
![Page 47: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/47.jpg)
Fast Solution
• The fringing occurs when the correlation between the color channels is incorrectly estimated
• One measure of this correlation is the color difference
• Can fix errors using median filtering
![Page 48: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/48.jpg)
Simple Demosaicing Algorithm (Freeman)
• Use linear interpolation to get first estimate• Compute difference images between color
channels• Median filter these difference images• Use filtered difference images to
reconstruct
![Page 49: Lecture 13: Cameras and Geometry - CS Department - Homemtappen/cap5415/lecs/lec12.pdf• Each set of parallel lines (=direction) meets at a different point –The vanishing point for](https://reader034.vdocuments.us/reader034/viewer/2022042023/5e7adee5435f7224e327546b/html5/thumbnails/49.jpg)
(Slide by Freeman)