![Page 1: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/1.jpg)
Marc LevoyComputer Science DepartmentStanford University
CS 178, Spring 2011
Panoramas
![Page 2: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/2.jpg)
! Marc Levoy
What is a panorama?
! a wider-angle image than a normal camera can capture
! any image stitched from overlapping photographs
! an extreme aspect ratio on a normal shot
2
![Page 3: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/3.jpg)
! Marc Levoy
Outline
! capturing panoramas
! stitching together a panorama
! perspective versus cylindrical projection
3
![Page 4: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/4.jpg)
! Marc Levoy
Panoramic cameras
4
swing-lens panoramic cameraflatback panoramic camera
![Page 5: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/5.jpg)
! Marc Levoy
Swing-lens panoramic images
5
San Francisco in ruins, 1906
101 Ranch, Oklahoma, circa 1920
![Page 6: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/6.jpg)
! Marc Levoy
Panoramic cameras
6
swing-lens panoramic cameraflatback panoramic camera
SLR on panning clamp motorized pan-tilt head
to avoid parallax errors,rotate around center of
perspective
![Page 7: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/7.jpg)
! Marc Levoy
Parallax errors
7
Photoshop aligned on the background, so only
foreground objects exhibit severe parallax errors
![Page 8: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/8.jpg)
Lee Frost, Val D’Orcia, Tuscany, Italy
Fuji GX 617
![Page 9: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/9.jpg)
Lee Frost, Volubilis, Morocco
![Page 10: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/10.jpg)
Lee Frost,Vertical Panoramas,Santorini
![Page 11: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/11.jpg)
Matthew Scott, Cuernos del Paine, Chile
![Page 12: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/12.jpg)
gigapan.org, Scanning Electron Micrograph (SEM) of barnacle
![Page 13: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/13.jpg)
gigapan.org, Scanning Electron Micrograph (SEM) of barnacle
![Page 14: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/14.jpg)
! Marc Levoy
Stitching images together to make a mosaic
14
![Page 15: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/15.jpg)
! Marc Levoy
What kind of transformation do we need?
15
translation?
rotation?
rotation?perspective!
![Page 16: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/16.jpg)
! Marc Levoy
Quick review of perspective projection
! these three image formation methods will produce the same perspective view on the p.p. (except for the size of the view)• all that matters is position of c.p. and orientation of p.p.
16
= =
pinhole camera photographic camera Albrecht Dürer’s drawing glass
p.p. p.p. p.p.
= center of perspective (c.p.)= projection of feature in scene onto picture plane (p.p)
![Page 17: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/17.jpg)
! Marc Levoy
Reprojecting an image ontoa different picture plane
! the view on any picture plane can be projected onto any other plane in 3D without changing its appearance as seen from a common center of projection
17
the sidewalk art of Julian Beever
![Page 18: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/18.jpg)
! Marc Levoy
Reprojecting panoramic imagesto a common picture plane
! the common picture plane of the mosaic replaces having had a wide-angle (non-fish-eye) camera in the first place
18
common p.p.of the mosaic
![Page 19: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/19.jpg)
! Marc Levoy
Homography
! perspective mapping between two p.p.’s using the same center of projection is called a homography• input and output x,y positions are related by a 3!3 matrix
19
p.p. #2
p.p. #1
![Page 20: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/20.jpg)
! Marc Levoy
Stitching images together to make a mosaic
! step 1: find corresponding features in a pair of image
! step 2: compute perspective from 2nd to 1st image
! step 3: warp 2nd image so it overlays 1st image
! step 4: blend images where they overlap one another
! repeat for 3rd image and mosaic of first two, etc.
20
![Page 21: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/21.jpg)
! Marc Levoy
! step 1: find corresponding features in a pair of image
! step 2: compute perspective from 2nd to 1st image
! step 3: warp 2nd image so it overlays 1st image
! step 4: blend images where they overlap one another
! repeat for 3rd image and mosaic of first two, etc.
Stitching images together to make a mosaic
21
Take CS 148:Introduction to Computer
Graphics (Aut)
Take CS 223B:Computer Vision (Win)
Also CS 448A:Computational
Photography (Win)
![Page 22: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/22.jpg)
! Marc Levoy
Example: the Matterhorn
22perspective projection
common picture
plane of mosaicimage
![Page 23: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/23.jpg)
! Marc Levoy
Using 4 shots instead of 3
23perspective projection
![Page 24: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/24.jpg)
! Marc Levoy
Cylindrical panoramas
! even works for 360° panorama
! project each image onto a cylinder
! a cylindrical image can be stored as a rectangular image
24
mosaic imagex
y
![Page 25: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/25.jpg)
! Marc Levoy
Cylindrical panoramas
! even works for 360° panorama
! project each image onto a cylinder
! a cylindrical image can be stored as a rectangular image
! to view without distortion, reproject part of the cylinder onto a picture plane representing the display screen• if your FOV is narrow, this view won’t be too distorted
25
mosaic imagex
y
(Flash demo)http://graphics.stanford.edu/courses/
cs178/applets/projection.html
![Page 26: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/26.jpg)
! Marc Levoycylindrical projection
Back to the Matterhorn
26
surface of cylinder
![Page 27: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/27.jpg)
! Marc Levoyblended
Back to the Matterhorn
27
surface of cylinder
![Page 28: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/28.jpg)
! Marc Levoy
Example
28perspective reprojection onto a plane
As mentioned in class, Photoshop does not perform the perspective reprojection shown here. Instead, it leaves you with the raw cylindrical image shown in the previous slide. On this image, straight lines are not straight, and the edges of the original photographs appear as curves. This is not a correct linear perspective. However, some panorama viewing software does perform this perspective reprojection, e.g. Microsoft’s HDView (Google for it).
![Page 29: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/29.jpg)
! Marc Levoy
Example
29cropped to avoid distortion
As I mentioned in class, the sequence of (1) projecting one or more images to a cylindrical surface, and (2) reprojecting that cylindrical image back to a planar surface, produces an image with no distortion, i.e. it is a correct linear perspective. However, it might be rather wide-angle, as shown in the previous slide. Unless you view this image with your face close to the display, which would be the correct viewpoint for such a wide-angle perspective, it will seem distorted. We covered this issue in the first lecture of the course. To reduce the requirement that you must place yourself so close to the display, you should crop the reprojected panorama, as is done in this slide. The resulting image is not so wide-angle, and can be viewed from a normal viewing distance without seeming distorted.
![Page 30: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/30.jpg)
! Marc Levoy
Spherical panoramas
! projections are to a sphere instead of a cylinder
! can’t store as rectangular image without distortion30
+
+
+
+
![Page 31: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/31.jpg)
! Marc Levoy
Recap! panoramas can be captured by a camera with a wide planar back,
a cylindrical back and a moving slit, or a rotating camera• rotate around the center of perspective to avoid parallax errors
! to assemble panoramas from a rotating camera, use corresponding features to compute a perspective warp that projects the images to a common picture plane, then blend them together
! for very wide angle or 360° panoramas, project the images to a common cylindrical surface, which can be stored as an ordinary (wide) rectangular image• reproject them to a picture plane for display
! spherical panoramas are possible, but cannot be stored as rectangular images without distortion
31 Quest ions?
![Page 32: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/32.jpg)
! Marc Levoy
Slide credits! Fredo Durand
! Alyosha Efros
! Steve Seitz
! Rick Szeliski
! Frost, Lee, Panoramic Photography, F+W Publications, 2005.
32
![Page 33: Panoramas - Computer Graphicsgraphics.stanford.edu/courses/cs178-11/lectures/...!21 Marc Levoy! step 1: find corresponding features in a pair of image! step 2: compute perspective](https://reader034.vdocuments.us/reader034/viewer/2022051920/600d2e1de07e36584977dfa1/html5/thumbnails/33.jpg)