computational ⊗⊘ photographygraphics.stanford.edu/courses/cs478/lectures/01092012... ·...
TRANSCRIPT
![Page 1: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/1.jpg)
⊕⊖ Computational ⊗⊘ Photography
Introduction
Jongmin Baek
CS 478 LectureJan 9, 2012
Monday, January 9, 12
![Page 2: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/2.jpg)
Background
• Sales of digital cameras surpassed sales of film cameras in 2004.
Monday, January 9, 12
![Page 3: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/3.jpg)
• Free film
• Instant display
• Quality surpasses film
• Records metadata
• shooting parameters, camera location & orientation
Digital cameras are cool
Monday, January 9, 12
![Page 4: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/4.jpg)
• Same experience as film cameras
• Set zoom and focus
• Set aperture and exposure
• Press shutter to take a single picture
• Essentially, film camera with bits (0/1) ?
Digital cameras are boring
Monday, January 9, 12
![Page 5: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/5.jpg)
Digital cameras are boring
• The most common type of digital camera today: cellphone camera.
Can we leverage the computational power?
Monday, January 9, 12
![Page 6: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/6.jpg)
Course Information
• When: M/W 2:30-3:45
• Where: Gates 392
• Lecturers:
• Jongmin Baek
• Dave Jacobs
• Kari Pulli (NVidia)
Monday, January 9, 12
![Page 7: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/7.jpg)
Course Information
• Office hours: TTh 2:30-3:45, Gates 360
• Grading:
• 2 Assignments (15% each)
• 1 Final project (70%)
• Perks:
• Loaner NVidia Tegra 3 tablet(Thanks Kari)
Monday, January 9, 12
![Page 8: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/8.jpg)
Course Information• (Mostly unenforced) Requirements:
• Basic knowledge in graphics or vision or photography (CS148, CS178, etc)
• Mathematical maturity
• Good programming skills
• Necessary: C++ or Java
• Helpful: OpenCV, OpenGL, ImageStack
Monday, January 9, 12
![Page 9: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/9.jpg)
Course Information
• E-mail: [email protected]
• URL: cs478.stanford.edu
• Schedule
• Lecture slides
• Schedule
Monday, January 9, 12
![Page 10: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/10.jpg)
Computational Photography: Definition
• Computational techniques that enhance or extend the capabilities of digital photography
• Output is an ordinary photograph, but one that could not have been taken by a traditional camera
Monday, January 9, 12
![Page 11: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/11.jpg)
Computational Photography:an Interdisciplinary Field
• Computer graphics
• Computer vision
• Image processing
• Signal processing
• Optics
• Embedded systems
Monday, January 9, 12
![Page 12: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/12.jpg)
DigitalPhotography
Image processingapplied to capturedimages to producebetter images.
Examples:Interpolation, Filtering, Enhancement, Dynamic Range Compression,Color Management,Morphing, Hole Filling, Artistic Image Effects, Image Compression,Watermarking.
Computational Processing
Processing of a setof captured images to create newimages.
Examples:Mosaicing, Matting,Super-Resolution,Multi-Exposure HDR,Light Field from Multiple View, Structure from Motion, Shape from X.
Computational Imaging/Optics
Capture of opticallycoded images and computational decoding to producenew images.
Examples:Coded Aperture,Optical Tomography,Diaphanography,SA Microscopy,Integral Imaging,Assorted Pixels,Catadioptric Imaging,Holographic Imaging.
ComputationalSensor
Detectors thatcombine sensing and processing tocreate smart pixels.
Examples:Artificial Retina,Retinex Sensors,Adaptive DynamicRange Sensors, Edge Detect Chips,Focus of ExpansionChips, Motion Sensors.
Computational Illumination
Adapting andControllingIllumination toCreate revealingimage
Examples:Flash/no flash, Lighting domes,Multi-flash for depth edges,Dual Photos,Polynomial textureMaps, 4D lightsource
Computational PhotographyFilm-like Photography
with bits Computational Camera Smart Light
[Nayar, Tumblin]
Monday, January 9, 12
![Page 13: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/13.jpg)
• To expand: insert pixel along seams that, if removed, will yield original image.
Seam Carving for Content-Aware Image ResizingAvidan, Shamir (SIGGRAPH 2007)
Monday, January 9, 12
![Page 14: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/14.jpg)
• To contract: remove pixels along the lowest-energy seams, found with dynamic programming
• Object removal for an application?
Seam Carving for Content-Aware Image ResizingAvidan, Shamir (SIGGRAPH 2007)
Monday, January 9, 12
![Page 15: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/15.jpg)
A Bayesian Approach to Digital MattingChuang et al. (CVPR 2001)
• Generate local color model for foreground, background.
• Probabilistically assign alpha to unclassified pixels.
Monday, January 9, 12
![Page 16: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/16.jpg)
Removing Camera Shake from a Single ImageFergus et al. (SIGGRAPH 2006)
Fast Motion DeblurringCho, Lee (SIGGRAPH Asia 2009)
Monday, January 9, 12
![Page 17: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/17.jpg)
Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid
Paris, Hasinoff, Kautz (SIGGRAPH 2011)
Image Smoothing via L0 Gradient MinimizationXu et al. (SIGGRAPH Asia 2011)
Monday, January 9, 12
![Page 18: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/18.jpg)
DigitalPhotography
Image processingapplied to capturedimages to producebetter images.
Examples:Interpolation, Filtering, Enhancement, Dynamic Range Compression,Color Management,Morphing, Hole Filling, Artistic Image Effects, Image Compression,Watermarking.
Computational Processing
Processing of a setof captured images to create newimages.
Examples:Mosaicing, Matting,Super-Resolution,Multi-Exposure HDR,Light Field from Multiple View, Structure from Motion, Shape from X.
Computational Imaging/Optics
Capture of opticallycoded images and computational decoding to producenew images.
Examples:Coded Aperture,Optical Tomography,Diaphanography,SA Microscopy,Integral Imaging,Assorted Pixels,Catadioptric Imaging,Holographic Imaging.
ComputationalSensor
Detectors thatcombine sensing and processing tocreate smart pixels.
Examples:Artificial Retina,Retinex Sensors,Adaptive DynamicRange Sensors, Edge Detect Chips,Focus of ExpansionChips, Motion Sensors.
Computational Illumination
Adapting andControllingIllumination toCreate revealingimage
Examples:Flash/no flash, Lighting domes,Multi-flash for depth edges,Dual Photos,Polynomial textureMaps, 4D lightsource
Computational PhotographyFilm-like Photography
with bits Computational Camera Smart Light
[Nayar, Tumblin]
Monday, January 9, 12
![Page 19: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/19.jpg)
Interative Digital PhotomontageAgarwala et al. (SIGGRAPH 2004)
Monday, January 9, 12
![Page 20: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/20.jpg)
Interative Digital PhotomontageAgarwala et al. (SIGGRAPH 2004)
Monday, January 9, 12
![Page 21: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/21.jpg)
Interative Digital PhotomontageAgarwala et al. (SIGGRAPH 2004)
Monday, January 9, 12
![Page 22: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/22.jpg)
Interative Digital PhotomontageAgarwala et al. (SIGGRAPH 2004)
Monday, January 9, 12
![Page 23: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/23.jpg)
High Performance Imaging using Large Camera ArraysWilburn et al. (SIGGRAPH 2005)
• 640 × 480 pixels ×30 fps × 128 cameras
• synchronized timing• continuous streaming• flexible arrangement
Monday, January 9, 12
![Page 24: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/24.jpg)
High Performance Imaging using Large Camera ArraysWilburn et al. (SIGGRAPH 2005)
Σ
Monday, January 9, 12
![Page 25: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/25.jpg)
Multi-Exposure Imaging on Mobile DevicesGelfand et al. (ACM Multimedia 2010)
short exposure(outside ☻)
long exposure(inside ☻)
combined result(everywhere ☻)
Monday, January 9, 12
![Page 26: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/26.jpg)
Image Deblurring with Blurry/Noisy Image PairsYuan et al. (SIGGRAPH 2007)
long exposure(blurry)
short exposure(dark)
same, scaled up(noisy)
joint deconvolution
Monday, January 9, 12
![Page 27: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/27.jpg)
Focusednear
Light Efficient PhotographyHasinoff, Kutulakos (ECCV 2008)
(+ many others)
• Combine many photos in a focal stack.
Focusedafar
Monday, January 9, 12
![Page 28: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/28.jpg)
Light Efficient PhotographyHasinoff, Kutulakos (ECCV 2008)
(+ many others)
Monday, January 9, 12
![Page 29: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/29.jpg)
Viewfinder AlignmentAdams, Gelfand, Pulli (Eurographics 2008)
• Store and align viewfinder images in real-time.
individual frames, aligned panorama
Monday, January 9, 12
![Page 30: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/30.jpg)
DigitalPhotography
Image processingapplied to capturedimages to producebetter images.
Examples:Interpolation, Filtering, Enhancement, Dynamic Range Compression,Color Management,Morphing, Hole Filling, Artistic Image Effects, Image Compression,Watermarking.
Computational Processing
Processing of a setof captured images to create newimages.
Examples:Mosaicing, Matting,Super-Resolution,Multi-Exposure HDR,Light Field from Multiple View, Structure from Motion, Shape from X.
Computational Imaging/Optics
Capture of opticallycoded images and computational decoding to producenew images.
Examples:Coded Aperture,Optical Tomography,Diaphanography,SA Microscopy,Integral Imaging,Assorted Pixels,Catadioptric Imaging,Holographic Imaging.
ComputationalSensor
Detectors thatcombine sensing and processing tocreate smart pixels.
Examples:Artificial Retina,Retinex Sensors,Adaptive DynamicRange Sensors, Edge Detect Chips,Focus of ExpansionChips, Motion Sensors.
Computational Illumination
Adapting andControllingIllumination toCreate revealingimage
Examples:Flash/no flash, Lighting domes,Multi-flash for depth edges,Dual Photos,Polynomial textureMaps, 4D lightsource
Computational PhotographyFilm-like Photography
with bits Computational Camera Smart Light
[Nayar, Tumblin]
Monday, January 9, 12
![Page 31: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/31.jpg)
Light Field Photography with a Hand-Held Plenoptic CameraNg et al. (SIGGRAPH 2005)
Monday, January 9, 12
![Page 32: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/32.jpg)
Light Field Photography with a Hand-Held Plenoptic CameraNg et al. (SIGGRAPH 2005)
4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens
Adaptive Optics microlens array 125μ square-sided microlenses
Monday, January 9, 12
![Page 33: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/33.jpg)
Light Field Photography with a Hand-Held Plenoptic CameraNg et al. (SIGGRAPH 2005)
Monday, January 9, 12
![Page 34: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/34.jpg)
Monday, January 9, 12
![Page 35: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/35.jpg)
Light Field Photography with a Hand-Held Plenoptic CameraNg et al. (SIGGRAPH 2005)
(Now known as “Lytro” camera.)
Near
Far
Monday, January 9, 12
![Page 36: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/36.jpg)
Spatiotemporal modulation of defocus blur(“coded aperture”)
Levin et al. (SIGGRAPH 2007)Veeraraghavan et al. (SIGGRAPH 2007)
Nagahara et al. (ECCV 2008)Levin et al. (SIGGRAPH 2009)
Monday, January 9, 12
![Page 37: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/37.jpg)
Image and Depth from a Conventional Camera with a Coded Aperture
Levin et al. (SIGGRAPH 2007)
conventional aperture coded aperture
Monday, January 9, 12
![Page 38: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/38.jpg)
Image and Depth from a Conventional Camera with a Coded Aperture
Levin et al. (SIGGRAPH 2007)
input(blurred)
output(deblurred)
depthmap
Monday, January 9, 12
![Page 39: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/39.jpg)
Visualizing Photons in Motion at a Trillion Frames per Second
Velten, Raskar, Bawendi (OSA 2011)
Monday, January 9, 12
![Page 40: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/40.jpg)
DigitalPhotography
Image processingapplied to capturedimages to producebetter images.
Examples:Interpolation, Filtering, Enhancement, Dynamic Range Compression,Color Management,Morphing, Hole Filling, Artistic Image Effects, Image Compression,Watermarking.
Computational Processing
Processing of a setof captured images to create newimages.
Examples:Mosaicing, Matting,Super-Resolution,Multi-Exposure HDR,Light Field from Mutiple View, Structure from Motion, Shape from X.
Computational Imaging/Optics
Capture of opticallycoded images and computational decoding to producenew images.
Examples:Coded Aperture,Optical Tomography,Diaphanography,SA Microscopy,Integral Imaging,Assorted Pixels,Catadioptric Imaging,Holographic Imaging.
ComputationalSensor
Detectors thatcombine sensing and processing tocreate smart pixels.
Examples:Artificial Retina,Retinex Sensors,Adaptive DynamicRange Sensors, Edge Detect Chips,Focus of ExpansionChips, Motion Sensors.
Computational Illumination
Adapting andControllingIllumination toCreate revealingimage
Examples:Flash/no flash, Lighting domes,Multi-flash for depth edges,Dual Photos,Polynomial textureMaps, 4D lightsource
Computational PhotographyFilm-like Photography
with bits Computational Camera Smart Light
[Nayar, Tumblin]
Monday, January 9, 12
![Page 41: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/41.jpg)
Coded Exposure Photography: Motion Deblurring using Fluttered Shutter
Raskar, Agrawal, Tumblin (SIGGRAPH 2006)continuous shutter
Monday, January 9, 12
![Page 42: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/42.jpg)
Coded Exposure Photography: Motion Deblurring using Fluttered Shutter
Raskar, Agrawal, Tumblin (SIGGRAPH 2006)fluttered shuttercontinuous shutter
Monday, January 9, 12
![Page 43: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/43.jpg)
A Dual In-Pixel Memory CMOS Image Sensor for Computational Photography
Wan et al. (Symp. VLSI Circuits 2011)
“Ghosting”
Monday, January 9, 12
![Page 44: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/44.jpg)
A Dual In-Pixel Memory CMOS Image Sensor for Computational Photography
Wan et al. (Symp. VLSI Circuits 2011)
Photodiode
Storage 1 Storage 2
Storage 3 Storage 4
Monday, January 9, 12
![Page 45: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/45.jpg)
DigitalPhotography
Image processingapplied to capturedimages to producebetter images.
Examples:Interpolation, Filtering, Enhancement, Dynamic Range Compression,Color Management,Morphing, Hole Filling, Artistic Image Effects, Image Compression,Watermarking.
Computational Processing
Processing of a setof captured images to create newimages.
Examples:Mosaicing, Matting,Super-Resolution,Multi-Exposure HDR,Light Field from Mutiple View, Structure from Motion, Shape from X.
Computational Imaging/Optics
Capture of opticallycoded images and computational decoding to producenew images.
Examples:Coded Aperture,Optical Tomography,Diaphanography,SA Microscopy,Integral Imaging,Assorted Pixels,Catadioptric Imaging,Holographic Imaging.
ComputationalSensor
Detectors thatcombine sensing and processing tocreate smart pixels.
Examples:Artificial Retina,Retinex Sensors,Adaptive DynamicRange Sensors, Edge Detect Chips,Focus of ExpansionChips, Motion Sensors.
Computational Illumination
Adapting andControllingIllumination toCreate revealingimage
Examples:Flash/no flash, Lighting domes,Multi-flash for depth edges,Dual Photos,Polynomial textureMaps, 4D lightsource
Computational PhotographyFilm-like Photography
with bits Computational Camera Smart Light
[Nayar, Tumblin]
Monday, January 9, 12
![Page 46: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/46.jpg)
Digital Photography with Flash and No-Flash Image PairsPetschnigg et al. (SIGGRAPH 2004)
Flash No-Flash Combined
Monday, January 9, 12
![Page 47: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/47.jpg)
Digital Photography with Flash and No-Flash Image PairsPetschnigg et al. (SIGGRAPH 2004)
Flash No-Flash Combined
Monday, January 9, 12
![Page 48: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/48.jpg)
Dark Flash PhotographyKrishnan, Fergus (SIGGRAPH 2009)
Infrared No-Flash Combined Groudtruth
Monday, January 9, 12
![Page 49: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/49.jpg)
High Accuracy Stereo Depth Map using Structured LightScharstein, Szeliski (CVPR 2003)
Monday, January 9, 12
![Page 50: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/50.jpg)
High Accuracy Stereo Depth Map using Structured LightScharstein, Szeliski (CVPR 2003)
(Used in Kinect, etc.)
scene depth map
Monday, January 9, 12
![Page 51: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/51.jpg)
DigitalPhotography
Image processingapplied to capturedimages to producebetter images.
Examples:Interpolation, Filtering, Enhancement, Dynamic Range Compression,Color Management,Morphing, Hole Filling, Artistic Image Effects, Image Compression,Watermarking.
Computational Processing
Processing of a setof captured images to create newimages.
Examples:Mosaicing, Matting,Super-Resolution,Multi-Exposure HDR,Light Field from Mutiple View, Structure from Motion, Shape from X.
Computational Imaging/Optics
Capture of opticallycoded images and computational decoding to producenew images.
Examples:Coded Aperture,Optical Tomography,Diaphanography,SA Microscopy,Integral Imaging,Assorted Pixels,Catadioptric Imaging,Holographic Imaging.
ComputationalSensor
Detectors thatcombine sensing and processing tocreate smart pixels.
Examples:Artificial Retina,Retinex Sensors,Adaptive DynamicRange Sensors, Edge Detect Chips,Focus of ExpansionChips, Motion Sensors.
Computational Illumination
Adapting andControllingIllumination toCreate revealingimage
Examples:Flash/no flash, Lighting domes,Multi-flash for depth edges,Dual Photos,Polynomial textureMaps, 4D lightsource
Computational PhotographyFilm-like Photography
with bits Computational Camera Smart Light
[Nayar, Tumblin]
?
Monday, January 9, 12
![Page 52: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/52.jpg)
Lots of Cool Stuff, but...
• Many of these techniques require modifying the camera.
• Many of these techniques require precise control of the camera parameters.
• Need a fully programmable and extensible platform!
• Not really available prior to 2010 until the advent of ...
Monday, January 9, 12
![Page 53: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/53.jpg)
The Frankencamera: an Experimental Platform for Computational Photography
Adams et al. (SIGGRAPH 2010)
• a sensible API to control a camera
Monday, January 9, 12
![Page 54: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/54.jpg)
Course Summary
• Learn theories behind cool computational photography projects.
• Attend lectures.
• Learn how to put the theories into practice on a mobile platform.
• Assignment #1
• Assignment #2
• Final projectMonday, January 9, 12
![Page 55: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/55.jpg)
Assignment Summary
• Assignment #1 (15%)
• Write an autofocus algorithm for a camera application on a Tegra 3 tablet.
• Assignment #2 (15%)
• Image processing using OpenCV or ImageStack on Tegra 3 tablet.
• Final project (70%)
• Do something cool (by yourself or in a pair.)
Monday, January 9, 12
![Page 56: Computational ⊗⊘ Photographygraphics.stanford.edu/courses/cs478/lectures/01092012... · 2012-01-09 · Computational Photography: Definition • Computational techniques that](https://reader033.vdocuments.us/reader033/viewer/2022042009/5e71754ec4404958e6432449/html5/thumbnails/56.jpg)
Questions?
Monday, January 9, 12