siggraph 2014 course on computational cameras and displays (part 1)
DESCRIPTION
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass). This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.TRANSCRIPT
Gordon Wetzsteinhttp://web.media.mit.edu/~gordonw
MIT Media Lab / Stanford University
Computational Cameras and Displays
Matthew O’Toolewww.dgp.toronto.edu/~motoole
University of Toronto
www.dgp.toronto.edu/~motoole/computationalcamerasanddisplays.html
evolution of digital imaging2000 20142002 2004 2006 2008 2010 2012
evolution of digital imaging2000 20142002 2004 2006 2008 2010 2012
Canon EOS D303.1 megapixels
evolution of digital imaging2000 20142002 2004 2006 2008 2010 2012
Canon EOS D303.1 megapixels
EOS 10D6.3 MPs
EOS D606.3 MPs
EOS 20D8.2 MPs
EOS 30D8.2 MPs
EOS 40D10.1 MPs
EOS 50D15.1 MPs
EOS 60D17.9 MPs
Canon EOS 70D20.2 megapixels
reso
lutio
n(m
egap
ixel
s)
2000 2014
evolution of digital imaging2000 20142002 2004 2006 2008 2010 2012
Canon EOS D303.1 megapixels
EOS 10D6.3 MPs
EOS D606.3 MPs
EOS 20D8.2 MPs
EOS 30D8.2 MPs
EOS 40D10.1 MPs
EOS 50D15.1 MPs
EOS 60D17.9 MPs
Canon EOS 70D20.2 megapixels
reso
lutio
n(m
egap
ixel
s)qu
antu
m
effici
ency
(%)
2000 2014
2000 2014
measurements by DxOMark.com
evolution of digital imaging2000 20142002 2004 2006 2008 2010 2012
Canon EOS D303.1 megapixels
EOS 10D6.3 MPs
EOS D606.3 MPs
EOS 20D8.2 MPs
EOS 30D8.2 MPs
EOS 40D10.1 MPs
EOS 50D15.1 MPs
EOS 60D17.9 MPs
Canon EOS 70D20.2 megapixels
reso
lutio
n(m
egap
ixel
s)qu
antu
m
effici
ency
(%)
2000 2014
2000 2014
measurements by DxOMark.com
nearing end of the megapixel race for cameras and displays…
evolution of digital imaging
“Retina” display: pixels imperceptible to the eye
nearing end of the megapixel race for cameras and displays
2000 20142002 2004 2006 2008 2010 2012
Canon EOS D303.1 megapixels
EOS 10D6.3 MPs
EOS D606.3 MPs
EOS 20D8.2 MPs
EOS 30D8.2 MPs
EOS 40D10.1 MPs
EOS 50D15.1 MPs
EOS 60D17.9 MPs
Canon EOS 70D20.2 megapixels
evolution of digital imaging
“Retina” display: pixels imperceptible to the eye
nearing end of the megapixel race for cameras and displays
2000 20142002 2004 2006 2008 2010 2012
Canon EOS D303.1 megapixels
EOS 10D6.3 MPs
EOS D606.3 MPs
EOS 20D8.2 MPs
EOS 30D8.2 MPs
EOS 40D10.1 MPs
EOS 50D15.1 MPs
EOS 60D17.9 MPs
Canon EOS 70D20.2 megapixels
compressive displays[Lanman et al. 2011]light field imaging [Ng et al. 05]
cameras and displays revolutionca
mer
as
LytroPelican Imaging
[Venkataraman et al. SIG Asia 2013]Raytrix
cameras and displays revolution
Nintendo 3DSGoogle Glass
disp
lays
cam
eras
LytroPelican Imaging
[Venkataraman et al. SIG Asia 2013]Raytrix
cameras and displays revolution
Microsoft Kinect for Xbox OneLeap Motion
Nintendo 3DSGoogle Glass
disp
lays
light
tr
ansp
ort
cam
eras
LytroPelican Imaging
[Venkataraman et al. SIG Asia 2013]Raytrix
cameras and displays revolution
Microsoft Kinect for Xbox OneLeap Motion
Nintendo 3DSGoogle Glass
disp
lays
light
tr
ansp
ort
use computation to reform digital imaging (display, capture, …)
cam
eras
LytroPelican Imaging
[Venkataraman et al. SIG Asia 2013]Raytrix
three domains of computation
computational cameras
three domains of computation
computational cameras
computational displays
three domains of computation
computational cameras
computational displays
physicalworld
computational light transport
speakers
Gordon Wetzstein
• assistant professor at Stanford University (EE)• received an award for his dissertation titled
“Computational Plenoptic Image Acquisition and Display”
• spent last 3 years as a Postdoc at MIT Media Lab’s camera culture group
speakers
Matthew O’Toole
Gordon Wetzstein
• Ph.D. student at the University of Toronto supervised by Kyros Kutulakos
• focus on light transport analysis
• assistant professor at Stanford University (EE)• received an award for his dissertation titled
“Computational Plenoptic Image Acquisition and Display”
• spent last 3 years as a Postdoc at MIT Media Lab’s camera culture group
overview
10 min | computational cameras (Wetzstein)
overview
30 min | computational displays (Wetzstein)
10 min | computational cameras (Wetzstein)
overview
35 min | computational light transport (O’Toole)
30 min | computational displays (Wetzstein)
10 min | computational cameras (Wetzstein)
overview
35 min | computational light transport (O’Toole)
30 min | computational displays (Wetzstein)
10 min | computational cameras (Wetzstein)
10 min | summary and Q&A (both)
takeaways from the course
• review current research topics and recent work in computational sensing and displays
• introduce core concepts, including plenoptic cameras, compressive displays and transport matrices
• provide a sense as to possible directions for future research
Computational CamerasPart 1: