siggraph 2014 course on computational cameras and displays (part 2)
DESCRIPTION
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass). This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.TRANSCRIPT
- 1. Computational Light Field Imaging Gordon Wetzstein MIT / Stanford University media.mit.edu/~gordonw displayblocks.org
2. Next-generation Imaging Devices computation optics & electronics human visual system interaction optics (compressive) computationsensing Computational & Compressive Displays Computational Imaging 3. Resources Conferences SIGGRAPH, CVPR, ICCV/ECCV, PROCAMS/CCD Workshop IEEE Int. Conference on Computational Photography (ICCP) OSA Computational Optical Sensing and Imaging (COSI) Courses SIGGRAPH 2012 course Computational Plenoptic Imaging SIGGRAPH 2014 course Time-of-Flight (after this, R 118-120) classes on Computational Photography (e.g., Stanford & MIT) 4. 2D position 2D direction x q The Light Field 5. sensor lenslet f Fixed trade-off between spatial and angular resolution! Integral Imaging Lippmann 1908 6. Scene from Above Lenslet Array [Lippman 1908], [Adelson and Wang 1992], [Ng et al. 2005] Integral Imaging # Sensor Pixels X #SensorPixelsY 7. Scene from Above Integral Imaging 8. Scene from Above # Sensor Pixels X / # Views X #SensorPixelsY/#ViewsY Integral Imaging: Spatio-Angular Resolution Tradeoff!Spatio-Angular Resolution Tradeoff! 9. Key Insight: Light Field is Redundant!Scene from Above 10. Exploit Redundancy Computationally Sparse Reconstruction Overcomplete Dictionaries Light Field Atoms Sparse Representation Optimal Optical Setup Mask-based Light Field Coding Compressive Light Field Photography ACM SIGGRAPH 2013 11. Captured2DImage4DReconstruction = Mask-Coded Projection 4DLightField Basis Pursuit Denoise: Sparse Coefficients! Compressive Light Field Photography ACM SIGGRAPH 2013 12. Output Switchable Light Field Photography ICCP 2014 13. Optics Algorithms Sensors Lytro.com Dappled Photography, Veeraraghavan, et al. ?An angle-sensitive CMOS imager for single- sensor 3D photography, Wang et al. Computational Photography 14. Phase gratings Two Interleaved Diodes Single Pixel Output 1 (D1) Output 2 (D2) Angle Sensitive Pixel 15. Angle (degrees) Response Amplitude (V) Single Pixel! Operating Principle: Talbot Effect Sum for instantaneous high-res 2D image! 16. Physical Layout Impulse Response 2D ASP Tile Demosaic image for quick low-res light field! 17. Computational Microscopy Nature Methods 2014Functional Brain Imaging C. elegans 18. Computational Microscopy Nature Methods 2014Functional Brain Imaging Captured Light Fieldoptics design by Marc Levoy 19. maximum intensity projection of volume 350um x 350 um x 24 um at 50Hz ~70 neurons in head region Nature Methods 2014 20. Nature Methods 2014 21. maximum intensity projection of volume 700um x 700 um x 200 um at 20Hz ~5,000 neurons zebrafish larvae Nature Methods 2014 22. Nature Methods 2014 23. Computational Imaging at SIGGRAPH 2014 Emerging Technologies Comp. Sensing & Display Papers Session, Tue 3:45-5:15, Hall A Displays Papers Session, Tue 10:45-12:15, Hall A