MIT Media Lab Camera Culture Introduction to Time-of-Flight Range Imaging Refael Whyte

MIT Media Lab Camera Culture Why Depth? Color photos have limited depth information

MIT Media Lab Camera Culture Time-of-Flight Cameras 18 October 2013 THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 3

MIT Media Lab Camera Culture Measuring Distance Illuminate scene with amplitude modulated light. Phase difference between transmitted and reflected light encodes distance. Transmitted and returning light is correlated to measure distance. 18 October 2013 THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 4

MIT Media Lab Camera Culture Measuring Distance Returning light is correlated with reference waveform. Reference waveform is phase shifted Measured points fit a sine wave. 5

MIT Media Lab Camera Culture Correlation Function

MIT Media Lab Camera Culture Measuring Distance

MIT Media Lab Camera Culture Real Data Measured correlation waveform of one pixel Compute phase and amplitude from FFT.

MIT Media Lab Camera Culture Camera Data Phase and amplitude images, 120x160 sensor.

MIT Media Lab Camera Culture Measurement Errors

MIT Media Lab Camera Culture Multipath Multiple propagation paths. In scene paths and lens flare effects. Still an unsolved problem. Use low reflective material around objects of interest to mitigate 18/10/2013 THE UNIVERSITY OF WAIKATO TE WHARE WANANGA O WAIKATO 11

MIT Media Lab Camera Culture Multipath The measured pixels value is the sum of both components on the complex plane. Reflect table tops and floors can see reflection in the amplitude image.

MIT Media Lab Camera Culture Other Sources Temperature drift. Jitter. Jitter in the modulation signal limits precision. Photon shot noise in pixel structure. Is assumed to be an independent guassian noise source. Readout noise. Noise added by ADC and readout process. Harmonic aliasing impacts accuracy.

MIT Media Lab Camera Culture Time-of-Flight at MIT Access to commercial Time of flight cameras and custom built ones Commercial ones from: Softkinetic, PMD and Mesa Imaging

MIT Media Lab Camera Culture Coke Bottle Light Sweep Visualization of light in a coke bottle http://www.youtube.com/watch?v=fSqFWcb4rE Light sweep with ToF Camera https://sites.google.com/site/tofvid/ Similar results 1000x reduction in cost of hardware.

MIT Media Lab Camera Culture Time of Flight Applications Next generation Kinect for Xbox One. http://www.youtube.com/watch?v=Hi5kMNfgDS 4 Object Scanning http://www.youtube.com/watch?v=IXr9zqz6Bjc Hand tracking http://www.youtube.com/watch?v=Qj4ymzTNw Dw Other http://www.youtube.com/watch?v=eAlSfEqBBhM

MIT Media Lab Camera Culture Other Applications Manufacturing Automation Robotic navigation. Rumoured to being developed for phones so the sky is the limit.