multi-camera time-of-flight systems | siggraph 2016
If you can't read please download the document
TRANSCRIPT
Slide 1
Computational Imaging with Multi-Camera Time-of-Flight SystemsFelix Heide*1,2Shikhar Shrestha*1Wolfgang Heidrich2,3Gordon Wetzstein12University of British Columbia3KAUST1Stanford Universitywww.computationalimaging.org* joint first authors
1
Time-of-Flight Imaging
Kinect for Xbox One
Google Tango Phone
Direct Sampling Transient Imaging [Velten et al. 2013]
Google Self-Driving Car
Facial capture in VR [Li et al. 2015]
Correlation Transient Imaging [Heide et al. 2013]
2
Multi-Camera ToF SystemMultiple Camera/SourcesPhased ArrayControl over modulation ExtensibleReproducible
3
light sourcecamera
Time-of-Flight ImagingPhase-shift
Time-of-Flight Imaging
4
Image formation
5
ElectronField
Photon
Charge storagebin 1Charge storagebin 2
Image formation
6
Image formation
1,2,3,4
7
Image formation
1,2,3,4
Static frameCaptured with TI OPT8241
Slow-Motion VideoCaptured with TI OPT8241
8
Time-of-Flight Imaging
Time-of-Flight Imaging Dynamic Scene
Doppler shift:
Orthogonal Frequencies:
9
Doppler Shift
10
Image formation Multicam
11
Image formation Frequency and Phase Synchronization
12
Image formation Frequency and Phase Synchronization
13
Multi-Camera ToF SystemExposure SynchronizationIndividual Sensor/Source Modulation SignalsHigh Frequency AccuracyCompatible with Recent ToF CamerasExtensibility
14
Multi-Camera ToF System High-Level System Architecture
15
Multi-Camera ToF System Key System Components
Direct Digital Synthesis (DDS)
Ch 0Ch 1Ch 2Ch 3Custom Board for End-to-End Signal ConditioningCombines a High Speed Comparator with Exposure Gating ICInterfaces Directly with TI ToF Camera
Signal Conditioning Circuitry
ToF Daughter BoardAD9959
Real-Time Controller
STMF4 Discovery Board
Controller responsible for triggering frames, controlling waveform generation and synchronizationRuns low-level firmware based on an interrupt driven state machine architecture
16
Multi-Camera ToF System System Workflow
Signal Conditioning Circuitry(Now on ToF Daughter Board)DDS BlockMCU Host Real-Time ControllerRef. Signal(Now uses SAW Oscillator @ 5MHz)PCToF Cameras
17
Multi-Camera ToF System System Workflow
Signal Conditioning Circuitry(Now on ToF Daughter Board)DDS BlockRef. Signal(Now uses SAW Oscillator @ 500 MHz)
Configure Modulation ParametersToF Cameras
18
Multi-Camera ToF System System Workflow
Signal Conditioning Circuitry(Now on ToF Daughter Board)DDS BlockRef. Signal(Now uses SAW Oscillator @ 5MHz)
Configure Camera Registers
19
Multi-Camera ToF System System Workflow
Signal Conditioning Circuitry(Now on ToF Daughter Board)DDS BlockRef. Signal(Now uses SAW Oscillator @ 5MHz)
Initiate a Frame
20
Multi-Camera ToF System System Workflow
DDS BlockRef. Signal(Now uses SAW Oscillator @ 5MHz)Start of Micro-Exposure/Quad
21
Multi-Camera ToF System System Workflow
Enable Modulation
22
Multi-Camera ToF System System Workflow
Modulation Signals to Sensor/Illum
23
Multi-Camera ToF System System Workflow
Raw Capture Data
24
Multi-Camera ToF System System Workflow
25
Multi-Camera ToF System Demo Unit Design
DC-DC Voltage RegulatorsHost MCUDDS BoardToF Daughter BoardToF Camera
26
Demo
27
Applications
Multi-Camera Interference
Parallel Phase Capture
Doppler Velocity Capture
Motion Non-Line-Of-Sight
Motion in Scattering Media
28
Applications
Multi-Camera Interference
Parallel Phase Capture
Doppler Velocity Capture
Motion Non-Line-Of-Sight
Motion in Scattering Media
29
Phased Camera Depth Imaging
30
Applications
Multi-Camera Interference
Parallel Phase Capture
Doppler Velocity Capture
Motion Non-Line-Of-Sight
Motion in Scattering Media
31
Simultaneous Doppler Velocity Imaging
32
Simultaneous Doppler Velocity Imaging
33
Applications
Multi-Camera Interference
Parallel Phase Capture
Doppler Velocity Capture
Motion Non-Line-Of-Sight
Motion in Scattering Media
34
Non-Line-of-Sight Motion Detection
35
Non-Line-of-Sight Motion Detection
36
Non-Line-of-Sight Motion Detection
37
Applications
Multi-Camera Interference
Parallel Phase Capture
Doppler Velocity Capture
Motion Non-Line-Of-Sight
Motion in Scattering Media
38
Motion in Scattering Media
39
Motion in Scattering Media
40
Multi-Camera ToF System
Open-Source Hardware Platform:
www.computationalimaging.org
41