depth from structured light ii: error analysis michael h. rosenthal april 19th, 2000
TRANSCRIPT
Depth from Structured Light II:Error Analysis
Michael H. Rosenthal
April 19th, 2000
4-19-00 Rosenthal - 2
Outline
• Review of Structured Light Projection
• Error in Ideal Model
• Lighting and Surface Property Errors
• Stripe Assignment Errors
• Motion Errors
• Structured Light in Practice
4-19-00 Rosenthal - 3
Structured Light Projection
• Camera and projector are calibrated
• Light stripes are projected onto scene
• Depth for each pixel is measured by intersecting pixel ray with stripe plane
4-19-00 Rosenthal - 4
Binary Stripe Encoding
• Pixels are “labeled” using binary encoding: light = 1, dark = 0
• High-order bits encode broad regions, low-order bits encode fine location
4-19-00 Rosenthal - 5
Error in Ideal Model
4-19-00 Rosenthal - 6
Error in Ideal Model
• Pixels and stripes have finite size
• All visible surfaces within the pixel/stripe intersection volume are sampled and lumped together
4-19-00 Rosenthal - 7
Error in Ideal Model
• Size of sampled volume depends upon stripe width, pixel width and projector orientation
• Corollary: depth resolution is controlled by the same factors
4-19-00 Rosenthal - 8
Error from Lighting and Reflection
4-19-00 Rosenthal - 9
Lighting and Reflection
• Prior assumptions:– Bright Lambertian
surface
– Path from projector to surface to camera
• Real world– Dark or shiny materials
(apples, metal, cloth)
– Things in your way Specular highlights interfere with structured light patterns
4-19-00 Rosenthal - 10
Lighting and Reflection
Preventive Methods:
• Capture images with projector fully on and fully off
• Use difference to measure intensity range for each pixel
• Threshold difference to find to find trouble spots
Images from structured light projected into a simple scene
4-19-00 Rosenthal - 11
Lighting and Reflection
•Minimum intensity image may have bright spots - specular highlights
•Maximum intensity image may have dark spots - shadows or dark objects
4-19-00 Rosenthal - 12
Lighting and Reflection
•Difference image shows range of intensity between on and off
•Narrow range corresponds to non-ideal behavior
•Threshold to identify trouble spots
•Exclude trouble spots from final depth image
4-19-00 Rosenthal - 13
Lighting and Reflection
• Other practical solutions:– Lambertian coatings
and sprays (talc)
– High dynamic range cameras
– Variable exposure times
4-19-00 Rosenthal - 14
Error in Stripe Assignment
4-19-00 Rosenthal - 15
Stripe Assignment Errors
• Stripe encoding requires binary decision for each pixel
• What happens if a pixel is misassigned to the wrong stripe (i.e. gets a 0 rather than a 1 when illuminated)?
Images from structured light projected into a simple scene
4-19-00 Rosenthal - 16
Stripe Assignment Errors
• Caused by random noise, low contrast, motion, edge uncertainty, etc.
• Magnitude depends upon significance of bit!
• 1st bit yields error N/2, 2nd bit yields error N/4, kth bit yields error N/2k
Binary stripe encoding (phase shifted method)
4-19-00 Rosenthal - 17
Stripe Assignment Errors
• Assume that probability of misassignment is p• Expected error from kth bit is p*N/2k • Total expected error E = p*N/2 + p * N/4 +… + p
* N/2k + … + p*N/2logN
• E = p*N*(1- 1/2logN)• For large N, E ~ p*N• For < 1% expected error, we must have < 1%
chance of misassignment
4-19-00 Rosenthal - 18
Error from Motion
4-19-00 Rosenthal - 19
Motion Errors
• What errors can motion cause in a structured light system?
• Two components of motion:– Along a pixel ray
– Pixel to pixel Moving object in a structured light system
4-19-00 Rosenthal - 20
Pixel to Pixel Motion
• Similar to stripe assignment error - “random” bits are flipped in affected pixels
Moving object in a structured light system
4-19-00 Rosenthal - 21
Motion on a Pixel Ray
• Motion along a pixel ray causes different stripes to be reflected over time
• Final pixel code is composed of bits from different positions
Motion along a pixel ray traverses projected stripes over time
4-19-00 Rosenthal - 22
Motion on a Pixel Ray
• Magnitude of error depends upon initial position and velocity
• Very large errors near edges in broad stripes (high order bits)
4-19-00 Rosenthal - 23
Motion on a Pixel Ray
Absolute error relative to final object position - note the chaotic dependence upon position and velocity
4-19-00 Rosenthal - 24
Structured Light in Practice
• Office of the Future uses it for surface measurement
• Augmented Reality Biopsy project has tested it for skin surface measurement and for internal surgery
• Videos!
4-19-00 Rosenthal - 25
Conclusions
• Four major classes of error in structured light:– Error intrinsic to ideal system - affects spatial resolution
– Lighting and reflection error - some regions of the scene will be unmeasurable
– Stripe assignment error - need reliable methods for binary categorization
– Motion error - yields large, unpredictable errors
• Despite these problems, structured light is useful and widely available