yannick franckenchris hermansphilippe bekaert hasselt university – tul – ibbt expertise centre...
DESCRIPTION
Vision based HCI 3D reconstruction Motivation [Chen et al., SPIE 2002][Gorodnichy et al., SPIE 2002] [Francken et al., CVPR 2008][Nehab et al., CVPR 2008]TRANSCRIPT
Yannick Francken Chris Hermans Philippe Bekaert
Hasselt University – tUL – IBBTExpertise Centre for Digital Media, Belgium
{firstname.lastname}@uhasselt.be
Goal
Geometric calibration of a camera w.r.t. a screen
• Vision based HCI
• 3D reconstruction
Motivation
[Chen et al., SPIE 2002] [Gorodnichy et al., SPIE 2002]
[Francken et al., CVPR 2008] [Nehab et al., CVPR 2008]
• Planar mirror
Related Work
[Funk and Yang, CRV 2007][Bonfort et al., ACCV 2006]
• Planar mirror• Spherical mirror
– Corner reflections
Related Work
[Tarini et al., Graphical Models 2005]
• Planar mirror• Spherical mirror
– Corner reflections– Edge reflections
Related Work
[Francken et al., CRV 2007]
• Planar mirror• Spherical mirror
– Corner reflections– Edge reflections– Surface reflection
• Increased accuracy•Less manual interventions•Robust screen reflection
detection
Our Approach
Concept
1. Mirror detection
2. Screen pixel labeling
3. 3D reconstruction
Mirror detection
1. Internal camera parameters K2. Background subtraction3. Edge extraction4. Ellipse fitting5. 2D ellipse to 3D sphere
Screen pixel labeling
Screen pixel labeling
Screen pixel labeling
Screen pixel labeling
Screen pixel labeling
Screen pixel labeling
Screen pixel labeling
Screen pixel labeling
Reflection mask
Reflection mask
Reflection mask
Reflection mask
Reflection mask
Reflection mask
Reflection mask
Reflection mask
3D reconstruction
• Reflected rayintersections
• Plane estimation• Grid estimation
Known parameters:
• Reflected rayintersections
• Plane estimation• Grid estimation
Result: 2D pixel u 3D location x x = M . u
3D reconstruction
Solution: Find 2D – 2D similarity transform
Overview
x = M . u
• Error as function of pattern refinement
Results
• Accuracy– Ground truth
– [Francken et al., CRV 2007]
– Our approach
• Error as function of sphere combinations
Results
• Error as function of sphere combinations
Results
• Error as function of sphere combinations
Results
• Error as function of sphere combinations
Results
• Screen-camera calibration using Gray codes
– Increased accuracy
– Less manual interventions
– Robust screen reflectiondetection
Conclusion
• Gradient patterns– Speed!– Quality?
• Camera defocus– Which patterns
are robust?
Future Work
