yannick franckenchris hermansphilippe bekaert hasselt university – tul – ibbt expertise centre...

Yannick Francken Chris Hermans Philippe Bekaert

Hasselt University – tUL – IBBTExpertise Centre for Digital Media, Belgium

{firstname.lastname}@uhasselt.be

Goal

Geometric calibration of a camera w.r.t. a screen

• Vision based HCI

• 3D reconstruction

Motivation

[Chen et al., SPIE 2002] [Gorodnichy et al., SPIE 2002]

[Francken et al., CVPR 2008] [Nehab et al., CVPR 2008]

• Planar mirror

Related Work

[Funk and Yang, CRV 2007][Bonfort et al., ACCV 2006]

• Planar mirror• Spherical mirror

– Corner reflections

Related Work

[Tarini et al., Graphical Models 2005]

• Planar mirror• Spherical mirror

– Corner reflections– Edge reflections

Related Work

[Francken et al., CRV 2007]

• Planar mirror• Spherical mirror

– Corner reflections– Edge reflections– Surface reflection

• Increased accuracy•Less manual interventions•Robust screen reflection

detection

Our Approach

Concept

1. Mirror detection

2. Screen pixel labeling

3. 3D reconstruction

Mirror detection

1. Internal camera parameters K2. Background subtraction3. Edge extraction4. Ellipse fitting5. 2D ellipse to 3D sphere

Screen pixel labeling

Screen pixel labeling

Screen pixel labeling

Screen pixel labeling

Screen pixel labeling

Screen pixel labeling

Screen pixel labeling

Screen pixel labeling

Reflection mask

Reflection mask

Reflection mask

Reflection mask

Reflection mask

Reflection mask

Reflection mask

Reflection mask

3D reconstruction

• Reflected rayintersections

• Plane estimation• Grid estimation

Known parameters:

• Reflected rayintersections

• Plane estimation• Grid estimation

Result: 2D pixel u 3D location x x = M . u

3D reconstruction

Solution: Find 2D – 2D similarity transform

Overview

x = M . u

• Error as function of pattern refinement

Results

• Accuracy– Ground truth

– [Francken et al., CRV 2007]

– Our approach

• Error as function of sphere combinations

Results

• Error as function of sphere combinations

Results

• Error as function of sphere combinations

Results

• Error as function of sphere combinations

Results

• Screen-camera calibration using Gray codes

– Increased accuracy

– Less manual interventions

– Robust screen reflectiondetection

Conclusion

• Gradient patterns– Speed!– Quality?

• Camera defocus– Which patterns

are robust?

Future Work

Questions?

yannick.francken@uhasselt.be http://research.edm.uhasselt.be/~yfrancken

x = M . u