Giuseppe Schirripa Spagnolo, Lorenzo Cozzella, Carla SimonettiDipartimento Ingegneria Elettronica, Università di Roma Tre, Italy e-mail: cozzella@ uniroma3.it - schirrip@uniroma3.it

17 - 21 May 2010 At Monte Verita, Locarno, Switzerland

IR FRINGE PROJECTION

FOR

3D FACE RECOGNITION

Overview

Face Recognition problem

SET-UP

Phase Unwrapping and inconsistent region

3D dataset allignement

Conclusion

Face Recognition

“ There is no landscape that we know as well as the human face. The twenty-five-odd square inches containing the features is the most intimately scrutinized piece of territory in existence, examined constantly, and carefully, with far more than an intellectual interest. ”

- by Gary Faigin. Faces are integral to human interaction.

Manual facial recognition is already used in everyday authentication applications

ID Card systems (passports, health card, driver’s license, etc. )

Surveillance operations (for instance in check point)

Advantages of face recognition

Photos of faces are widely used in passports and driver’s licenses where the possession authentication protocol is augmented with a photo for manual inspection purposes; therefore, there is wide public acceptance for this biometric identifier.

Face recognition systems are the least intrusive from a biometric sampling point of view, requiring no contact, nor even the awareness of the subject.

The biometric works, or at least works in theory, with legacy photograph data-bases, videotape, or other image sources

Face recognition can, at least in theory, be used for screening of unwanted individuals in a crowd, in real time.

It is a fairly good biometric identifier for small scale verification applications.

Disadvantages of face recognition

A face needs to be well lighted by controlled light sources in automated face authentication systems

Face currently is a poor biometric for use in a pure identification protocol

An obvious circumvention method is disguise

There is some criminal association with face identifiers since this biometric has long been used by law enforcement agencies

... continue Face Recognition

Imaging

Single image (2D & 3D)

Video sequence

Infrared & near infrared (facial thermogram)

Facial recognition requires 2 steps: Facial Detection Facial Recognition

Typical Facial Recognition technology automates the recognition of faces using one of two 2 modeling approaches: Face appearance

2D Eigen faces 3D Morphable Model

Face geometry 3D Expression Invariant Recognition

Facial Recognition

2D vs 3D

2D face recognition methods are sensitive to lighting, head orientations, facial expressions and makeup.

2D images contain limited information

3D Representation of face is less susceptible to isometric deformations (expression changes).

3D approach overcomes problem of large facial orientation changes

3D Face Recognition

In this work, the 3D model of the face is achieved by projecting near infrared light modulated by a sinusoidal fringe pattern on the face.

The system

In this work, the 3D model of the face is achieved by projecting near infrared light modulated by a sinusoidal fringe pattern on the face.

... continue The system

Structured light is obtained by the interference of the two fields diffracted by a saw-tooth phase grating. The fringe patterns, distorted by the surface roughness, are captured by a high-resolution image camera.

... continue The system

The fringe patterns, captured by the image camera, are processed with the aid of the Fourier transform analysis and a procedure of unwrapped phase-map able to minimize and to fill holes generate by shadows and facial hair (like beard, mustache).

... continue The system

Phase Unwrapping

Original surface Reconstructed surfaceCurves extracted

Example of reconstruction

The method is experimentally simple, has a low-cost set-up, requires only single image as input, is easy to be integrated in systems of control and access.The system can work in real time (necessity of an only acquisition) and projecting non visible light (no damages to the retina).The system can be easily hidden so that it is difficult to discover.

... continue The system

Face Capture System SET-UP

Set up details can be found in:G. SCHIRRIPA SPAGNOLO, D. AMBROSINI, "Diffractive optical element-based profilometer for surface inspection", Optical Engineering 40, pp. 44-52 (2001)

Face Capture System - SET-UP

Face Capture System - SET-UP

Procedure

Phase Unwrapping and inconsistent region

The case of an image containing regions without phase information

Interpolation of phase data in regions of inconsistent

Phase inconsistencies are handled by excluding invalid pixels from the unwrapping process through the assignment of zero-valued weights;

an initial approximate estimation of the phase data in the region (pixels) of inconsistent data is performed by “standard weighted least-squares algorithm”. At the inconsistent zones is assigned zero weight.

a rewrapping procedure is used to obtain wrapped phase map in region of inconsistence data (so to obtain a phase map without initial inconsistencies).

Interpolation of phase data in regions of inconsistent

Unfortunately, least-squares algorithms produce unwrapped phase data that are not congruent with the original phase. Therefore, rewrapping the obtained phase usually gives a phase map different from the input. To manage this shortcoming, a congruence operation is needed.

To obtain the congruent phase a correlation procedure is used.

In particular, using the phase information “of good quality”, present in the original phase around to the regions of inconsistent data, a iterative procedure that uses the correlation algorithm can be performed. This procedure allows to reconstruct the congruent phase in the zone of inconsistent data.

0 10 20 30 40 50 60 70 80 90 100-25

-20

-15

-10

-5

0

5

10

15

20

Real unwrapped phase

Initial wrapped phase

Esteemed unwrapped phase

Rewrapped phase

Procedure that allows to reconstruct the

congruence

Inconsistent data

with binary mask

Reconstructed region

Interpolation of phase data in regions of inconsistent

an initial approximate estimation of the phase data in the region (pixels) of inconsistent data is performed by “standard” weighted least-squares algorithm. At the inconsistent zones is assigned zero weight.

a rewrapping procedure (plus congruence procedure) is used to obtain wrapped phase map without initial inconsistencies;

a new unwrapped phase map is obtained by the same algorithm used in step-a and a quality map with non-zero weights in the inconsistent zones.

To reduce the problem of the “standard weighted least-squares algorithm” we propose an enhancement of the technique.

Interpolation of phase data in regions of inconsistent

Interpolation of phase data in regions of inconsistent

To reduce the problem of regions of inconsistence, another approach consists to acquire (at the same time with more cameras) more images and subsequently to performed data fusion to produce a unique detailed 3D model.

Align two partially-overlapping meshesgiven initial guessfor relative transform

Interpolation of phase data in regions of inconsistent

Iterative Closest Point (ICP)

Iterative Closest Point (ICP) is an algorithm employed to match two clouds of points. This matching is used to reconstruct 3D surfaces from different scans, to localize robots, etc.

The algorithm is very simple and is commonly used in real‐time. It iteratively estimates the transformation (translation, rotation) between two raw scans.

Inputs: two raw scans, initial estimation of the transformation, criteria for stopping the iteration.

ICP essential steps

Associate points by the nearest neighbor criteria.

Estimate the parameters using a mean square cost function.

Transform the points using the estimated parameters.

Iterate (re‐associate the points and so on).

Aligning 3D Data

How to find correspondences: User input? Feature detection? Signatures?

Alternative: assume closest points correspond

Aligning 3D Data

Converges if starting position “close enough“

Aligning 3D Data

Beginning from the image 3D are individualized the features point that allows to calculate the rigid transformation for the alignment of the image and for the comparison with the database

36

74 23

Extraction of feature points

Conclusion

Advantages : The proposed system uses light source in the near

infrared light spectrum. This will reduce the influence of other light sources.

Therefore, a new phase unwrapping procedure is realized to minimize and to fill holes generate by shadows and facial hair.

The device, in the current configuration, is able to scan human faces in a short time and the face control can be make in "invisible" way.

Besides using infrared radiation, the system avoids risks for the human subject submitted to the control.

Conclusion

Drawbacks : For real-time 3D shape measurement, the whole

processing includes data acquisition, phase wrapping, phase unwrapping, coordinate calculation, 3D rendering, confront with database It is very difficult for a single CPU to accomplish all these procedures parallel computation it is necessary