1

Multimodal Interaction

Eye Gaze and Head Movement Tracking

Iris Recognition

Dr Pradipta Biswas, PhD (Cantab)

Assistant Professor

Indian Institute of Science

http://cpdm.iisc.ernet.in/PBiswas.htm

2

What is Eye Tracking & Gaze Control

• Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement.

• Gaze control is about effecting computer action by changing the direction of one’s gaze (eye movement), blinking or dwelling on an object.

2

2

Eye movement

3

4

How Eye Tracking Works

• Most Commonly used technique is Pupil Centre and Corneal Reflection Technique.

• Simple calibration procedure (usually following a shape around screen required for each user.

• Infrared-sensitive video takes rapid pictures of eye.

• Infrared LED illuminates the eye.

• LED reflects small amount of light off the cornea & through the pupil onto the retina.

• Bright pupil allows image processor to locate centre of pupil.

• Tracker can then locate where the person is looking on the screen based on the relative positions of the pupil centre and corneal reflection within the video image of the eye

3

Types of Eye Tracker

• Non-intrusive• Attached to device (e.g.: Facelab)

• Mobile (e.g.: Tobii X series)

• Intrusive• Glass based (e.g.: SMI Eye Glass)

• Head attached

• Lens based (very early models)

• Electrodes (early models)

5

Comparison

• Non-Intrusive• Records natural interaction

• Have issues with ambient illumination, screen size and head movement

• Intrusive• Needs to wear glasses or head mounted device

• Supports head movement

• Works for small and big screen devices

• Mobile phone, big display etc

6

4

Types of Technology

• Infra red based

• Video based

• Electrode / Lens based (early models)

7

Comparison

• Infra red based• Accurate• Needs to install infra red trackers• Costly

• Video based• Less accurate• Works with existing webcam

• Some video based eye trackers need special camera though it is still less costly than infra red ones

• Recent work also investigating use of low cost infrared tracker (e.g.: EyeTribe Technology, $99 infrared ET)

8

5

Types of Applications

9

Passive Eye Tracking

Theory of Visual Perception

6

Points of Fixation

11

Area of interest

12

7

Applications

• Analyzing points of fixation and eye movement to investigate • Areas of interest in a display

• Reading behaviour

• Affect state of user

• Visual impairment • Nystagmus – irregular eye movement

• Design of billboard, traffic sign etc.

13

Active Eye Tracking

Gaze Control Interface

8

Types of Eye Gaze Movement

• Saccades

• Small Pursuits

• Vergance

15

Issues with gaze control

• Strain

• Accuracy

• Selection• Midas Touch Problem

16

9

Multimodal Eye Tracking

• MAGIC System• Selection using mouse

• Eye Tracking and BCI• Selection through imagined action detected through

EEG

• Eye Tracking and Assistive Technology• Selection through single switch scanning

17

Target Prediction for Eye Gaze Target Prediction for Eye Gaze Target Prediction for Eye Gaze Target Prediction for Eye Gaze

Tracking SystemTracking SystemTracking SystemTracking System

18

A Neural Network is used to

detect the phase of movement

(ballistic vs homing)

10

Iris Recognition

19

Introduction

• It is considered to be the most accurate biometric technology available today.

� Iris recognition is a method of

biometric identification and

authentication that use pattern-

recognition techniques based on

high resolution images of the

irises of an individual's eyes .

11

What is Iris ?

The colored ring around the pupil of the eye is called

the Iris

What is Iris ?

The iris is a thin circular diaphragm, which lies between the cornea and the lens of the human eye.

The iris is perforated close to its centre by a circular aperture known as the pupil.

The function of the iris is to control the amount of light entering through the pupil.

The average diameter of the iris is 12 mm, and the pupil size can vary from 10% to 80% of the iris diameter

12

What is Iris ?

� The iris consists of a number of layers, the lowest is the epithelium layer, which contains dense pigmentation cells. The stromal layer lies above the epithelium layer, and contains blood vessels, pigment cells and the two iris muscles.

What is Iris ?

�The density of stromal pigmentation determines the colour of the iris.

�The externally visible surface of the multi-layered iris contains two zones, which often differ in colour An outer ciliary zone and an inner pupillary zone, and these two zones are divided by the collarette – which appears as a zigzag pattern

13

Why the Iris?

�Externally visible highly protected internal organ.�Unique patterns.�Not genetically connected unlike eye color.�Stable with age.� Impossible to alter surgically.�Living Password, Can not be forgotten or copied.�Works on blind person.�User needs not to touch appliances.�Accurate , faster , and supports large data base.

Why the Iris?

14

Why the Iris?

Comparison between cost and accuracy

History of Iris Recognition

1997-1999

1987

1987

1980

�The concept of Iris Recognition was first proposed by Dr. Frank Burch in 1939.

�It was first implemented in 1990 when Dr. John Daugman created the algorithms for it.

�These algorithms employ methods of pattern recognition and some mathematical calculations for iris recognition.

15

Methods Of IRIS Recognition System

�In identifying one’s iris, there are 2 methods for its

recognition and are:

1. Active

2. Passive

�The active Iris system requires that a user be anywhere

from six to fourteen inches away from the camera.

�The passive system allows the user to be anywhere

from one to three feet away from the camera that

locates the focus on the iris.

Iris Recognition Diagram

Image

Acquisition

Iris

Segmentation Normalization

Feature

Encoding

Feature

Matching

Iris Templates

Database

Eye Image Iris Region

Feature points in the

iris region

Iris Template

Identify or Reject

Subject

16

OriginalOriginalOriginalOriginal irisirisirisiris imageimageimageimage presentpresentpresentpresent inininin thethethethe databasedatabasedatabasedatabase

Formation of a circular contour around the iris

The circular contour is concentric with the circular

pupil

Removing the portion of the iris occluded by the

eyelids

Applications

• ATMs

• Computer login: The iris as a living password.

• National Border Controls

• Driving licenses and other personal certificates.

• Benefits authentication.

• Birth certificates, tracking missing.

• Credit-card authentication.

• Anti-terrorism (e.g.:— suspect Screening at airports)

• Secure financial transaction (e-commerce, banking).

• Internet security, control of access to privileged

information.

17

Head Tracker

Related technology to gaze control

Types of Head tracker

• Helmet Based

• Video based• http://www.cameramouse.com

• Attaching Gyroscopic Tracker

• Similar issues with intrusive and non-intrusive head trackers as with gaze control

34

18

Take away pointsTake away pointsTake away pointsTake away points

• Description to a new modality of interaction

• Different types of eye trackers and their comparison

• Introduction to Iris recognition technology

• Demonstration of applications of eye tracker

• Introduction to head tracking

35