Alternative Communication with Ocular Tracking

By: James McIntyre

Introduction

● There are many types of medical issues that inhibit communication

● There are a lot of types of communication for people who can move some muscles, but

some patients can’t do much but move their eyes.

● These immobile patients need to be able to communicate with people and this technology

can allow it.

● In recent years, hardware and software have been created to enable communication

using nothing but the information of where/what someone is looking.

History of ocular tracking

● The study of eye movements started in the late 1800s.

● Progressed from horizontal motion to two dimensions.

● First head mounted eye tracker was invented in 1948, before this the head was restricted

while tracking the eyes focus. This used a small white speck placed into the eye before the

tracking.

● In the 1970s we began realizing that certain light reflections can help us track the eye

separately from head movement, this really paved the way for modern eye tracking.

How it works - Software

● A way of recording and responding to “inputs” from eye tracking hardware is using the

familiar program MATLAB.

● One specific software that incorporates MATLAB for this is The Eyelink Toolbox (Free

online download with subscription) which uses two programs, MATLAB, and

Psychophysics toolbox.

● My favorite Eye tracking device company, Tobii, also has a downloadable program to help

people figure out how to incorporate MATLAB with their eye tracking hardware.

Short MATLAB Program That Uses the Eyelinkand Psychophysics Toolboxes to Create a Simple Gaze-Dependent Display

How it works - Hardware

● Most commercial eye trackers measure point-of-regard by the corneal reflection/pupil

center method.

● An infrared camera can emit a light in order to see the pupil light up, and see a corneal

reflection, and sense where the eye is looking based off of the relation of these two things

● This image shows how the camera can tell exactly where the eye is focused on by the

reflection. →

How it works - Hardware

● There are many types of devices that can be used to track the patient's gaze.

● There are many types of ocular tracking devices:

○ Wearable

○ Screen mounted

○ Screenless

● These are all used for different things, mainly research.

● The one most used for communication is the screen mounted

How it works - Hardware

● The screen mounted eye tracking device work best, because there can be so many options

of what the user can do, such as verbalize words, enabling communication.

● This can also be used to communicate with things such as the climate control of the room,

the shades, the lights, the television, or even with the internet to surf the web or send

emails or skype and things like this.

How it works - Hardware

● The best device on the market right now was released in 2013 and is called the

Tobii ATI I-series, for augmentative and alternative communication (AAC).

● This device is basically a tablet, with a built in eye tracking device, and can be connected

to things like the thermostat and the lights.

● It costs about $4000-$9000 which is not super expensive considering how much it can

improve a patient's life.

● Image of this device →

Tobii ATI I-15 (15 inch)

Issues

● It is difficult to get the manual command or “click” of a mouse, because if it were a blink, it

would be affected by the naturalness of the user's blink.

● But if you have the focus automatically call for a command, you start to see a “Midas

Touch” effect, as in, everywhere you look, you issue a command, which usually is not

intentional.

● We can program the computer system to react accordingly to smaller or larger eye

gestures, trying to not disrupt the user unintentionally and make each input more precise.

Issues

● These devices can be coupled with another input device such as a mouse or button if the

user is capable of using one.

● These devices are also unreliable and break very easily.

● The nicest and most expensive eye tracking devices need multiple repairs per year as well

as needing regular maintenance of the hardware.

Questions?

REFERENCES

[1] Jacob, Robert J.k., and Keith S. Karn. “Eye Tracking in Human-Computer Interaction and Usability Research.” The Mind's Eye, 2003, pp. 573–605., doi:10.1016/b978-044451020-4/50031-1.https://pdfs.semanticscholar.org/5f1f/69d3f8807a5abc490955823ecfec55b15bb9.pdf[2] Cornelissen, Frans W., et al. “The Eyelink Toolbox: Eye Tracking with MATLAB and the Psychophysics Toolbox.” Behavior Research Methods, Instruments, & Computers, vol. 34, no. 4, 2002, pp. 613–617., doi:10.3758/bf03195489.https://link.springer.com/content/pdf/10.3758%2FBF03195489.pdf[3] Poole, Alex, and Linden J. Ball. “Eye Tracking in HCI and Usability Research.” Encyclopedia of Human Computer Interaction, pp. 211–219., doi:10.4018/978-1-59140-562-7.ch034.https://books.google.com/books?id=h9iZh_I1YREC&lpg=PA211&dq=eye%20tracking&lr&pg=PA211#v=onepage&q&f=false[4] Jason S.Babcock - Jeff B.Pelz - Proceedings of the Eye tracking research & applications symposium on Eye tracking research & applications - ETRA'2004 - 2004https://dl.acm.org/citation.cfm?id=968386[5] Majaranta, Päivi, and Kari-Jouko Räihä. “Twenty Years of Eye Typing.” Proceedings of the Symposium on Eye Tracking Research & Applications - ETRA '02, 2002, doi:10.1145/507075.507076.https://dl.acm.org/citation.cfm?id=507076[6] Shepherd, Tracy A., et al. “Reliability of Speech Generating Devices: A 5-Year Review.” Augmentative and Alternative Communication, vol. 25, no. 3, 2009, pp. 145–153., doi:10.1080/07434610902996104.http://www.tandfonline.com/doi/citedby/10.1080/07434610902996104?scroll=top&needAccess=true[7] “Tobii Unveils New Family of Eye-Controlled AAC Devices, Gives Voice to Those With Communication Challenges.” Tobii.com - Tobii Is the World Leader in Eye Tracking, Tobii, 31 Aug. 2015, www.tobii.com/group/news-media/press-releases/tobii-unveils-new-family-of-eye-controlled-aac-devices-gives-voice-to-those-with-communication-challenges/