designing gaze interaction systems
DESCRIPTION
Designing gaze interaction systems. John Paulin Hansen The IT University of Copenhagen Denmark. Short Message System (SMS). Mobile Maximum of 160 characters Very popular among the 15-24 age group Asynchronous Discreet Language culture with slang and abbreviations. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/1.jpg)
Designing gaze interaction systems
John Paulin HansenThe IT University of
CopenhagenDenmark
![Page 2: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/2.jpg)
![Page 3: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/3.jpg)
Short Message System (SMS)
• Mobile• Maximum of 160 characters• Very popular among the 15-24 age
group• Asynchronous• Discreet• Language culture with slang and
abbreviations
![Page 4: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/4.jpg)
% SMS among mobile phone users
Source: Smoreda, Z., Thomas, F. (2000): ”Use of SMS in Europe”
![Page 5: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/5.jpg)
The problem with SMS
• Low efficiency (WPM = 7.93)
• Difficult to learn• Few and small
buttons• Mode-dependent• No mental models
to build on
![Page 6: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/6.jpg)
Solutions to mobile texting
• Virtual QWERTY • Speech recognition • Hand printing (Graffiti)• OPTI, FOCL, ABC-
tapping etc. • BUT:
– still not truly mobile and discreet
– may be difficult to learn– not always 100 %
reliable– Far from QWERTY-
efficiency
?
![Page 7: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/7.jpg)
Gazetalk
• Digital cameras (web and Firewire)
• Type-to-talk • Browser• E-mail• Adaptive word prediction• Freeware and Open
Source• Danish, English and
Japanese versions by 2003
![Page 8: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/8.jpg)
Gaze based interaction
• First system for people with disabilities in 1990 (Frey et. al.)
• Several efficient systems available today - used by e.g. people with ALS
• Why then yet another system?
![Page 9: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/9.jpg)
Problems with existing systems
– Quite a large portion of users is not able to get a sufficiently good calibration
– Most systems are only for stationary, indoor use
– Requires a rather complicated installation by computer experts
– The price is prohibitive (>50.000 d.kr. + PC) which prevents potential users from testing whether this interaction mode would work for them
![Page 10: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/10.jpg)
Question• Date: Tue, 06 Feb 2001 14:41:07 +0200
• Hi there
• I stay in South Africa and my father has had a brain stem stroke. • He cannot move or speak. He can communicate with his eyes. • We are currently using an alphabet chart, but it takes too long • and he gets tired quickly.
• I'm looking for some type of EyeMouse. Something we can plug • into a PC, Design like a virtual keyboard.
• Can you please advise me. • I will appreciate any help.
• Thanks • Nuno Lourenco
![Page 11: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/11.jpg)
Answer• Dear Nuno,
• I'm sorry to tell you but eye trackers that are accurate enough for your • father to indicate at which letter your father is looking at are much too expensive • and are not available for private use. • If your father still can hear, you can ask him to look left or right or up or down • under certain conditions. It may work as a yes or no. • • • • -- • Lo Bour, PhD
• Department of Neurology/Clinical Neurophysiology, H2-222• Academic Medical Centre, University of Amsterdam
![Page 12: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/12.jpg)
Video Demo
QuickTime™ and aDV - PAL decompressorare needed to see this picture.
![Page 13: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/13.jpg)
• UI test with eye tracking system
QuickTime™ and aDV - PAL decompressorare needed to see this picture.
![Page 14: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/14.jpg)
Freeware versions 2003
Danish Japanese English
![Page 15: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/15.jpg)
Design Process
• 2000: Design students made prototypes
• 2001: Usability students tested first version with limited functionality
• 2002: ALS patients to test version 1.0
![Page 16: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/16.jpg)
Big button browser
![Page 17: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/17.jpg)
Big button browser
![Page 18: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/18.jpg)
Low res head- & eye tracking
QuickTime™ and aCinepak decompressorare needed to see this picture.QuickTime™ and aCinepak decompressorare needed to see this picture.
Color: Mean-shift
Shape: Active appearence models
![Page 19: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/19.jpg)
Gaze determination
Web cam Video cam
![Page 20: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/20.jpg)
Ongoing Experiments
• 12 Japanese students• 4 hours of training• Typed 20 JP
characters per minute ~10 WPM with gaze (tested with ”Quick glance”)
• 12 % errors - 3 % with mouse
sec/activation
0,400
0,600
0,800
1,000
1,200
1,400
1,600
m-click m-dwell eye-dwell
![Page 21: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/21.jpg)
Comments from novice users of gaze dwell time
typing:• ”A bit difficult to get used to not
dwelling at un-intended buttons .. reacts in panic”
• ”Difficult to orient yourself without activating something”
• ”It can be hard to keep starring at the buttons you wanth”
![Page 22: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/22.jpg)
Loose coupling of gaze and pointer
• 35 mouse activations analysed:– 40% gaze remaind at
dwelling key– 40% went to other
key durring dwell periode
– 12% went to text field durring dwell periode
– 8% the eye was at another key durring all the dwell periode
![Page 23: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/23.jpg)
Theoretical upper limit
• ”Perfect” Eye tracker and trained user, utilizing all the predictions :– 2 characters per activation– 1,2 seconds per activation (500 ms
dwells. cf. Japanese results)
= 100 characters per minute = 20 wpm, = 24 wpm, 300 ms dwells
![Page 24: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/24.jpg)
Reaching for the moon?
• People talk at 160 to 200 WPM !!!!• Non-typist write at 20 - 35 WPM• AAC typing systems 2 - 26 WPM (1977-
1985)• SMS (Multitap) at 8 WPM • Eye typing systems from 1 to 7 WPM• Birger Bergmann Jeppesen peaked at 12
WPM (Wiwik + QuickGlance)
![Page 25: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/25.jpg)
How fast is fast enough?
• ”It should be clear that speed, in itself, should not be an object, but rather proficiency and ease of operation. On the other hand, when there is a lot to say, or when there is a need for extensive personal interchange, a minimum speed of 25 - 30 wpm is really needed to keep the thought moving”
William G. Pierpont(2001): ”The art & skill of Radio-Telegraphy”
![Page 26: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/26.jpg)
Other approaches• Dario D. Salvucci (1999): 822 ms
pr. character, 10 seconds to find the right word in a data base with 1000 words. User performance 9 - 28 wpm.
![Page 27: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/27.jpg)
DASHER
• Developed by David Mackay, Cambridge University, with Stephen Hawking ”in mind”
• 25 WPM after one hour• 34 WPM when expert• Requires a ”Drivers
license”• ”Fast hands-free writing
by gaze direction” Nature 418:838 (August 2002)
![Page 28: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/28.jpg)
User needs
• Fast• Robust• Daily expressions• Mobile• Changes with
progress of diseases
• Affordable
QuickTime™ and aYUV420 codec decompressorare needed to see this picture.
Video cam Web cam
![Page 29: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/29.jpg)
Progress of ALS disease
• Loss of voice• Reduced control of arms and legs• Reduced control of fingers• Reduced head movements• Just eye movements
![Page 30: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/30.jpg)
ALS Scenarios
QuickTime™ and aDV - PAL decompressorare needed to see this picture. QuickTime™ and aDV - PAL decompressorare needed to see this picture. QuickTime™ and aDV - PAL decompressorare needed to see this picture. QuickTime™ and aDV - PAL decompressorare needed to see this picture.
Keyboard Mouse Head Gaze
![Page 31: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/31.jpg)
ALS Scenarios
Keyboardand mouse
Mouse orjoystick
Head Gaze
![Page 32: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/32.jpg)
Mobile Scenarios
Plane Coach Metro Strolling
![Page 33: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/33.jpg)
Classes of Mobile Devices
Mobile phone PDA LaptopPrice $10-$100 Hundreds of $ Thousands of $
Storage <10 MB Hundreds of MB Tens of GBCPU <10 MHz 50-200MHz ~1GHzSize ~3x10 cm ~6x10cm ~20x30cmInput Keys Keys,
touchscreenand pen
QWERTY + pointingdevice
Screenarea
~10 cm2 Tens of cm2 Hundreds of cm2
Purpose Communica-tion
Organizer/ datacollection
Office tasks
![Page 34: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/34.jpg)
Gaze and voice interaction with mobile phones
A. Jameson, Germany research center for artificial intelligence.
![Page 35: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/35.jpg)
Helmet mounted displays
QuickTime™ and aDV - PAL decompressorare needed to see this picture.
![Page 36: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/36.jpg)
Gaze interaction for special tasks
• Industrial inspection
• Field-communication
• Security• Medical
information systems
![Page 37: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/37.jpg)
”Hands-free”-gaze interaction for all
• Private mobile-communication:– Faster than SMS– Note taking during
video conferences and
– Dynamic indication of attention from several participants
![Page 38: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/38.jpg)
Transfer of innovations
ALS- Patient:– No voice
– Just head- or gaze control
– Imprecise pointing
– Acceleration of text-input
Mobile user– Noise or need for
private communication – Hands-free interaction
– Shaky environments or small displays
– Word prediction on mobile units with limited number of keys
![Page 39: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/39.jpg)
Future applications
![Page 40: Designing gaze interaction systems](https://reader038.vdocuments.us/reader038/viewer/2022102615/568147b5550346895db4fbd1/html5/thumbnails/40.jpg)
www.gazetalk.org:The IT-University of CopenhagenTokyo Institute of TechnologyThe Danish Muscular Dystrophy Association…and hopefully more to come
Exhibition at the IST Conference, Copenhagen,November 4 -6 , 2002.