windows of the soul slide 1 uva - 8/24/98 windows of the soul? windows of the soul? what eye...

Windows of the Soul Slide 1 UVA - 8/24/98

Windows of the Soul?Windows of the Soul?

What Eye Movements Tell Us

About Software Usability

University of VirginiaCharlottesville, Virginia

August 24, 1998


Investigators• Steve Ellis - Principal Investigator

– Bell Labs / Lucent Technologies, Room 1K-411, 200 Laurel Ave, Middletown NJ 07748, [email protected]

• Ron Candrea

– Bell Labs / Lucent Technologies, Room 2D-425, [email protected]

• Jason Misner – Systems Engineering, University of Virginia, Charlottesville, VA,

22903, [email protected]

• Christopher Sean Craig– University of Virginia, [email protected]

• Christopher P. Lankford

– University of Virginia, [email protected]

• Thomas E. Hutchinson – University of Virginia, [email protected]


Goals Of Our Investigation

• Assess the value of eye tracking technology for user interface evaluation and usability testing

• Identify eye tracking technologies most suitable for the usability lab

• Identify improvements in current eye tracking system capabilities for use in the usability lab

• Develop guidelines for purchase and use of commercial eye tracking systems


Previous Eye Movement-BasedPsychological Research and Applications

• Reading Research (Just & Carpenter, 1980; Rayner, 1978)

• Studies of Cognition and Elementary Information Processing (Just & Carpenter, 1976; Rayner, 1984)

• Social Psychology/Interpersonal Communications (Kleinke, 1986)

• Military Applications (Harris & Christhilf, 1980; Birkmire et al, 1993)

• Marketing, Advertising (Krugman et al, 1994)

• Usability Studies (Benel, Ottens & Horst, 1991; Card, 1984)

• Other Ergonomic Studies (Wierda & Maring, 1993)

• Bioengineering, Control (Jacob, 1991)


Criteria for Choosing an Eye Tracking System for the Usability Lab

• Unobtrusive

• Easy and convenient to use

• Few or no restrictions on user population

• Gaze direction accuracy of 0.5º or better (i.e., 1 cm at 20 in).

• Effective tracking range of 30º or better

• High sampling rate

• Inexpensive

• Work with off-the-shelf applications

• Powerful and flexible set of tools for data collection and analysis


The Human Eye


0 1 2 3

Contact Lens

Electro-Oculography

Dual Purkinje Image

Artificial NeuralNetworks

PupilCenter/Corneal Ref

Pupil Tracking

Limbus Tracking

Accurate

Robust

Economic

Unobtrusive

HiMed

Subjective Comparison ofEye Tracking Techniques

Lo


Pupil-Center/Corneal Reflection Method

ERICA System


ERICA System - Continued

A view of the infrared image processed by ERICA. The relative position vector between the center of the bright-eye and glint determines the position of the user’s eye-gaze.


A Pilot Study of Web Page Design

• Evaluate eye tracking data as a user interface evaluation and design tool

• Apply to a representative usability study of Web page design

• Understand how best to collect, analyze and interpret the eye tracking data


Pilot Study Methodology

4 Web page designs 4 tasks performed by all testers

1. Do xyz

2. Do abc

3. Do re me

4. Do nada

16 independent testers

4 testers / page design;

1 page design / tester


Original Web Page Design

• 12 items in web page.

• Each item ranged from3 -12 lines of text with 5 - 9 words/line.

• About half the graphics/images were animated.

• Scroll about 5 times to see entire page. Testers saw only 2.5 items per screen.


Many Hyperlinks Design

• Tripled number of text hyperlinks.


No-Image Web Page Design

• Replaced graphics / images with text labels.

• Original hyperlinks.


Dense Text Design

• Book-like, full-width layout.

• Eliminate graphics / images, two-column appearance and bordered rows.

• Scroll about 2 times to see entire page. Testers saw 5 of the 12 items on one screen.


Tested Web Page Designs


Data Collection and Analysis

• Playback

• Gaze Points - collected raw sampled data

• Gaze Trails

• Dynamically defined analysis measures

– Fixation Points

– LookZones

• Pupil Diameter


Playback

• Real-time “video playback”

• Tracks scrolling events


• About 10 -12 samples/sec.

Gaze Point Analysis —Raw Sampled Data


Gaze Trails


Gaze Trails with Gaze Points

• Overlay the sampled gaze points on the gaze trails

• Gaze trails optional

• Pop-up details on mouse-over each gaze point


Fixation Points - Example 1

• User defined during analysis, after capture. Parameters include:

– Duration

– Area

– Minimum # gaze points

• More accurate indication of tester’s viewing & processing


Fixation Points - Example 2


Fixation Point Parameter Setting


Determining Fixation Points

Property Parameter Value

Minimum Time Duration of a Fixation

200 msec

Maximum Time Between Gaze Points

750 msec

Maximum Diameter of a Fixation

50 pixels

Minimum Number of Gaze Points in a Fixation

3


LookZones


LookZone Analysis

Notice that the subject, searching for the Wireless links, skipped over most pictures and focused on scanning the text blocks for targets.


Bitmap Screen Analysis Options


3D LookZone Analysis In

Bitmaps


Vary Gaze Trail WidthBy Pupil Diameter


Data Analysis - Review of Tasks

• Task 1: Free Browse

• Task 2: Directed Search– What is the name of Lucent’s new on-line technology

magazine (“e-zine”)?

• Task 3: Directed Search– What is the name of the software that lets you search for

patterns in large masses of data?

• Task 4: Directed Search and Judgment– How many links on this site will take you to pages dealing

with “Wireless” technologies, such as that used in cellular phones.


0 10 20 30 40 50 60 70 80 90

Search & Count - W ireless

Search for Data Software

Search for E-Zine

Free Browse

Average in Seconds

Comparison of Tasks

Mean Time to Complete Tasks


Review of Web Site Designs

• Four Screen Layouts

– Original Text Block/Graphics Format

– Many Hyperlinks

– No Images

– Dense Text


0 20 40 60 80 100

Dense Text

No Images

Many Hyperlinks

Original*

Seconds

Comparison of Web Designs

Mean Time to Complete Tasks&

& Average of e-zine and wireless search tasks.

* Note: Testers of “Original” design were least experienced group and probably exaggerated the times on this simpler design compared with testers using the busier “Many Hyperlinks” page.


0 10 20 30 40 50 60 70 80

Dense Text

No Images

ManyHyperlinks

Original

Seconds & E-zine search task P-value = 0.03

Mean Time to First Fixationin Target LookZone&


0 20 40 60 80 100 120 140 160

Dense Text

No Images

Many Hyperlinks

Original

Mean Number of Fixations&

& Wireless search task P-value = 0.37


0 0.1 0.2 0.3 0.4 0.5 0.6

Dense Text

No Images

ManyHyperlinks

Original

Seconds

Mean Average Fixation Duration&

& Wireless search task

P-value = 0.02


0 20 40 60 80 100 120

Dense Text

No Images

Many Hyperlinks

Original

Total Fixations inNon-Target LookZones&

& E-zine search taskP-value = 0.13


0 1 2 3 4 5 6 7

Dense Text

No Images

Many Hyperlinks

Original

Average Questionnaire Ratings

Subjective Comparisonof Web Designs

Highest


Results and Conclusions

• Correlation among most results

– Dense text design elicited best performance

• Eye movement data confirmed and provided insights into testers performance

– Dense text design led to fewer, shorter fixations

• Results should NOT be viewed as conclusive

– Too few subjects, groups differed in skill level

– Some experimental design flaws, confounds


Next Steps

• Replicate the study

– Eliminate the potential confounds

• (Systematic group differences, scrolling, etc.)

– Benchmark subjects’ reading speed

• Expand the study to see if results are generalizable

• Extend eye tracking methodology to other software applications


References• Benel, D. C. R., Ottens, D. & Horst, R. (1991). Use of an eyetracking system in the usability laboratory. Unpublished

paper written for The Human Factors Society 35th Annual Meeting. San Francisco, CA

• Birkmire, D. P., Karsh, R., Barnette, R. D., Pillalamarri, R. & DeBastiani, S. (1993). Eye movements in search and target acquisition. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting (pp. 1305-1309). Santa Monica, CA: Human Factors and Ergonomics Society

• Card, S. K. (1984). Visual search of computer command menus. In H. Bouma & D. G. Bouwhuis (Eds.). Attention and Performance X: Control of Language Processes (pp. 67-96). London: Erlbaum

• Harris, Sr. R. L. & Christhilf, D. M. (1980). What do pilots see in displays? In Proceedings of the Human Factors Society 24th Annual Meeting (pp. 22-26). Santa Monica, CA: Human Factors and Ergonomics Society

• Jacob, R. J. K. (1991). The use of eye movements in human-computer interaction techniques: What you look at is what you get. ACM Transactions on Information Systems, 9 (April), 152-169

• Just, M. A. & Carpenter, P. A. (1976). The role of eye-fixation research in cognitive psychology. Behavior Research Methods and Instrumentation, 8, 139-143

• Just, M. A. & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329-354

• Kleinke, C. L. (1986). Gaze and eye contact: A research review. Psychological Bulletin, 100, 78-100

• Krugman, D. M., Fox, R. J., Fletcher, J. E., Fischer, P. M. & Rojas, T. H. (1994). Do adolescents attend to warnings in cigarette advertising? A eye-tracking approach. Journal of Advertising Research, 34, 39-52

• Rayner, K. (1978). Eye movements in reading and information processing. Psychological Bulletin, 85, 618-660

• Rayner, K. (1984). Visual selection in reading, picture perception, and visual search: A tutorial review. In H. Bouma & D. G. Bouwhuis (Eds.), Attention and Performance X: Control of Language Processes (pp. 67-96). London: Erlbaum

• Wierda, M. & Maring, W. (1993). Interpreting eye movements of traffic participants. Chapter 21 in D. Brogan, A. Gale & K. Carr (Eds.), Visual Search 2 (pp. 287-300)

windows of the soul slide 1 uva - 8/24/98 windows of the soul? windows of the soul? what eye...

Documents