Eye Tracking MethodologyLuke Li

From digitaltrends.com

Agenda What is eye tracking?

◦ Terminology◦ Hypothesis

Design◦ When to use eye tracking◦ Sample task objectives

Prepare Conduct Analyze and report

◦ Example: Google vs. Bing◦ Statistical metrics◦ Example: Gazing at faces

What is eye tracking?“Eye tracking is simply following the trail of where a person is looking.”

---- Nielson & Pernice

From Goldberg & Tang, 2011

What is eye tracking?Terminology

From Goldberg & Tang, 2011

We see the world through our eyes. Our eye do not move in one smooth and panning way. Instead, they move in spurts and rests between each movement.

When the eye is resting on something, it’s called a fixation

The rapid movements from one fixation to the next are calledsaccades

Eye tracker tracks the time, duration and location of our eyes’ fixations and the saccades between fixations. During saccades, we are blind. We see only during fixations, while the eye is holding still.

What is eye tracking?Hypothesis

However, it holds true in experimental

scenarios when participants are looking at visual stimulus: people usually pay attention to and think about what they are looking at.

Throughout the years, one of the strongest validations for the power of eye tracking is the Mind-Eye Hypothesis (Just & Carpenter, 1980):

Is that always the case in real life? Not necessarily… people might be looking at the traffic light while thinking about their upcoming dinner.

What people are looking at What people are thinking about≈

From Goldberg & Tang, 2011

DesignWhen to use eye tracking?

• When you are in later stages of product development

• Working prototype

• When your testing purpose is to diagnose usability issues

• When you are comparing different designs or trying to reach a consensus

• When you want to learn beyond users’ self-report

• When you care about the process more than the task results

DesignWhen to use eye tracking?

• When you have participants and resources at your disposal

• 5-10 participants for qualitative studies, and 20-30 participants for quantitative studies

• Eye Tracking only allows single sessions

• When the eye tracker equipment and staff are available to help

• On-site or conference sessions; not recommended for remote sessions

DesignWhen to use eye tracking?

• When you can allocate sufficient time to the project and coordinate with others.

• 6 weeks (with some eye tracking experience) to up to 10 weeks (without any eye tracking experience)

Week Objectives

1 Learn about relevant eye tracking knowledge and previous eye tracking studies

2-3Design research questions, tasks and possible analysis metrics in the study. Set up meetings

with eye tracking experts on the team or other researchers to get advice and guidance.

4 Prepare the testing stimulus of your study. Create tests and tasks in eye tracking software.

5-6 Pilot test your study and iterate your study design. Recruit participants.

7-8 Conduct your study with on-site participants

9-10 Analyze all the eye tracking data you collected, and generate result deliverables

DesignSample task objectives

• Research Question Guide• E.g., you are interested in how users use this UI component:

• How long does it take for users to see this UI component?

• How long does it take for users to go from seeing the UI component to actually clicking it? (Uncertainty Time)

• E.g., you are interested in what areas on the UI attract users’ attention?

• What areas do users spend most of their time looking at?

• Do different users have similar browsing / scanning strategies?

• Do different strategies associate with user backgrounds?

Prepare

• Set up testing environment in eye tracker (Tobii eye tracker example)

• Design eye tracking tasks and set up analysis metrics accordingly

• Plan specific testing procedures (intro, calibration, test, questionnaires)

Conduct

• Conference or laboratory setting• Typically the first 2-3 slides are general

introduction of the experiment

• A pilot test is needed in order to minimize the learning effect

• Each task should be preceded by an instruction screen, often followed by a briefly presented fixation screen

• Minimize talking as it may result in head movement (in conflict with think aloud protocols)

Analyze and report

• Familiarize yourself with eye tracking analysis concepts• Fixations & Saccades

• Area of Interest (AOI) Analysis

• Visits

• Replay and Segment your data• Replay to view dynamic gaze recording

• Use "Visualization" to get a general qualitative sense of the eye tracking data

• View the Gaze Plot and Heat Map

• Use statistical analysis metrics to analyze your eye tracking data

Analyze and report

• Analysis (Tobii eye tracker example)

• Replay Scan-path• On the bottom of the page, there

is a blue timeline. Drag the right-hand slider of the timeline back and forth to reveal the scan-path strategy.

• Replay Heat Map• Click the Heat Map icon. Make

sure multiple participants are selected.

Analyze and report Example: Google vs. Bing (2009)

Source: User Centric, 2009

Source: User Centric, 2011

Analyze and report Example: Google vs. Bing (2011)

Analyze and reportStatistical metrics

• How long does it take for users to see the AOI(s)?• Time to first fixation

• On average, how long are users' fixation on the AOI(s)?• Fixation duration

• How many times do users fixate on the AOI(s) in total? • Fixation count

• On average, how many fixations do users have before seeing the AOI(s)?

• Fixation before

• How long is the “uncertainty time”? • Time from First Fixation to Next Mouse Click

Analyze and report Example: Gazing at faces

From Li & Tottenham, 2011

Thank you for your attention!