Eye Tracking the User Experience of

Mobile: What You Need to Know

Jon Strohl, David Hawkins

October 17, 2014

User Focus| Washington, D.C.

@forsmarshgroup

2

• Why mobile is important

• Advantages and limitations of eye

tracking

Agenda

• Case study

• How to eye track the UX of mobile

@jonstrohl @uxhawk #userfocus2014

3 http://www.pewinternet.org/three-technology-revolutions/

http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/

Technology Revolutions

% o

f A

me

rica

n a

du

lts

90% of American adults have a cell phone

58% have a smartphone

42% have an tablet

32% have an e-reader

@jonstrohl @uxhawk #userfocus2014

4 http://www.pewinternet.org/2013/09/19/cell-phone-activities-2013/

% o

f A

me

rica

n a

du

lts

% o

f A

me

rica

n a

du

lts

Cell Phone Activities @jonstrohl @uxhawk #userfocus2014

Traditional UX research is

good at explaining what

people say and do, not

what they think and feel.

Why should we use eye tracking?

@jonstrohl @uxhawk #userfocus2014

6

UX Data

SATISFACTION

& DIFFICULTY

RATINGS

MODERATOR

FOLLOW-UP

VERBAL

RESPONSES

REAL-TIME

+/- DIAL

TIME

ON PAGE/

TASK

REACTION

TIME

SELECTION/

CLICK

BEHAVIOR

SUCCESS/

FAIL

RATE

CONVERSION

RATE

ELECTRODERMAL

ACTIVITY

(EDA)

FACIAL

EXPRESSION

ANALYSIS

EYE

TRACKING

OBSERVATIONAL SUBJECTIVE

IMPLICIT BEHAVIORAL

ANALYSIS

PUPIL

DILATATION

@jonstrohl @uxhawk #userfocus2014

7 *Gazeplot of four participants when provided the task: “Let’s say you want to compare

Eye-Tracking Limitations

• A tool, not a methodology

• Task dependent

• Does not necessarily add

value for all research

questions

• Time consuming

• Higher costs

@jonstrohl @uxhawk #userfocus2014

8

• Versatile tool

• Informs us about:

– Engagement

– Difficulty

– Findability

– Processing order

– Comprehension

Eye Tracking Advantages

• Great proxy measure for how

attention is allocated

@jonstrohl @uxhawk #userfocus2014

9

*Heatmap generated by using the number of fixations that occurred in the first thirty seconds after providing the task: “Let’s say

you want to enter a TV show that you just watched into the app. Please enter a TV show you recently watched.”

Eye tracking paper Compare eye movements

across devices

Tablet App

Homepage Tablet Web Homepage Android App Homepage

He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design.

Morgan Kaufmann.

@jonstrohl @uxhawk #userfocus2014

10 He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design.

Morgan Kaufmann.

Users read what they need to read

*Gazeplot of fixations that occurred when participant was on the username entry page while completing the task: “Let’s say you

want to enter a TV show that you just watched into the app. Please enter a TV show you recently watched.”

@jonstrohl @uxhawk #userfocus2014

Issue: Vague error messages increase visual workload.

11

UX Best Practice: Specifically explain the error, and place the messages near where the error

occurs so the user can quickly fix the error and move on.

• “How do I advance to the

next screen?”

• “It seems like it's stuck on

the screen.”

M

Gaze Plot: After getting an

error message, the

participant had to search all

over the screen to find the

missing field.

Gazeplot of fixations that occurred after the participant clicked “Ok.”

Report with other data sources @jonstrohl @uxhawk #userfocus2014

12

Modern eye tracking

How to set up a mobile eye tracking study

13

Do I need eye tracking?

Think about measures

Select your eye tracker Calibrate

Collect Data Cleanse &

Prepare Data

Visualize

Qual

Analyze

Quant

Eye Tracking Process @jonstrohl @uxhawk #userfocus2014

14

*Fixation count heatmap during the task: “Let’s say you are interested in comparing the costs of several colleges. Please compare the cost of schools that you are interested in attending.”

*Bojko, Aga (2013). Eye Tracking the User Experience: A Practical Guide to Research. Rosenfeld Media.

Measures of attraction

Area noticeability measures

Percentage of participants

who fixated on an AOI

Number of fixations prior to first fixation

on an AOI

Time to first fixation on an

AOI

Area interest measures

Number of fixations on an

AOI

Total dwell time on an AOI

Percentage of time on an AOI

Emotional arousal

measures

Pupil diameter

@jonstrohl @uxhawk #userfocus2014

15 *Bojko, Aga (2013). Eye Tracking the User Experience: A Practical Guide to Research. Rosenfeld Media.

Measures of performance

Target findability measures

Percentage of participants

who fixated on the target

Number of fixations prior to first fixation on

the target

Time to first fixation on the

target

Target recognizability

measures

Number of gaze visits to the

target prior to target selection

Time from first fixation on

target to target selection

Cognitive processing measures

Average fixation duration

Mental workload measures

Pupil diameter

@jonstrohl @uxhawk #userfocus2014

16

Do I need eye tracking?

Think about measures

Select your eye tracker

Calibrate System

Collect Data Cleanse &

Prepare Data

Visualize

Qual

Analyze

Quant

Eye Tracking Process @jonstrohl @uxhawk #userfocus2014

17

Devices to Test

18

Choosing an eye tracker @jonstrohl @uxhawk #userfocus2014

19

Do I need eye tracking?

Think about measures

Select your eye tracker

Calibrate System

Collect Data Cleanse &

Prepare Data

Visualize

Qual

Analyze

Quant

Eye Tracking Process @jonstrohl @uxhawk #userfocus2014

20

Calibrate the system @jonstrohl @uxhawk #userfocus2014

21

Do I need eye tracking?

Think about measures

Select your eye tracker

Calibrate System

Collect Data Cleanse &

Prepare Data

Visualize

Qual

Analyze

Quant

Eye Tracking Process @jonstrohl @uxhawk #userfocus2014

22

Prepare data @jonstrohl @uxhawk #userfocus2014

23

Cleanse data

Original data Final data

Start with original data that was recorded

Outliers Capture

rate

issues

Offsets

Check for outliers in the data

Check for capture rate issues

Check for offsets

Data is ready to be analyzed

@jonstrohl @uxhawk #userfocus2014

24

Do I need eye tracking?

Think about measures

Select your eye tracker

Calibrate System

Collect Data Cleanse &

Prepare Data

Visualize

Qual

Analyze

Quant

Eye Tracking Process @jonstrohl @uxhawk #userfocus2014

25

Create gaze plots @jonstrohl @uxhawk #userfocus2014

26

Create heat maps @jonstrohl @uxhawk #userfocus2014

27

Do I need eye tracking?

Think about measures

Select your eye tracker

Calibrate System

Collect Data Cleanse &

Prepare Data

Visualize

Qual

Analyze

Quant

Eye Tracking Process @jonstrohl @uxhawk #userfocus2014

28

Create Areas of Interest (AOIs)

@jonstrohl @uxhawk #userfocus2014

29

Pros Cons

• Versatile

• Informs us about:

― Engagement

― Difficulty

― Findability

― Processing order

― Comprehension

• Time consuming

@jonstrohl @uxhawk #userfocus2014

• Capture more of the UX

• Higher cost

• Mobile device stand

changes the natural

environment

• Won’t answer all

research questions

Thank you!

• Twitter: @forsmarshgroup

• LinkedIn: http://www.linkedin.com/company/fors-marsh-group

• Blog: www.forsmarshgroup.com/index.php/blog

Jonathan Strohl

@jonstrohl

jstrohl@forsmarshgroup.com

User Focus 2014 | Washington, DC