Usability and Eye Tracking

Marco PretoriusUsability Manager & Researcher

UNISA: School of Computing

Agenda• Introduction

– What is usability?

– User-centered design

– Eye tracking

• Benefits and ROI

• UNISA usability laboratory

• Studies and results

• Other Interesting figures and examples

• Questions

Introduction

What is usability?

• Lack of usability:

– Play video

What is usability?

• Usability means making products and

systems easier to use, and matching them

more closely to user needs and

requirements

• ISO9241: Usability is the

– effectiveness

– efficiency and

– satisfaction

– with which specified users

– achieve specified goals

– in particular environments

What is usability?

• Effectiveness

– Can users achieve what they need by using the

product?

• Ease of learning

– How fast can a user who has never seen the interface

learn to use it?

• Efficiency of use

– How fast can users complete the task?

• Memorability

– Can users remember enough to reuse the interface

effectively?

• Error prevention

– Can users complete tasks without making errors?

• Satisfaction

– How much does the user like using the system?

User-centered design

• ISO 13407: User-centered design is

characterised by:

– the active involvement of users and a

clear understanding of user and task

requirements

– an appropriate allocation of function

between users and technology

– the iteration of design solutions

User-centered design

• Goal

– To develop easy-to-use products

– that lead to increased user satisfaction

– meet your organizational or business objectives

• Difference between other design

philosophies

– user-centered design tries to optimize the user

interface around how people can, want, or need

to work

– rather than forcing users to change how they

work to accommodate the system or function

User-centered design

• ISO13407

• 4 essential activities in UCD

http://www.upassoc.org/usability_resources/about_usability/

what_is_ucd.html

User-centered design

• 1. Specify context of use

– Identify the people who will use the product, what

they will use it for, and under what conditions

they will use it

User-centered design

• 2. Specify requirements

– Identify any business requirements or user goals

that must be met for the product to be successful

User-centered design

• 3. Design

– Producing designs and prototypes

– This part of the process may be done in stages,

building from a rough concept to a complete

design.

User-centered design• 4. Evaluation

– The most important part of this process is that

evaluation - usability testing with actual users - is

as integral as quality testing is to good software

development

User-centered design

• The stages are carried out in an iterative

fashion

– Cycle repeated until the project's

usability objectives have been attained

– Critical - participants accurately reflect

the profile of your actual users

– Talk directly to the user at key points in

the project to make sure the site will

deliver upon their requirements

User-centered design

• Methods used:

– Requirements and data gathering

• Interview

• User observation

• Contextual inquiry

• User group meeting

• Focus groups

• JAD sessions

• Surveys

• Bulletin boards/discussion groups

• Web logs

User-centered design

• Methods used:

– Usability testing

• Task walkthroughs

• Performance-based tests

• Heuristic evaluations

• Preference tests

• A-B testing

What is usability?

• Usability testing involves

– measuring the performance of users on

tasks with regard to

• the ease of use,

• the task time, and

• the user‟s perception of the experience

• of the product, software application, website

or systems

– Based on Performance

– Purpose: Feedback

What is usability?• What Usability testing is not

– Not market research – gathering opinions

– not just a milestone to be checked off on the project

schedule

– not finished when the last participant leaves

• Involves systematic observation under controlled

conditions to determine how well people can use the

product

– watching people trying to use something for its

intended purpose

• Consider the findings, set priorities, and CHANGE

the prototype or site based on what happened in the

usability test

• A technique to determine eye movement

and eye fixation patterns of a person

• The human eye moves by alternating

between

• Saccades• Quick movement of the eye

• Move focus from one area to the next

• Fixation• Time spent looking at the newly found area

Eye tracking

Benefits and ROI

Benefits and ROI•63% of software projects exceeded their estimates

because (Lederer and Prassad ‟92)

•Frequent request for changes by users

•Overlooked tasks

•Users‟ lack of understanding of their own tasks

•Insufficient user <> analyst communication

•80% of software life cycle costs occur during

maintenance phase (Pressman „92)

•80% of maintenance comes from unforseen/unmet user

requirements (Martin „83)

•60% of maintenance phase is due to re-work because

user requirements were not clear at start (Standish „83)

•Benefits for an organisation:

•Save on development and maintenance by:

•Saving on development costs

•Decreasing the development time

•Reducing maintenance costs

•User-centred design

•Usability testing very early and throughout

development

•With a rigorous user-centered methodology, you

can ensure successful, on-time delivery – and

avoid the rework that's necessary later if you

don't talk to users in the first place

Benefits: usability

•Benefits for an organisation:

•Increase Total Revenue by increasing the

number of:• Transactions

• Conversion and hence turnover

• Returning customers

• New customers through word of mouth

advertising

Benefits: usability

•Benefits for an organisation:

•Stimulate the use of the website by:•Increasing the success ratio for a visitor

•Decreasing the number of errors and show

stoppers

•Increasing efficiency for users (less time to

complete a task)

•Increasing users satisfaction

Benefits: usability

•Benefits for a user:

•Understand a site faster and achieve their

goals faster

•Have a positive experience using the

website

•Develop trust in the company

•Perform tasks without help from telephone

or e-mail •http://www.2c.nl/en/what_is_usability/advantages_usability_test.php

Benefits: usability

•Usability and eye tracking:

•All the benefits of traditional usability results

•Eye tracking can add new and interesting insights

•Provide insights that are not available from traditional usability testing methods

•Scan paths; Pattern of fixations

•Time spent looking at various display elements

•Insight into deployment of visual attention•Participant strategies

Benefits: combination

Benefits - Examples:•Human Factors International

•Staples.com•67% more repeat customers

•31-45% reduced drop-off rates

•10% better shopping experience

•80% increased traffic

•Increased revenue – sales up by 491%

•Utility company in Canada – Intranet•Training hours from 8 hours to 15 minutes

•Energy company in US – Intranet•300 support calls per day to ZERO

UNISA usability lab

UNISA usability lab

• A usability laboratory is a state-of-the-art

facility designed to support the observation

of HCI

• Users are brought into a controlled

environment, in which they are asked to do

specific tasks within specific timeframes

• Evaluators

– observe the problem(s) the participant might have

– videotape the participant

– analyse the data

UNISA usability lab

• Observer room

• Participant room

• Separated by one-way mirror

• Tobii 1750 eye tracker

• Audio and Video Recording equipment

• Event logging, eye tracking software

Typical data collected

• Live video recordings

– Screen

– Mouse and keyboard movements

– Facial expression

• Audio

• Eye tracking video with cursor

• Eye tracking data files

• Post-test questionnaire

• Monitoring of tasks

Usability Measures

• Effectiveness• Task completion rate

• Number and percentage of tasks completed with and without assistance

• Error rate recovery

• Efficiency• Task completion time

• Real-time events

• Satisfaction• Post-test questionnaire

• Number of fixations

• Fixation duration

• Number of fixations on each Area of Interest

• Number of gazes on each Area of Interest

• Time to 1st fixation on target Area of Interest

• Scan path

Eye tracking measures

Studies and results

MyUnisa – Assignment Submission Tool

• Participant Profile

– Full-time UNISA students who have to submit

assignments online

– 10 participants

– 5 male, 5 female

– 7 different languages

– 5 expert, 5 non-expert Web users

MyUnisa – Assignment Submission Tool

Task example: Submit a PDF file

• Task completion: 70%

• Assistance needed: 40%

• Errors made: 100%

• Median task completion time: 115.60 seconds

• Play video: (PDF)

MyUnisa – Assignment Submission Tool

• All participants made errors– Did not select appropriate file format

– Error message:

• “ERROR: The type of file does NOT match the selected file type.

(PDF!=DOC)”

• All non-expert received error at least twice

• 3 experts received error once

– Error message not comprehended

• Difference between PDF and Word not

understood

MyUnisa – Assignment Submission Tool

• Eye Tracking

– Step 1 page: File selection till “continue” button click for Step 2

• Main goal: did participant read “File Format”

– Only 3 participants with several fixations on “File Format” label and drop-

down box

• Still made the error

Eye tracking results: Task 1

MyUnisa – Assignment Submission Tool

• Eye Tracking:

– Step 1: Error message

• Main goal: did participant read the error message

– All participants read error message

• Only 3 participants got it right after one error

Eye tracking results: Task 1

MyUnisa – Assignment Submission Tool

• PDF task – Recommendations:– File Type

• Should be picked up automatically

– Error message

• Simple language to be used

• PDF!=DOC is a computer term

– FAQ and instructions should contain more information about file

format and PDF

– Help Video File to teach new users

MyUnisa – Assignment Submission Tool

• Post-test questionnaire: Participant comments:– “MyUnisa is a very good tool in my life, saves time driving to

UNISA or Post Office to submit assignments.”

– “MyUnisa is at the cutting edge in educational pile drive.”

– “Think it would be difficult for a person who rarely uses computers.

They may have a problem understanding things like the file type.”

– “It will be very useful if users could be shown how to use this

website. I had a friend who had to show me how to work it.”

• Prototype tool

• Allows network managers at Nelson Mandela Metropolitan

University to analyse and explore application performance on

network

• Novel visualisation techniques

• Visualise application delay performance

• Integrated Tertiary Software application implemented at University

• Participants: Sound knowledge in domain of network

management

• Pilot study: 6 participants

• Case study: 9 participants

Network Management tool

Results

• Network Overview Task• Determine which Virtual Local Area Network has the highest

maximum total delay

• Answer obtained from graph or text

• Usability results• 100% task completion rate

• 100% correct answers

• Eye tracking results• Graphical AOI

• Most number of fixations

• Greatest fixation percentage

• Very few fixations on textual AOI

Results• Subnet View Task

• Determine the highest server delay

• Usability results

• 100% task completion rate

• Graph was filtered quick

• 0% correct answers

• Long time to find answers

• Eye tracking results

• Mean of 34.33 fixations on graph to find answer

• Difficulty of extracting information from the graph

• Participants fixated on wrong areas of Y-axis

Other interesting figures &

examples

Other interesting figures & examples

• Telkom corporate website – eye tracking video

• Karnaugh maths application – usability video

Incorporate usability in your

workspace!!

Conclusions• Learn about usability

– Websites

• Usability Professionals Association

• Human Factors International

– Books

• Prioritizing Web Usability – Nielsen, Loranger

• Institutionalization of Usability – Schaffer

– Join a Usability forum

• CUA, HFI forums

• SA UX Forum

• LinkedIn

– Take a University course

Conclusions

• Encourage usability in your project lifecycle

– User-centered design

• Third-party usability

– Write usability guarantees into vendor contracts

– Work with vendors to ensure usability (from the

start!!!)

• Its about the user!

Conclusions• Usability can help you!

– Websites

– Intranets

– Systems

– Products

• UNISA – usability facility available

• To discuss usability testing/research potential, contact:

– Marco Pretorius

– pretomc@unisa.ac.za / marco.pretorius@gmail.com

• Play video