usability with project lecture 14 – 31/10/08

© Simeon Keates 2008

Usability with ProjectLecture 14 – 31/10/08Dr. Simeon Keates


Exercise – Part 1

Last week you were asked to prepare your user trial protocols Today – put them into practice

Perform a pilot study of the usability of your web-site with at least 1 user

Remember – the principal aim is to “test the test” • (or “trial the trial” or “evaluate the evaluation”…)


Exercise – Part 2

Prepare a progress presentation for the board for Friday Show that good progress is being made

Summarise:• The tasks performed • The data collected• Whether the user liked the site• Whether the user could use the site (e.g. complete the tasks)• What you think is working well in the design• What you think needs to be looked at more closely in the design• Any changes you would like to make to the site and protocol


Exercise - Practicalities

Remember to print out copies of your protocol

Allow plenty of blank space for adding observation notes

Allocate one person to do the pre-session briefing and debrief

Allocate one person to be the facilitator (the person who directs the user)

The remaining members act as observers


Cognitive models


The Power Law of Practice

Tn = T1 n-α

α = 0.4, T1 = 60s, T2 = 45.5s (24% faster), T10 = 23.9s (60%faster)


Cognitive modelling – Dealing with uncertainty

The Uncertainty Principle states that decision time T increases with uncertainty about the decision to be made:

T = Ic H

Where: H is the information-theoretic entropy of the decision;

Ic = 150 [0~157] ms/bit

For n equally probable alternatives (Hick’s Law) :

H = log2(n + 1)

More generally:

€

H = pi log2( 1 pii∑ +1)


Cognitive modelling – The Model Human Processor

Time_taken = x τp + y τc + z τm

Where : x, y and z are integers

τp = time for perceptual processor

τc = time for cognitive processor

τm= time for (simple) motor function


Motor skills – Fitts’ Law

A person wishes to hit this target:

where:

Startx0 x1 x2

S

D

€

Tpos = IM log2(2D /S)

€

IM =−(τ p + τ c + τ m )

log2 ε


Merging the models

One basic merged model is the Keystroke Level Model (KLM):

Texecute = TK + TP + TH + TD + TM + TR

Where TK = total time spent keystroking = nk tk (# * time per stroke)

• Time per stroke determined experimentally

TP = total time spent pointing (from Fitts’ Law)

• Assume, say, 1.1 s per pointing action

TH = total time spent homing (moving hands between devices)

• Assume 0.4 s per homing

TD = total time spent drawing = tD (nD, lD) (i.e. f(#, total length))

• Example: 0.9nD + 0.16lD

TM = total time to mentally prepare

• Assume 1.35 s per preparation

TR = total system response time


Using the KLM

[Note: M = mental prep, K = keyboard, P = pointing] Rule 0: Insert Ms in front of all Ks that are not part of argument strings

proper. Place Ms in front of all Ps that select commands Rule 1: If an operator following an M is fully anticipated in an operator

just previous to M, then delete the M (e.g. PMK -> PK) Rule 2: If a string of MKs belongs to a cognitive unit (e.g. name of a

command), then delete all Ms but the first one Rule 3: If a K is a redundant terminator (e.g. terminates a command

immediately following the terminator of its argument), then delete the M in front of it

Rule 4: If a K terminates a constant string (e.g. a command name), then delete the M in front of it, but if the K terminates a variable string (e.g. an argument string) then keep the M in front of it


An more generic approach - GOMS

The user’s cognitive structure consists of: A set of Goals A set of Operators A set of Methods A set of Selection rules


GOMS – a quick breakdown

Goals: Symbolic structures that define a state of affairs to be achieved• Examples: GOAL: EDIT-MANUSCRIPT or GOAL: MODIFY-TEXT• Goals can comprise sub-goals

Operators: Elementary perceptual, motor or cognitive acts whose execution is

necessary to change any aspect of the user’s mental state or to affect the task environment• Examples: GET-NEXT-PAGE or GET-NEXT-TASK


GOMS – a quick breakdown

Methods: Procedures for accomplishing a goal – must be pre-learned at

performance time (i.e. user already knows them)• Contain sets of Operators

Selection rules: Rules for helping the user decide which method to use to accomplish the

goal• Example: if_such_and_such_is_true_then_use_method_M1_else_use_M2

To summarise: Several Operators make up a Method, and Selection rules are used to determine the best Method to reach the Goal


Using models of interaction

Fundamentally, you need to perform a comprehensive task analysis

The models indicate suggested performance for each sub-task

Those models help you to predict the performance of the interface

This can be used:• In design: Estimate performance using standard parameters to optimise your

design• In usability trials: Estimate the performance and compare with actual

observed data – investigate significant discrepancies


Extending to Universal Access applications – The MHP


Experiment I – Testing the MHP for motor-impaired users

User Description

PJ3 Tetraplegia (from head injury)

PJ4 Muscular Dystrophy

PJ5 Spastic Quadriplegia Cerebral Palsy

PJ6 Athetoid Cerebral Palsy

PJ7 Friedrich’s Ataxia

PJ8 Athetoid Cerebral Palsy


Perceptual response times

User Group τp Delay(ms)

τp Smooth(ms)

Card, Moran & Newell: Able-bodied

100[50~200]

100[50~200]

Cambridge:Able-bodied

80[70~100]

80[70~90]

Cambridge:Motion-impaired

100[70~120]

100[70~120]


Cognitive cycle times

User Group τc (ms)


70[25~170]


90[90~100]


110[100~130]


Motor function times

User Group τm (ms)


70[30~100]


70[60~80]


210[100~310]


Reaction times

User Group Predicted(ms)

Observed(ms)


310[100+70+(2*70)]

-


310[80+90+(2*70)]

320


630[100+110+(2*210)]

620


Explaining the observed motor times (100-310 ms)

Theoretical interaction is:• Press the button (motor function)• Release button (motor function)

Consequently, either • very slow motor function times

or• extra steps being inserted


Identifying the delays

c & p calculated as for Experiment I

m button-down and button-up times separated

Motor function and reaction time tasks performed Range of input devices used• mouse• touchpad button• space bar• EasyBall


The MS EasyBall


User descriptions

User Description τp (ms)

τc (ms)

PI3 Athetoid Cerebral Palsy

100 120


100 100


90 110

PI7 Friedrich’s Ataxia 90 110


Results

User Input Device

Button down Button up

Mean time (ms)

SD Mean time (ms)

SD

PI3 Mouse 230 35 210 34

PI5 MouseSpace barTrackpad

1708090

425

14

150115100

262219

PI6 MouseSpace barTrackpad

1008080

281417

90110110

231320

PI7 MouseSpace barTrackpadEasyBall

210120200220

71394367

180240230310

43865880


Results


Results

τm

?


PI7 results – Motor function task

0%

5%

10%

15%

20%

25%

30%0-

25

50-7

5

100-

125

150-

175

200-

225

250-

275

300-

325

350-

375

400-

425

450-

475

Time (ms)

Fre

qu

en

cy o

f o

ccu

ren

ce (

%) Button-

down

Button-up

© Simeon Keates 2008Background

The MHP results

0

50

100

150

200

250

300

350

Mo

tor

resp

on

se t

ime

(m

s)

A B C D E PI4 PI6 PI3 PI7 PI5 PI8

User

Able-bodied

Motion-impaired

~c


Conclusions

Extra cognitive cycles are being inserted Interaction process is:

• Decide to press button (cognitive) - OPTIONAL• Press the button (motor) - REQUIRED• Decide to release button (cognitive) - OPTIONAL• Release button (motor) - REQUIRED


Sources of extra cognitive steps

Users always in learning mode?

Users being overly careful?

Extra cognitive load from impairment?


Implications for use of user models

Individual components were comparable

However• method of combination was not

Therefore• need to verify user model assumptions before use


Implications for design

Users “add” own extra cognitive load

Need to support users by:• Minimising user uncertainty• Minimising cognitive load from program• Maximising interface intuitiveness• Maximising useful feedback


Extending to Universal Access applications – Cursor control


Symptoms associated with ageing and Parkinson’s

Symptoms: Essential tremor Restricted motion Reduced strength Poor hand-eye co-ordination Fatigue


Examples of motor-impaired cursor control

0

100

200

300

400

500

600

700

0 200 400 600 800 1000


Cursor movement theories

Fitts’ Law• Relates target distance and width to time

Movement Optimization Model• Initial, pre-planned ballistic move• (Optional) Secondary corrective submovements• Submovements based on visual feedback

Analysis of movement paths• Describes effect of changes in distance, width and height of target• Longer distances => higher peak velocity• Smaller target => longer deceleration phase

Initial studies [Hwang et al., 2004] suggest NOT universally applicable


Cursor measures(MacKenzie et al - CHI 2001)

Target Re-Entry (TRE)

Task Axis Crossing (TAC)

Movement Direction Change (MDC)

Orthogonal Direction Change (ODC)


Cursor measures (cont.)

Movement Offset (MO)• mean deviation of points from task axis ( y )• signed

Movement Error (ME)• average deviation of points from task axis• unsigned

Movement Variability (MV)• standard deviation of points from task axis

Missed Click (MCL)

Path Length / Task Axis Length (PL/TA)

Additional measures


Cursor measures (cont.)

Can distinguish between motor impaired and able-bodied users• As “groups”• Keates et al. ASSETS 2002

Can they do more?

Designed to explain why differences exist


User trials - The users

4 groups of users

• IBM interns (Y) – mean age 23, SD = 2.0• IBM regulars (A) – mean age 47, SD= 9.4• Older adults (OA) – mean age 79, SD = 4.5• APDA members (P) – mean age 57, SD = 5.2

6 users per group


User trials - The experimental methodology

Fitts’ Law type task• 3 target sizes• 3 target distances• 36 target acquisitions per target session

• 4 of each size/distance combination

• Random angle of approach to target

4 target sessions per user session (144 target acquisitions)

Interviews between each target session

Post-session debrief


User trials – Qualitative results

21 difficulties reported with mouse use, e.g.:• Keeping hand steady when navigating• Slipping off menus• Losing the cursor• Moving in the desired direction• Running out of room on the mouse pad• Mouse ball getting stuck (and/or dirty)

12 compensatory strategies, e.g.:• Avoid use of menus• Switch hands• Consciously go slower• Pause before clicking


User trials – Quantitative results

0

0.5

1

1.5

2

2.5

3

3.5

Group OA Group P Group Y Group A

Target activation times

0

20

40

60

80

100

120


No. of incorrect clicks

Peak velocities

0

1

2

3

4

5



User trials – Cursor measures (cont.)

Cursor measure Group OA Group P Group Y Group A p()

Path length / task axis length (PL/TA)

1.79 1.21 1.38 1.41 0.036

Missed clicks (MCL) 0.13 0.04 0.06 0.02 0.010

Task axis crossings (TAC) 3.32 2.55 2.71 2.49 0.018

Target re-entries (TRE) 0.88 0.46 0.58 0.59 0.913

Movement direction changes (MDC)

11.94 10.80 8.40 7.48 0.267

Orthogonal direction changes (ODC)

9.34 4.61 4.36 4.46 0.006

Movement error (ME) 33.00 21.40 24.61 28.78 0.152

Movement offset (MO) 26.74 18.22 20.76 24.34 0.258

Movement variability (MV) 30.53 18.93 23.31 27.45 0.200


User trials – Nature of movement observed

Differences in peak velocity do not explain all of target activation time differences

Theory: Target user movements are like able-bodied movements only more of them needed to complete the task

00.5

11.5

22.5

33.5

44.5

5


No. of pauses >100 msec

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6


No. of pauses >250 msec


User trials – Nature of movement observed

Submovements can distinguish between user groups (p<0.01)• Submovements are significantly related to:

• Path length / task axis length (PL/TA)• Missed/incorrect clicks (MCL)• Task axis crossings (TAC)• Target re-entries (TRE)• Movement direction changes (MDC)• Orthogonal direction changes (ODC)

• Submovement not significantly related to:• Movement error (ME)• Movement offset (MO)• Movemenet variability (MV)

Cumulative measures

Normalised measures


User trials – Pauses and cursor measures

Cursor measure # of 100 msec pauses

# of 250 msec pauses

Path length / task axis length (PL/TA) 1% 5%

Missed clicks (MCL) 1% 5%

Task axis crossings (TAC) 1% n.s.

Target re-entries (TRE) 1% 1%

Movement direction changes (MDC) 1% 1%

Orthogonal direction changes (ODC) 1% 1%

Movement error (ME) n.s. n.s.

Movement offset (MO) n.s. n.s.

Movement variability (MV) n.s. n.s.


User Trials – Where the pauses occur

0

50

100

150

200

250

300

3505

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95

10

0

% of movement

No

. of

pa

us

es

Young

Adult

Senior

PD


User trials – Verification pauses

0

50

100

150

200

250

300

475 425 375 325 275 225 175 125

msec before mouse down

No

. of

pa

us

es

Young

Adult

Senior

PD


User trials – Peak velocities

Peak velocity & Distance to target – strong +ve correlation Peak velocity & Target size – weak –ve correlation Peak velocity & User group:

0

1

2

3

4

5



User trials – Incorrect click results

No. of incorrect clicks as target session progresses No. of incorrect clicks as target size No. of incorrect clicks is not related to target distance No. of incorrect clicks is related to user group:

0

20

40

60

80

100

120



User trials – Incorrect click results (cont.)

Question – Can we tell the difference between correct and incorrect clicks?

No significant difference in:• Pause before• Duration of press

Significant difference in:• Distance moved while button pressed • Number of events while button pressed

Note: these are across ALL incorrect clicks• Across user groups• Across types of incorrect clicks


User trials – Observed types of incorrect clicks

210 incorrect clicks observed Near misses – within 50% of target radius (110) Not-so-near misses – >50% and <100% of target radius (35) Slipped clicks (32) Accidental clicks – >200% of target radius (9) Two button press (5)• User presses left button• While button down, user also presses right button• User releases right button, then left button• Not predicted!

Middle button press (2) Unclear (17)


Conclusions – Movement behaviour of older adults

Lower peak velocities• Do not explain all of longer target acquisition times

Increased # of pauses• Lack of confidence? Lack of expertise?

Most likely – difficulty getting on to target• High correlation of pauses with TRE, MDC and ODC• # of pauses towards (but not only at) end of movement

Do not follow single large ballistic / single homing phase Many (smaller) submovements towards target


Conclusions – Movement behaviour of users with PD

Many measures consistent with age of group• Which is more dominant – effects of age or PD?

Initiating movement can be difficult• “Bump” at start of pauses graph• Shared feature with older adults

Tend to be very deliberate• Lowest peak velocity• Second lowest error rate

Slight movement of mouse when pressing button• Multiple pauses in last half second of button press• Increased verification pause time


Summarising the differences

Younger adults (IBM interns)• Shortest (1), fastest (1), more errors (3) - slapdash

• “I can fix it”

• Games culture?

Adults (IBM regulars)• Shorter (2), faster (2), fewest errors (1)

• Best compromise between speed and accuracy?

Parkinson’s users• Longer (3), slowest (4), fewer errors (2)

• Slow, but sure

Older adults• Longest (4), slower (3), most errors (4)

• Lack of expertise

• Difficulty acquiring target


Exercise


Exercise

On Wednesday(-ish) you performed a pilot study

Today, make any changes you identified to your usability protocol

Also, make any changes to your web-site based on the feedback that you obtained

Please mail your finalised protocols to Anne and me

usability with project lecture 14 – 31/10/08

Documents