Cognitive Walkthrough

Judy Kay

CHAI: Computer human adapted interaction research group

School of Information Technologies

growing set of methods: HE, TA, CW

3

Overview

• Predictive method

• Cognitive Walkthrough

• Benefits

• Disadvantages

251,000 results from Google,

10,800 from Google Scholar

4

• Focus on learnability• Appropriate for novice or casual users

• cf Think-Aloud?

• Does not focus on speed• cf Think-Aloud?

• Sequence is not known prior to inspection• Assesses user success and recovery from errors• Conducted by experts

• cf Think-Aloud?

5

Model of Exploratory Learning

• User has task– 0. will user understand this sub-task is needed?

• Explores system for useful elements– 1. will correct action be obvious? eg button visible

– 2. will user understand instructions? eg user understands the label on the button

• Selects one to try• User interprets system response

– 3. will user know if progress has been made?

Extended Cognitive Walkthrough

Takes account of mental model…..

7

• User has task– 0. will user understand this sub-task is needed

(given their mental model)

• Explores system for useful elements– 1. will correct action be obvious?

– (given their mental model)

– 2. will user understand instructions?

– (given their mental model)

• Selects one to try• User interprets system response

– 3. will user know if progress has been made?

– (given their mental model)

How to add a mental model?

http://www.nngroup.com/articles/mental-models/

What users and user beliefs matter for this context?

“A mental model is what the user believes about the system at hand.

• based on belief, not facts … they base their predictions [and actions] about the system on their mental models …

• Individual users each have their own mental model . … one of usability's big dilemmas is the common gap between designers' and users' mental models. …

• mental models are in flux

10

Example with basic CW

• Design a cash-operated machine for quick, easy purchase of train tickets by passengers, without training

• Abstract user goals:• Buy a ticket to the required destination

• Determine whether I can afford to buy the ticket to a particular destination

Adapted from Newman and Lamming,

Interactive System Design, 1995)

11

Example with basic CW

• Design a cash-operated machine for quick, easy purchase of train tickets by passengers, without training

• Designer goal breakdown to subgoals:• Determine fare to pay

– indicate destination

– indicate one-way or return

• Dispense ticket– pay money

– get ticket and change

Adapted from Newman and Lamming,

Interactive System Design, 1995)

12

The interface – lo-fi prototype

Concrete user task

Class activity: define 3 concrete tasks

14

The interface – lo-fi prototype

Concrete user task

Assuming you are at Bondi Junction,buy a one-way ticket to Town Hall

16

The task:

Buy a one-way ticket to Town Hall

17

How designer wants it to work – method 1 where use does not use keypad• Click destination

• ie Town Hall

• Click journey type• ie one way

18

• 0. Task?– Buy one-way ticket to Town Hall

• 1. Is correct action obvious?– Two possible sub-tasks:

• Destination

• One way / return

– Will user know both must

be set? (MM?)

19

How to Help the User?

• Indicate extra information needed• Indicate steps 1 and 2 can be done in either order• Give some feedback on effect of each select action• Reorganise layout so that Steps 1 and 2 are

followed by the fare display

We alter our lo-fi prototypeand check this

21

• 2. Will user understand instructions?

• 3. Will user know if progress is being made? (ie interpret machine action as intended by the designer)

We alter our lo-fi prototypeand check this

22

• 2. Will user understand instructions?– Yes – due to labels and instructions

• 3. Will user know if progress is being made? ?– Yes (buttons light up, new state appears)

23

24

• 1. Correct action obvious?– Yes

• 2. Will user understand instructions?– Yes

• 3. Will user know if progress is being made?– Yes

25

Paying

• 1. Correct action obvious?

• 2. Will user understand instructions?

• 3. Will user know if progress is being made?

What are the cases to consider for payment (in cash)

Consider case where user has exact change

27

Paying

Designer intends user to • Pay in money and click • Click “4. Press for ticket”• User then lifts flap to collect the ticket

28

Paying – exact change case

• 1. Correct action obvious?– Yes

• 2. Will user understand instructions?– Yes

• 3. Will user know if progress is being made? Unclear (no feedback on money accepted so far)

We alter our lo-fi prototypeand check this

30

Revised design gives feedback on amount paid so far

Consider other cases eg user realises they have insufficient money

32

Buying a Ticket: Insufficient Money?

• 1. Correct action obvious?

33

Summary of Flaws (so far)

• Confusion about need for steps 1 and 2• No feedback on amount deposited• No means to get money back• So far …...

34

Goals and Tasks

In this example:

Goal: buy a ticket

Sub-goals: (determine fare) and pay

User tasks: concrete cases used in CW

35

Extended cognitive walkthrough

• Adding user's mental model

• What does user believe?• How do you find this out?• What did we assume about the user’s mental

model?• What differences are there in the MM for:

• A novice user

• An expert user

Class activity:

List aspects of the user’s MM that would be relevant to the train ticket interface

What are the implications of some likely cases:

user familiar with existing interfaceuser familiar with a different bus ticket interface

38

Potential GOTCHAs

• Need a prototype that is complete enough for a walkthough of an interesting concrete case

• But this is a technique for early in design process, where designer is ready to change it

• You then need to define:• the user groups, and their relevant MMs• a good set of tasks• the correct steps (intended by designer)

• Evaluator must imagine people’s thoughts on first use of UI• Keep referring to the mental model• Carefully assess vocabulary/text in terms of

mental model• Repeat process over the tasks, and for each

important class of mental model

39

Summary of uses of CW

• Relatively inexpensive in our very, very lightweight approach

• Desk check– No users

– Better with expert evaluators

• Generally applicable

• Novice, casual, intermittent users

• Focus on learnability

40

Summary of usefulness

• Really useful technique, even for designer

• Better if done by • Outsider

• Expert

• But students and non-expert evaluators still can gain from using it

• Part of early usability evaluation because• Low cost

• No users needed