1 evaluation using user studies. usability is it a “good” interface? in what ways? usability:...

1

Evaluation UsingUser Studies

Usability Is it a “good” interface?

In what ways? Usability:

How well users can use the system’s functionality Dimensions of usability:

Learnability: is it easy to learn? Efficiency: once learned, is it fast to use? Memorability: is it easy to remember what you

learned? Errors: are errors few and recoverable? Satisfaction: is it enjoyable to use?

User and task requirements:cognitive models - keystroke level model

Predict performance times for common operations based on knowledge of human motor system

7 basic operatorsK - keystroking - actually striking keys

B - pressing a mouse button

P - pointing, moving the mouse at a target

H - homing - switching the hand between mouse and keyboard

D - drawing lines using the mouse

M - mentally preparing for physical action

R - system response (may be ignored)

M-operators in KLM

Initiating a task – pause while user considers what should be done

Making a strategy decision – which option to take?

Remembering something – e.g., a filename Finding something on the screen (here the

location is not well known) Verifying that what has been done or is about

to be done is correct

Typical KLM timesOperatorK

B

P

HDMR

RemarksPress key good typist (90 wpm) average typist (40 wpm) non-typistMouse button press down or up clickPoint with mouse Specific movement Average movementHome hands to/from keyboardDrawing Mentally prepareResponse from system

Time (s)

0.120.281.20

0.100.20

Fitts’ law1.100.40domain dependent1.20measure

Example of KLM

Deleting a file from the desktop on a Mac Method 1: drag to the wastebasket Operator sequence:

Initiate the deletion (M) Find the file icon (M) Point to file icon (P) Press and hold mouse button (B) Drag file icon to wastebasket (P) Release mouse button (B)

Total predicted time = 2M + 2P + 2B = 4.8 secs

Example of KLM Deleting a file from the desktop on a Mac Method 2: using an accelerator key Operator sequence:

Initiate the deletion (M) Find the file icon (M) Point to the file icon (P) Click – i.e., press and release mouse button (BB) Move hand to keyboard (H) Press ‘Apple’ and ‘Delete’ keys (KK) Move hand back to mouse (H)

Total predicted time = 1P + 2B +2K + 2M +2H = 5.1 seconds

Design implications from Gestalt Psychology

Proximity – group related items close together and separate unrelated ones

Alignment – place related items along an imaginary line. Align items of equal importance and indent subordinate ones

Consistency – make related items look the same Contrast – make unrelated items look different

What do you see?

proximity

similarity

symmetry

continuity

closure

12Original

13Proximity

14Alignment

15Repetition

Examples of Bad Design … and Why

Elevator controls and labels on the bottom row all look the same, so it is easy to push a label by mistake instead of a control button

People do not make same mistake for the labels and buttons on the top row. Why not?

From: www.baddesigns.com

Visibility - Example

• Control panel for an elevator

• How does it work?

• Push a button for the floor you want?

• Nothing happens - Push any other button?

Still nothing.

• What do you need to do?

• It is not visible as to what to do!

Visibility …need to insert room card in slot by buttons to get elevator to work!

How would to make this action more visible?

• Make card reader more obvious• Provide an auditory message that says

what to do (which language?)• Provide a big label next to the card reader

that flashes when someone enters

• Make relevant parts visible• Make what has to be done obvious

Logical or ambiguous design?

Where do you plug the mouse?

Where do you plug the keyboard?

top or bottom connector?

Do the color coded icons help?

From: www.baddesigns.com

How to design more logically

- A. provides direct adjacent mapping between icon and connector

- B. provides color coding to associate the connectors with the labels

Mapping

Relationship between controls and their movements and the results in the world

Why is this a poor mapping of control buttons?

Mapping

Why is this a better mapping?

The control buttons are mapped better onto the sequence of actions of fast rewind, rewind, play and fast forward

Mapping Which controls go with which rings (burners)?

A B C D

Why is this a better design?

Internal and external consistency

Internal consistency refers to designing operations to behave the same within an application Difficult to achieve with complex interfaces

External consistency refers to designing operations, interfaces, etc., to be the same across applications and devices Very rarely the case, based on different designer’s

preference Most successful in product families (e.g MS Office) Op. Sys. vendors may define style guidelines

External Inconsistency …

Keypad numbers layout

1 2 3

4 5 6

7 8 9

7 8 9

1 2 3

4 5 6

0 0

(a) phones, remote controls (b) calculators, computer keypads

Usability Problem Example: Unexpected Occurrence of Events

Usability Measures – 5 Often Used Time to learn

How long does it take for typical members of the community to learn relevant task?

Speed of performance How long does it take to perform relevant benchmarks?

Rate of errors by users How many & what kinds of errors are made during benchmark tasks?

Retention over time Frequency of use and ease of learning help make for better user

retention

Subjective satisfaction Do they like it? Allow for user feedback via interviews, free-form comments and

satisfaction scales

compliant interaction

state evident inmechanical buttons

rotary knobs reveal internal state and can be controlled by both user and

machine

Evaluation Techniques

Evaluation

tests usability and functionality of system

occurs in laboratory, field and/or in collaboration with users

evaluates both design and implementation

Cognitive Walkthrough

Proposed by Polson et al. evaluates design on how well it supports

user in learning task usually performed by expert in cognitive

psychology expert ‘walks though’ design to identify

potential problems using psychological principles

Cognitive Walkthrough (ctd)

For each task walkthrough considers what impact will interaction have on user? what cognitive processes are required? what learning problems may occur?

Analysis focuses on goals and knowledge: does the design lead the user to generate the correct goals?

33

Questions Cognitive Walkthrough

1. Is the next goal clear at this stage?

2. Is the appropriate action obvious?

3. Is it clear that this action leads to the goal?

4. What problems are there in performing the action?

Cognitive Walkthrough: How (cont.)

Walk through the task while answering these Questions: Will the user know what to do? Will the user see how to do it? Will the user understand from feedback whether

their action was correct?

Heuristic Evaluation

Proposed by Nielsen and Molich.

usability criteria (heuristics) are identified design examined by experts to see if these are

violated Example heuristics

system behaviour is predictable system behaviour is consistent feedback is provided

Heuristic evaluation `debugs' design.

The Procedure

Several independent evaluators each uses the same checklist each works alone each makes a list of usability problems

Combine lists into a single list works well as a group activity

37

“Think Aloud” Protocols “Single most valuable usability engineering method” Get user to continuously verbalize their thoughts Find out why user does things

What thought would happen, why stuck, frustrated, etc. Encourage users to expand on whatever interesting But interferes with timings May need to “coach” user to keep talking

Unnatural to describe what thinking Ask general questions: “What did you expect”, “What are you thinking

now” Not: “What do you think that button is for”, “Why didn’t you click here” Will “give away” the answer or bias the user

Alternative: have two users and encourage discussion

38

Analyzing the data

Numeric data Example: times, number of errors, etc. Tables and plots using a spreadsheet Look for trends and outliers

Organize problems by scope and severity Scope: How widespread is the problem? Severity: How critical is the problem?

Physiological measurements emotional response linked to physical changes these may help determine a user’s reaction to

an interface measurements include:

heart activity, including blood pressure, volume and pulse. activity of sweat glands: Galvanic Skin Response (GSR) electrical activity in muscle: electromyogram (EMG) electrical activity in brain: electroencephalogram (EEG)

40

1. Visibility of system status

Keep users informed about what is going on What page they are on and what part of a

process Provide appropriate feedback

About what system is doing, and how input is being interpreted

E.g. in XXX product, "really ungroup?" -- loses associated behavior

Eye tracking

head or desk mounted equipment tracks the position of the eye

eye movement reflects the amount of cognitive processing a display requires

measurements include fixations: eye maintains stable position. Number and

duration indicate level of difficulty with display saccades: rapid eye movement from one point of interest

to another scan paths: moving straight to a target with a short fixation

at the target is optimal

43

2. Match between system and the real world

Terminology in user’s language Not computer terminology

Language from user’s perspective “You have bought…” not “We have sold you…”

Use common words, not “techno-jargon”

Error messagesand feedback refer touser objects

Allow full-length names E.g. “Hit any key to continue”

44

3. User control and freedom Easy to abort: Cancel buttons

Cancel order, cancel changing a profile Easy to Undo

Web issue: what does “Back” button do? Example: many sites can get confused if use back button

Users (even experts) will make errors E.g. in XXX product,

no way to get out of editing a text field

45

4. Consistency and standards Same command always have the same effect Locations for information, names of commands Give the user a mental model of the system Size, location, color, wording, function, sequencing, etc.

E.g., color purple? Following standards helps

Web: use templates or CSS, style guides Seems easy, but often not followed; e.g. in XXX

naming "F#1.C#1" vs. "F#1", "C#1" consistent with industry standards: e.g., Copy

purple?

46

5. Error prevention

Selection rather than entry www.Expedia.com: question, when ambiguous city (e.g.

Columbus)

Remove or gray-out illegal choices Not common for web pages

Confirmation Avoid modes

Definition: same user action has different results Make unavoidable modes visible E.g. Typing "daytime" to a mail program

47

6. Recognition rather than recall

Make objects, actions, options visible See and pick it, not generate it Short-term memory= 7 ± 2 items; 30 sec to 2 min

unless interrupted Menus rather than type-in (but short enough) Prompts provide format and limits Don't require retyping of remembered information Pervasive, generic rules (cut/paste) E.g. in Aegis, remembering altitude

48

Example:prompts

What is a DTIC user code and how to get one?

49

Example: prompts (Print)

50

Error Messages, cont.

Blame the system, not the user “Unrecognized” vs. “illegal” command

No humor or snide comments Easy error recovery Can have multiple levels of messages E.g. in XXX product, “can't save file” — why

not?

51

Bad Error Messages

52

More bad error messages!

53

Another Bad Example

http://stinet.dtic.mil/

54

Another Bad Example

55

Another Bad Example

56

Pretty Good Example Pretty Good: travel.yahoo.com: Says what to do to fix it

But language is inconsistent

57

Help and Documentation True walk up and use? Most people will not read documentation

If do, then First time product is used, or else In a panic, so need information right away

Iterative design of documentation needed SuperBook application answer found in 4.3 minutes,

compared to 7.6 minutes before fixing Help system is an extra feature to learn “Help doesn’t”

If need to add help, maybe fix the feature? Use documentation writers to help refine the system

Good quality writing

58

Good Help Example

NSFreportsystem What

& Why

Firefox: No Zoom Indicator

59

Worst interfaces

60

Kitchen timer

62

Turn down the TV

63

Xerox Real Business (worst 2011)

64

Mathcad hard to learn at first hard to keep track of variable / whole thing fails

because of one small error / not good at explaining source of errors

use of arrow-keys vs. space-bar to move things in an equation expected arrow keys to work

4+ different versions of the equals-sign / hard to distinguish on the screen

new version can’t read files from other versions hard to remember after not using it for a while (lots of

tricks needed, hard to remember)

Examples of non-usable systems

EclipseHard learning curve, bad for beginners

Too many windows, confusing layout, too much going on (advanced options)

ISIS pops up a window, pop-up blocker problems can’t use browser’s back button gives course instructor number instead of name must enter schedule-number instead of clicking on course ID and

number Must use semester as a code, and if you get this wrong it’s

confusing what’s wrong Interface has pictures/images for links, not the conventional

under-lined links not integrated into COD, so must copy info over from COD to

ISIS Kicks you off too quickly Too often overloaded, too many users Unavailability at certain hours / don’t know when it’s unavailable /

status not visible easily Links on main page are too similar SSN number use

68

175 design Guidelines These are primarily intended to guide you to a good design, not to be used for evaluation1) Help users plan goals, tasks2) Provide a clear model of how users view system in terms of tasks3) Help users with system model, metaphors, work context4) Design to match user’s conception of high-level task organization5) Help users understand what system features exist and how they can be used in their

work context6) Help users decompose tasks logically7) Make clear all possibilities for what users can do at every point8) Keep users aware of system state for planning next task9) Keep the task context visible to minimize memory load10)Help users plan the most efficient ways to complete their tasks11)Keep users aware of task progress, what’s been done and what’s left to do12)Provide cognitive affordances at the end of critical tasks to remind users to complete

the transaction13)Provide effective cognitive affordances that help users get access to system

functionality14)Help users know/learn what actions are needed to carry out intentions

Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

Guidelines, cont.15) Help users know how to do something at action/object level16) Help users predict outcome of actions17) Help users determine what to do to get started18) Provide a cognitive affordance for a step the user might forget19) Support user with effective sensory affordances in presentation of cognitive

affordances20) Make cognitive affordances visible21) Make cognitive affordances noticeable22) Make text legible, readable23) Control cognitive affordance presentation complexity with effective layout,

organization, and grouping24) Present cognitive affordance in time for it to help the user before the

associated action25) Help user determine actions with effective content/meaning in cognitive

affordances26) Design cognitive affordances for clarity27) Use precise wording in labels, menu titles, menu choices, icons, data fields28) Use a verb and noun and even an adjective in labels where appropriate.29) Avoid vague, ambiguous terms.

69Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

Guidelines, cont.30) Be as specific to the interaction situation as possible31) Clearly represent work domain concepts32) Use dynamically changing labels when toggling33) Provide cognitive affordances to indicate formatting within data fields34) Constrain the formats of data values to avoid data entry errors35) Provide clearly marked exits36) Provide clear “do it” mechanism37) Be predictable; help users predict outcome of actions with feed-forward

information in cognitive affordances38) Make choices distinguishable39) Be consistent with cognitive affordances40) Use consistent wording in labels for menus, buttons, icons, fields41) Use similar names for similar kinds of things42) Do not use multiple synonyms for the same thing43) Use the same term in a reference to an object as the name or label of the

object44) Use different terms for different things, especially when the difference is subtle

70Quoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

Guidelines, cont.

71

45) Be consistent in the way that similar choices or parameter settings are made46) Decompose complex instructions into simpler parts47) Use appropriate layout and grouping by function to convey content and meaning48) Group together objects and design elements associated with related tasks and

functions49) Do not group together objects and design elements that are not associated with

related tasks and functions50) Support user choices with likely and useful defaults51) Provide the most likely or most useful default selections52) Offer most useful default cursor position53) Relieve human short term memory loads by maintaining task context visibly or

audibly for the user54) Support human memory limits with recognition over recall55) Avoid requirement to retype or copy from one place to another56) Support special human memory needs in audio interaction design57) Avoid cognitive indirectness58) Be complete in your design of cognitive affordances; include enough information for

users to determine correct action


Guidelines, cont.

72

59) Prevent loss of productivity due to hesitation, pondering60) Use enough words for unambiguous labels61) Add supplementary information, if necessary62) Give enough information for users to make confident decisions63) Give enough alternatives for user needs64) Employ usage-centered wording, the language of the user and the work

context, in cognitive affordances65) Find ways to anticipate and avoid user errors in your design66) Help users avoid inappropriate and erroneous choices67) Disable buttons, menu choices to make inappropriate choices unavailable68) Gray out to make inappropriate choices appear unavailable69) But help users understand why a choice is unavailable70) Provide a clear way to undo and reverse actions71) Offer constructive help for error recovery72) Avoid confusing modalities73) Distinguish modes clearly74) Use “good modes” where they help natural interaction without confusion75) Support human memory limitations in the design of task structure76) Support user with effective task structure and interaction control77) Provide alternative ways to perform tasks


Guidelines, cont.

73

78) Provide shortcuts79) Provide logical grouping in layout of objects80) Group together objects and functions related by task or user work activity81) But avoid grouping of objects and functions if they need to be dealt with

separately82) Support task thread continuity by anticipating the most likely next task,

step, or action83) Make the most of user’s work84) Do not requiring users to re-enter data85) Retain user state information86) Avoid the feeling of loss of control87) Give direct manipulation support88) Always provide a way for the user to “bail out” of an on-going operation89) Support users making physical actions with effective sensory affordances

for sensing physical affordances90) Support user with effective physical affordances for manipulating objects,

help in doing actions91) Avoid physical awkwardness92) Accommodate physical disabilities93) Design layout to support manual dexterity and Fitts’ law


Guidelines, cont.

74

94) Support targeted cursor movement by making selectable objects large enough

95) Group clickable objects related by task flow close together96) But not too close, and do not include unrelated objects in the grouping97) Design physical movement to avoid physical overshoot98) Include physicality in your design when the alternatives are not as

satisfying to the user99) Check your functionality for missing features100)Check your functionality for non-user-interface software bugs101)Avoid too much automation and loss of user control102)Help the user by automating where there is an obvious need103)Provide feedback for all user actions104)Provide progress feedback on long operations105)Request confirmation as a kind of intervening feedback106)But don’t overuse and annoy107)Support user with effective sensory affordances in presentation of

feedback108)Make feedback visible109)Make feedback noticeable110)Locate feedback within the user’s focus of attention


Guidelines, cont.

75

111)Make feedback large enough to notice112)Make text legible, readable113)Control feedback presentation complexity with effective layout,

organization, and grouping114)Help users detect error situations early115)Maintain a consistent appearance across similar kinds of feedback116)Maintain a consistent location of feedback presentation on the screen to

help users notice it quickly.117)Use the most effective feedback presentation medium118)Consider audio as alternative channel119)Help users understand outcomes with effective content/meaning in

feedback120)Design feedback for clarity121)Support clear understanding of outcome (system state change), so users

can assess effect of actions122)Give clear indication of error conditions123)Be complete in your design of feedback; include enough information for

users to fully understand outcomes and be either confident that their command worked or certain about why it didn’t

124)Prevent loss of productivity due to hesitation, ponderingQuoted from Chapter 19 of H. Rex Hartson and Pardha S. Pyla, The UX Book: Ensuring a Quality User Experience, to be published by Morgan Kaufmann / Elsevier in 2011.

Guidelines, cont.

76

125)Add supplementary information, if necessary126)Give enough information for users to make confident decisions about the

status of their course of interaction127)Help users understand what the real error is128)Give enough information about the possibilities or alternatives so user can

make an informed response to a confirmation request129)Design feedback wording, especially error messages, for positive

psychological impact130)Make the system take blame for errors131)Be positive, to encourage132)Provide helpful, informative error messages, not “cute” unhelpful messages133)Employ usage-centered wording, the language of the user and the work

context, in displays, messages, and other feedback134)Be consistent with feedback135)Label outcome or destination screen or object consistently with starting

point and action136)Organize feedback for ease of understanding137)Provide user control over amount and detail of feedback138)Give only most important information at first; more on demand


Guidelines, cont.

77

139)Organize information displays for ease of understanding140)Eliminate unnecessary words141)Group related information142)Control density of displays; use white space to set off143)Columns are easier to read than wide rows144)Use abstraction per Shneiderman’s “mantra”: Overview first; zoom and

filter; details on demand145)Employ usage-centered wording, the language of the user and the work

context146)Avoid the use of anthropomorphism in interaction designs147)Avoid using first-person speech in dialogue148)Avoid condescending offers to help149)Avoid poor attempts at humor150)Avoid violent, negative, demeaning terms151)Avoid use of psychologically threatening terms, such as “illegal”, “invalid”,

“abort”152)Avoid use of the term “hit” instead of “press” or “click”153)Avoid irritation with annoying sound and color in displays154)Use color conservatively155)Use pastels, not bright colors


Guidelines, cont.

78

156)Be aware of color conventions (e.g., avoid red, except for urgency)157)Watch out for focusing problem with red and blue158)Avoid fancy or cute design without a real purpose159)Make presentation of text legible160)Make font size large enough for all users161)Use good contrast with background162)Use mixed case for extensive text163)Avoid too many different fonts, sizes164)Use legible fonts165)Use color other than blue for text166)Accommodate sensory disabilities and limitations167)Allow user settings, preference options to control presentational

parameters168)Accommodate different levels of expertise/experience with preferences169)Don’t let affordances for new users be performance barriers to experienced

users170)Be helpful with Help171)Do not try to achieve the appearance of simplicity by just reducing

usefulness172)Organize complex systems to make the most frequent operations simple


Guidelines, cont.

79

173)Use consistent layout/location for objects across screens174)Maintain custom style guides to support consistency175)Use structurally similar names and labels for objects and functions that are

structurally similar176) ….and the list isn’t even finished


1 evaluation using user studies. usability is it a “good” interface? in what ways? usability:...

Documents

proximity slide

original slide

measure slide

repetition slide

seconds slide

correct slide

different slide

mouse button p