an overview of usability evaluation #15 1. outline what is usability evaluation ? why perform...
Post on 03-Jan-2016
226 Views
Preview:
TRANSCRIPT
2
Outline
What is usability evaluation ?
Why perform usability evaluation ?
Types of usability evaluations
What can we learn from: Heuristic evaluation ? Usability testing ?
How do we conduct : Heuristic evaluation ? Usability test ?
3
What Is Usability Evaluation
A systematic process aimed at assessing the fit between a UI design and human capabilities within a task context
It is a central element of the UI design process performed throughout the UI development process
4
Why Perform Usability Evaluations?
Find usability problems in an interface design
Assess compliance with style guide ( e.g., MS windows )
Compare alternative UI components Icon design Input/output technologies
Assess the worth/usefulness of the software in overall job context
5
Evaluation Methods
Evaluation Categories Evaluation Requirements
Evaluation Technique
Usability inspection methods
A static prototype A UI design expert
Heuristic evaluationEvaluation against guidelinesCognitive walkthrough
User-based Evaluations A dynamic prototype A usability analyst
QuestionnairesObservational usability studyFormal usability study with quantitative data analysisControlled experiments
Analytic Evaluations A UI designer with expiries in analytic techniques
Keystroke level modelGOMSGrammars
6
Questions of a Usability Evaluation (1)
Are the functions made available in a convenient, task oriented way? we need to have task knowledge
Does the system anticipate the skill and knowledge of the user? we need to have user knowledge
Does the design meet general rules of good user interface design? we need to have UI knowledge
7
Questions of a Usability Evaluation (2)
Is the system compliant with other applications running in the user environment? We need to know the style guide knowledge
How fast can users learn to use the system?
At what speed can users perform various task?
How likely are users to complete a given task?
9
Evaluation Against Guidelines and Rules
A process in which each UI element (e.g., menu choice, icon, button, pointer, radio button) is examined against an existing set of general guidelines and a specific set of design rules (the style guide) applicable to a specific product Mil STD 1476 F Windows style guide
Performed by one or more UI design experts who have a thorough familiarity with general UI design guidelines and the product/corporate style guide
10
Guidelines and Rules
Guidelines are accepted principles for interface design
Rules specify the interface appearance or action
11
Examples of Guidelines
Guidelines Displays should be consistently formatted Displays should be uniquely identified Use short simple sentences Employ units of measurement that are familiar to the user
12
Examples of Design Rules
Design Rules The character stroke width of a system font shall be at least 2 pixels
thick F10 (and Shift+Menu) exits the menu bar and returns the location cursor
to the previous object with focus
13
Pro and Cons of Evaluating Against Guidelines
Pros Provides information on basic design issues Finds a broad range of usability problems
Cons Dose not assess whether system meets user's and task
needs ( can be compliant and still have poor design) Time consuming Guidelines/rules don't exist for all areas of UI design
14
Heuristic Evaluation
Popular and widely used structured review of a UI
Objective is to generate a list of potential usability problems
Evaluator assumes the user's role and identifies problems from a user's perspective
Criteria for "a problem" is a set of recognized usability principles called "Heuristics"
15
Heuristics Identified by Nielsen (1993)
Use simple and natural dialogue
Speak the users' language
Minimize the users' memory load "‘
Be consistent
Provide feedback
Provide clearly marked exit
Provide shortcuts
Provide good error messages
Prevent errors
16
Conducting a Heuristic Evaluation
Collect background information Identify typical users, scenarios, previous feedback,
usability goals
Inspect the flow of the interaction from screen to screen
Inspect screens one at a time against the heuristics
Generate an inspection problem report Lists and prioritize the usability issues, fixes and/or
redesigns
17
Who Should Inspect ?
Usability specialists often find more “problems” than evaluators with no usability experience (or computer experience only)
Usability specialists with knowledge about a particular kind of interface being developed (Double specialists) find more usability problems than “regular” usability specialists
How Many Inspectors ? Single evaluator finds only about 35% of the problems
Increasing the number of evaluators from 2 to 5 increases the number of problems found up to around 75% of all the problems
Percentof ProblemsFound
5Number of inspectors .
Nielsen 1993
18
19
Types of Problems Uncovered by Heuristic Evaluation
Missing or difficult-to-access functionality
Limited or inappropriate task flow
Limited navigational cues
Inappropriate feedback
Cluttered screens
20
Pros and Cons of Heuristic Evaluation
Pros Does not involve users, Relatively inexpensive Finds a broad range of major and minor usability problems Maximized by using multiple evaluators Less intimidating to developers then usability testing
Cons Subjective and dependent on HCI skills of the evaluators and
their knowledge of the task and the users Depends on how realistically and to what degree the system is
exercised Not exhaustive
22
Usability Testing
A set of methods of user-based evaluations Questionnaires Observational usability studies Formal usability studies with quantitative data analysis Controlled experiments
Observe and measure how users interact with an application
Focus on the direct feedback from end users interacting with the system
Should be the ultimate goal of every evaluation plan because it involves real end users
23
The Nature of Usability Testing
Merges several user-based evaluation methods into a single evaluation process Observation Interviews Testing
Each method illuminates a different aspect of usability
Performed after a design (or parts of a design) have been fine tuned based on usability inspection techniques
Performed before a prototype is handed over to developers and a product is sent out
24
When is Usability Testing Useful
Test early to: Evaluate an individual aspect of the design Significantly affect the design Provide quick answers to developer
may involve fewer users may collect less data
Test late to: Verify the entire application
Stable design Full functionality
Asses the impact of the design on the user Controlled variables
25
Where Is Usability Testing Performed
User office environment Users are in their natural surroundings Easier to recruit users But -- Uncontrolled environmental setting Interruptions Variety of computer configuration No observation by development team permitted
Usability Lab Controlled setting Consistent computer configuration Data collection equipment Permits unobtrusive observation by development team
26
Who Participates in Usability Testing ?
Evaluators Usability specialists
Participants Potential users
Observers Members of the design/development team
27
How Many Participants to Include ?
At least 2 from each distinct user group
2 - 3 at earlier stages of the evaluation when focus is on gross usability issues
6 and up (per user group) at later stages of the evaluation when focus is on performance assessment
Remember, the objective in usability evaluation is not to uncover statistical differences, only design issues
28
Measures of Usability
Time to complete task
Completion rate
Number of errors
Types of errors
Severity of errors
Number of requests for help
Number of trials to become proficient in using the system
Comparative ratings
Subjective ratings
29
What to Expect From Test Participants
Do the unexpected
Have preconceived ideas
Do not always ask for help
Fail to follow instruction
Quickly develop habits
Are afraid of breaking system
Are apologetic
30
Data Collection Techniques
Video Taping User’s interactions with the application User’s facial expressions
Audio Taping User comments Observer comments
Data collection applications Keystroke capture Indexed videotape
Questionnaires
Interviews Open ended Structured
31
Summary
Start evaluation early in the design process and continue to evaluate throughout the development cycle This will minimize the likelihood of a major usability
problem emerging during the later phases of the development
Incorporate a variety of evaluation methods One method cannot predict or identify all the potential
usability issues
Include at least one user-based evaluation method in your evaluation plan
top related