1 cs 430: information retrieval lecture 13 usability 1 guest lecture: gilly leshed
Post on 21-Jan-2016
214 Views
Preview:
TRANSCRIPT
1
CS 430: Information Retrieval
Lecture 13
Usability 1Guest Lecture: Gilly Leshed
2
Course Administration
Midterm Examination
When: Wednesday, October 13, 7:30 to 9:00 p.m.
Where: Upson B17
About: All material covered before the midterm break
For a sample paper and further information see the Examinations page on the web site.
Note the instructions about laptop computers.
3
Course Administration
Assignment 2
To help understanding of how to calculate the similarity between a query and a document, Lecture 11, Slide 24 has been expanded to three slides.
The paper describes how to calculate the inner product between two vectors. Lecture 11, Slides 24-26 now describe how to scale the inner product to calculate the cosine of the angle between two vectors.
With the test data, the documents are all similar in length, so that a ranking based on inner product will be close to a ranking based on cosine.
4
Lecture Outline
• N-Tier Model
• User-Centered Approach
• Requirements
• Design
• Implementation
• Evaluation
• Summary
5
User InterfaceUser Interface
Business LogicBusiness Logic
DataData
N-Tier Model
Web pageWeb page
Search engineSearch engine
Index file, data filesIndex file, data files
Focus of this lecture
N-Tier model
6
User-Centered Approach
• Requirements
– Who are the users?
– What do they want?
– What do they need?
• Designing the user interface
• Implementing the user interface
• Evaluating the user interface
User-Centered Approach
Requirements
Design
Evaluation
Implementation
7
RequirementsWho are the users?
• Understanding the users via ethnographic research• Descriptions of users:
– Demographic characteristics– Computer usage background– Job description and work environment– Disabilities:
• Color blindness• Language issues• Typing issues
• Personas – archetypes of users, describing behavior patterns, goals, skills, attitudes, and environment (Cooper, 2003)
Requirements
The study of people in their natural settings; a descriptive account of social life and culture, based on qualitative methods (e.g. detailed observations, unstructured interviews, analysis of documents)
The study of people in their natural settings; a descriptive account of social life and culture, based on qualitative methods (e.g. detailed observations, unstructured interviews, analysis of documents)
8
RequirementsWhat do Users Need?
• Defining user interface requirements
• Based on task analysis:
– Task definition
– Context definition
• Several ways to accomplish task analysis:
– Ethnographic research
– Scenario-based analysis
– Discussion with users and subject-matter experts (SME)
Requirements
9
RequirementsWhat do Users Want?
• When asking users, they often:– Provide their attitudes, not their needs– Bend the truth to be closer to what they think you want
to hear– Rationalize their behavior
“I would have seen the button if it had been bigger”
• Instead of asking users what they want:– Watch what they actually do– Do not believe what they say they do– Definitely don't believe what they predict they may do
in the futureRequirements
10
Non-functional Requirements
Performance, Reliability, Scalability, Security…
• Example: Response time (Nielsen, 1994)0.1 sec – the user feels that the system is reacting instantaneously1 sec – the user will notice the delay, but his/her flow of thought stays uninterrupted10 sec – the limit for keeping the user's attention focused on the dialogue
Requirements
11
DesignMental Models
Also called conceptual model
• What a person thinks is true about a system, not necessarily what is actually true
• Similar in structure to the system that is represented
• Simpler than the represented system. A mental model includes only enough information to allow accurate predictions (i.e. no data structures)
• Allows a person to predict the results of his actions
Design
12
System Model of an ArticleSystem Model of an ArticleUser Model of an ArticleUser Model of an Article System Model of an ArticleSystem Model of an ArticleUser Model of an ArticleUser Model of an Article
DesignMental Models – Example 1
The article’s body and meta-data are conceived as a whole.
An article’s meta-data is available in one database and its data is available in a separate database.
Design
13
DesignMental Models – Example 2
System Model of a search engine
System Model of a search engine
User Model of a search engine
User Model of a search engine
The search engine retrieves return hits directly from their source
The search action does not involve accessing the documents’ source
Design
14
DesignUser Interface Design Guidelines
• Consistency – Appearance, controls, and function– Both within the system and to similar systems
• Feedback • User in control• Recognition rather than recall• Easy reversal of actions
– Error handling• Consider different expertise:
– Novice, intermediate and expert users
Design
15
Designing the Search PageMaking Decisions
• Overall organization:
– Spacious or cramped
– Division of functionality to different pages
– Positioning components in the interface
– Emphasizing parts of the interface
• Query insertion: insert text string or fill in text boxes
• Interactivity of search results
– Retrieve the information from the results
– Narrow the search
• Performance requirements
Design
16
Google Spacious organization
Spacious organization
17
Yahoo! cramped organization
cramped organization
18
AltaVista
Division of functionality to different pages
Division of functionality to different pages
19
ACM Digital Library
Emphasized components
Emphasized components
20
ACM Digital Library advance searchDifferent query insertion ways
Different query insertion ways
21
DesignReturn Hits
• A snippet is a short record that a search system returns to describe and link to a hit.
• Example: Web search “Nielsen evaluation heuristics”
Heuristic Evaluation
... Jakob Nielsen's Online Writings on Heuristic Evaluation. How to conduct a heuristic evaluation; A list of ten recommended heuristics for usable interface design ... www.useit.com/papers/heuristic/ - 5k - Cached - Similar pages
Design
22
Designing the Return HitsMaking Decisions
• Dynamic (generated from query+ document) or precomputed (from document only)
• Content only or with related information
• Highlighting of search terms
• Length vs. number on page
User must understand why the hit was returned
Design
23
Dynamic Return Hits Dynamic snippets
Dynamic snippets
24
Precomputed Return Hits Precomputed snippets
Precomputed snippets
25
Implementation
• Use HTML, Java, Visual Basic, WinForms…
• Make sure the user interface is loosely-coupled with the search engine underneath
User InterfaceUser Interface
Business LogicBusiness Logic
DataData
Implementation
26
Evaluation
• The process of determining the worth of, or assigning a value to, the user interface on the basis of careful examination and judgment.
• Make sure your system is usable before launching it!
• Two categories of evaluation methods:
– Empirical evaluation: with users
– Analytical evaluation: without users
Evaluation
27
Evaluation
But what is usability?From ISO 9241-11, Usability comprises the following aspects:• Effectiveness – the accuracy and completeness with which users
achieve certain goals– Measures: quality of solution, error rates
• Efficiency – the relation between the effectiveness and the resources expended in achieving them– Measures: task completion time, learning time, clicks number
• Satisfaction – the users’ comfort with and positive attitudes towards the use of the system– Measures: attitude rating scales
Evaluation
28
Evaluation with Users
Testing the system, not the users!
Stages of evaluation with users:
Preparation
Sessions conduct
Analysis of results
Evaluation
29
Evaluation with UsersPreparation
• Determine goals of the usability testing“The user can find the required information in no more than 2 minutes”
• Write the user tasks“Answer the question: how hot is the sun?”
• Get participants
– Use the descriptions of users from the requirements phase to detect potential users
Evaluation
30
Evaluation with UsersSessions Conduct
• Conduct the session– Usability Lab– Simulated working
environment
• Observe the user– Human observer(s)– Video camera– Audio recording
• Inquire satisfaction dataEvaluation
31
Evaluation with UsersResults Analysis
• If possible, use statistical summaries
• Pay close attention to areas where users
– were frustrated
– took a long time
– couldn't complete tasks
• Respect the data and users' responses, don't make excuses for designs that failed
• Note designs that worked and make sure they're incorporated in the final product.
Evaluation
32
Evaluation without Users
• Assessing systems using established theories and methods
Evaluation techniques• Heuristic Evaluation (Nielsen, 1994)
– Evaluate the design using “rules of the thumb”• Cognitive Walkthrough (Wharton et al, 1994)
– A formalized way of imagining people’s thoughts and actions when they use the interface for the first time
• Claims Analysis – based on scenario-based analysis– Generating positive and negative claims about the
effects of features on the user
Evaluation
33
Summary
User InterfaceUser Interface
Business LogicBusiness Logic
DataData
Summary
Requirements
Design
Implementation
Evaluation
top related