![Page 1: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/1.jpg)
1
CS 430: Information Retrieval
Lecture 13
Usability 1Guest Lecture: Gilly Leshed
![Page 2: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/2.jpg)
2
Course Administration
Midterm Examination
When: Wednesday, October 13, 7:30 to 9:00 p.m.
Where: Upson B17
About: All material covered before the midterm break
For a sample paper and further information see the Examinations page on the web site.
Note the instructions about laptop computers.
![Page 3: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/3.jpg)
3
Course Administration
Assignment 2
To help understanding of how to calculate the similarity between a query and a document, Lecture 11, Slide 24 has been expanded to three slides.
The paper describes how to calculate the inner product between two vectors. Lecture 11, Slides 24-26 now describe how to scale the inner product to calculate the cosine of the angle between two vectors.
With the test data, the documents are all similar in length, so that a ranking based on inner product will be close to a ranking based on cosine.
![Page 4: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/4.jpg)
4
Lecture Outline
• N-Tier Model
• User-Centered Approach
• Requirements
• Design
• Implementation
• Evaluation
• Summary
![Page 5: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/5.jpg)
5
User InterfaceUser Interface
Business LogicBusiness Logic
DataData
N-Tier Model
Web pageWeb page
Search engineSearch engine
Index file, data filesIndex file, data files
Focus of this lecture
N-Tier model
![Page 6: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/6.jpg)
6
User-Centered Approach
• Requirements
– Who are the users?
– What do they want?
– What do they need?
• Designing the user interface
• Implementing the user interface
• Evaluating the user interface
User-Centered Approach
Requirements
Design
Evaluation
Implementation
![Page 7: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/7.jpg)
7
RequirementsWho are the users?
• Understanding the users via ethnographic research• Descriptions of users:
– Demographic characteristics– Computer usage background– Job description and work environment– Disabilities:
• Color blindness• Language issues• Typing issues
• Personas – archetypes of users, describing behavior patterns, goals, skills, attitudes, and environment (Cooper, 2003)
Requirements
The study of people in their natural settings; a descriptive account of social life and culture, based on qualitative methods (e.g. detailed observations, unstructured interviews, analysis of documents)
The study of people in their natural settings; a descriptive account of social life and culture, based on qualitative methods (e.g. detailed observations, unstructured interviews, analysis of documents)
![Page 8: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/8.jpg)
8
RequirementsWhat do Users Need?
• Defining user interface requirements
• Based on task analysis:
– Task definition
– Context definition
• Several ways to accomplish task analysis:
– Ethnographic research
– Scenario-based analysis
– Discussion with users and subject-matter experts (SME)
Requirements
![Page 9: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/9.jpg)
9
RequirementsWhat do Users Want?
• When asking users, they often:– Provide their attitudes, not their needs– Bend the truth to be closer to what they think you want
to hear– Rationalize their behavior
“I would have seen the button if it had been bigger”
• Instead of asking users what they want:– Watch what they actually do– Do not believe what they say they do– Definitely don't believe what they predict they may do
in the futureRequirements
![Page 10: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/10.jpg)
10
Non-functional Requirements
Performance, Reliability, Scalability, Security…
• Example: Response time (Nielsen, 1994)0.1 sec – the user feels that the system is reacting instantaneously1 sec – the user will notice the delay, but his/her flow of thought stays uninterrupted10 sec – the limit for keeping the user's attention focused on the dialogue
Requirements
![Page 11: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/11.jpg)
11
DesignMental Models
Also called conceptual model
• What a person thinks is true about a system, not necessarily what is actually true
• Similar in structure to the system that is represented
• Simpler than the represented system. A mental model includes only enough information to allow accurate predictions (i.e. no data structures)
• Allows a person to predict the results of his actions
Design
![Page 12: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/12.jpg)
12
System Model of an ArticleSystem Model of an ArticleUser Model of an ArticleUser Model of an Article System Model of an ArticleSystem Model of an ArticleUser Model of an ArticleUser Model of an Article
DesignMental Models – Example 1
The article’s body and meta-data are conceived as a whole.
An article’s meta-data is available in one database and its data is available in a separate database.
Design
![Page 13: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/13.jpg)
13
DesignMental Models – Example 2
System Model of a search engine
System Model of a search engine
User Model of a search engine
User Model of a search engine
The search engine retrieves return hits directly from their source
The search action does not involve accessing the documents’ source
Design
![Page 14: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/14.jpg)
14
DesignUser Interface Design Guidelines
• Consistency – Appearance, controls, and function– Both within the system and to similar systems
• Feedback • User in control• Recognition rather than recall• Easy reversal of actions
– Error handling• Consider different expertise:
– Novice, intermediate and expert users
Design
![Page 15: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/15.jpg)
15
Designing the Search PageMaking Decisions
• Overall organization:
– Spacious or cramped
– Division of functionality to different pages
– Positioning components in the interface
– Emphasizing parts of the interface
• Query insertion: insert text string or fill in text boxes
• Interactivity of search results
– Retrieve the information from the results
– Narrow the search
• Performance requirements
Design
![Page 16: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/16.jpg)
16
Google Spacious organization
Spacious organization
![Page 17: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/17.jpg)
17
Yahoo! cramped organization
cramped organization
![Page 18: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/18.jpg)
18
AltaVista
Division of functionality to different pages
Division of functionality to different pages
![Page 19: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/19.jpg)
19
ACM Digital Library
Emphasized components
Emphasized components
![Page 20: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/20.jpg)
20
ACM Digital Library advance searchDifferent query insertion ways
Different query insertion ways
![Page 21: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/21.jpg)
21
DesignReturn Hits
• A snippet is a short record that a search system returns to describe and link to a hit.
• Example: Web search “Nielsen evaluation heuristics”
Heuristic Evaluation
... Jakob Nielsen's Online Writings on Heuristic Evaluation. How to conduct a heuristic evaluation; A list of ten recommended heuristics for usable interface design ... www.useit.com/papers/heuristic/ - 5k - Cached - Similar pages
Design
![Page 22: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/22.jpg)
22
Designing the Return HitsMaking Decisions
• Dynamic (generated from query+ document) or precomputed (from document only)
• Content only or with related information
• Highlighting of search terms
• Length vs. number on page
User must understand why the hit was returned
Design
![Page 23: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/23.jpg)
23
Dynamic Return Hits Dynamic snippets
Dynamic snippets
![Page 24: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/24.jpg)
24
Precomputed Return Hits Precomputed snippets
Precomputed snippets
![Page 25: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/25.jpg)
25
Implementation
• Use HTML, Java, Visual Basic, WinForms…
• Make sure the user interface is loosely-coupled with the search engine underneath
User InterfaceUser Interface
Business LogicBusiness Logic
DataData
Implementation
![Page 26: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/26.jpg)
26
Evaluation
• The process of determining the worth of, or assigning a value to, the user interface on the basis of careful examination and judgment.
• Make sure your system is usable before launching it!
• Two categories of evaluation methods:
– Empirical evaluation: with users
– Analytical evaluation: without users
Evaluation
![Page 27: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/27.jpg)
27
Evaluation
But what is usability?From ISO 9241-11, Usability comprises the following aspects:• Effectiveness – the accuracy and completeness with which users
achieve certain goals– Measures: quality of solution, error rates
• Efficiency – the relation between the effectiveness and the resources expended in achieving them– Measures: task completion time, learning time, clicks number
• Satisfaction – the users’ comfort with and positive attitudes towards the use of the system– Measures: attitude rating scales
Evaluation
![Page 28: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/28.jpg)
28
Evaluation with Users
Testing the system, not the users!
Stages of evaluation with users:
Preparation
Sessions conduct
Analysis of results
Evaluation
![Page 29: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/29.jpg)
29
Evaluation with UsersPreparation
• Determine goals of the usability testing“The user can find the required information in no more than 2 minutes”
• Write the user tasks“Answer the question: how hot is the sun?”
• Get participants
– Use the descriptions of users from the requirements phase to detect potential users
Evaluation
![Page 30: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/30.jpg)
30
Evaluation with UsersSessions Conduct
• Conduct the session– Usability Lab– Simulated working
environment
• Observe the user– Human observer(s)– Video camera– Audio recording
• Inquire satisfaction dataEvaluation
![Page 31: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/31.jpg)
31
Evaluation with UsersResults Analysis
• If possible, use statistical summaries
• Pay close attention to areas where users
– were frustrated
– took a long time
– couldn't complete tasks
• Respect the data and users' responses, don't make excuses for designs that failed
• Note designs that worked and make sure they're incorporated in the final product.
Evaluation
![Page 32: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/32.jpg)
32
Evaluation without Users
• Assessing systems using established theories and methods
Evaluation techniques• Heuristic Evaluation (Nielsen, 1994)
– Evaluate the design using “rules of the thumb”• Cognitive Walkthrough (Wharton et al, 1994)
– A formalized way of imagining people’s thoughts and actions when they use the interface for the first time
• Claims Analysis – based on scenario-based analysis– Generating positive and negative claims about the
effects of features on the user
Evaluation
![Page 33: 1 CS 430: Information Retrieval Lecture 13 Usability 1 Guest Lecture: Gilly Leshed](https://reader035.vdocuments.us/reader035/viewer/2022062806/5697bfdc1a28abf838cb0d52/html5/thumbnails/33.jpg)
33
Summary
User InterfaceUser Interface
Business LogicBusiness Logic
DataData
Summary
Requirements
Design
Implementation
Evaluation