how to conduct usability studies: a librarian primer
Post on 17-Oct-2014
591 views
DESCRIPTION
Slides used for RUSA webinar on September 4, 2013TRANSCRIPT
How to Conduct Usability Studies A Librarian Primer
Tao Zhang, PhD Digital User Experience Specialist Purdue University Libraries [email protected]
Sponsored by ALA RUSA #RUSAUX
Ilana Barnes Business Information Specialist Purdue University Libraries [email protected]
Learning Objectives
2
• Participants will be able to describe the usability concept and it's applications to libraries.
• Participants will be aware of the basic work!ows and techniques needed when doing usability tests.
• Participants will be aware of resources for usability studies and where to "nd out more.
Agenda
3
• What is a usability study? • Usability study process • Conducting the study • Analyzing Results • Communicating • Where to "nd out more
About Us
Ilana Barnes Tao Zhang
4
Tell us about you!
What type of library do you work in? A) Public B) Academic C) Other
How familiar are you with Usability Studies/Tests? A) Very familiar B) Somewhat familiar C) Not familiar but excited to learn!
Chat box: Areas you especially want us to talk about
5
What is User Experience?
6
De"nition
Usability study refers to “evaluating a product or service by testing it with representative users. “
- http://www.usability.gov/how-to-and-tools/methods/usability-testing.html
7
How UX "ts into Reference Services
• Decisions that user experience research can help you with: • What should our libraries front page look like? • What do our undergraduates expect from the library? • What would we like our undergraduates to expect from the library
that they do not know or understand or need?
• Understanding your users aims and needs help you help them and show your impact at the same time.
8
How to plan your usability study topic
• What are my objectives? • How much time do I have? • How many teammates do I have? • Who am I presenting this to? • Is this for an existing product?
9
When to Test
• Early or middle stages of development • Types:
Exploratory | Assessment | Comparison | Validation
• Start preparing at least three weeks before actual test
• DO pilot tests
10
Usability Study Process
11
Recrui'ng Preparing Conduc'ng Analyzing Presen'ng
Choosing Features Creating Tasks Preparing Test Script
Recruiting
• Determining the target audience (behavior) • Finding representative members
– Behavioral criteria, demographics – Avoid sampling bias
• Convincing participants to come – Incentives – Informed content (duration of study, data to collect)
• Scheduling – Reminding – Deal with no shows
12
Recruiting
Best people to invite: • People who are going to need the service in the
near future • People who have used a competing/previous
version service in the recent past Number of participants: • 5-8 • Also consider representativeness and tasks
13
Choosing Features
• Input from: Development team | Stakeholders | Users • Content inventory & task analysis
14
Creating Tasks
Tasks should be: • Reasonable and doable • Described in terms of end goals • Speci"c Run through tasks to see how long each will take Determine what quantitative data to collect
15
Discussion
You need to do a usability study of your library’s website, because users "nd it difficult to "nd information they want on the website. What are some possible tasks you can use in the usability study?
16
Writing A Study Script
• Introduction (5 min) • Warm-up interview, demographics survey (5 min) • Scenarios and tasks (20-30 min) • Interview (10-15 min) • Wrap-up (5 min)
17
Conducting the Study
The physical layout with observer
18
Conducting the Study
The physical layout without observer
19
Conducting the Study
TechSmith Morae www.techsmith.com/morae.html
20
Recorder
Observer #1
Participant Test Moderator
Observer
Manager
Observer #2 …
Conducting the Study
21
http://silverbackapp.com/
Conducting the Study
Testing on mobile devices
22
IPEVO Ziggi-HD High-Definition USB Document Camera
DIY Mobile Usability Testing - SXSW Interactive 2012 http://www.slideshare.net/beleniq/diy-mobile-usability-testing-sxsw-interactive-2012
Moderation
• Follow script but allow some !exibility • Encourage participants to “think aloud” • Ask about expectations • Ask “why” a lot • Suggest solutions when participants are stuck • Discuss with observer right after test
23
Two Basic Techniques
Observation Interview
24
Observation
• Watch, Understand • Limit the time users have to perform a task • Encourage users to think aloud with neutral
prompts • Elicit detailed information
25
Interview
• Introduction • Warm-up • General issues • Deep focus • Wrap-up
26
Nondirected Interview
Prepare nondirected questions • Focused on the person answering it • Focused on experience, not extrapolation • Concentrate on immediate experience • Avoid judgmental language • Focus questions on a single topic • Keep questions open-ended • Avoid binary questions
27
Nondirected Interview
Examples “Is this a useful feature?” “Is this feature valuable to the work you do right now?” “Do you think this feature would be better if it was also available on smart phones?”
28
Nondirected Interview
Running the interview • De"ne (technical) terms • Don’t force opinions • Restate answers • Never say the participant is wrong • Work with a partner
29
Analyzing Results
• Collecting observations – Notes from moderator and observer – Quantitative information: task successfulness,
navigational path, number of steps, errors, time to complete, subjective ratings (System Usability Scale)
– Qualitative feedback from participants
• Extracting trends – Repeated concerns – Multiple issues with possible common origin – Identify effects on user experience
30
Analyzing Results
• A list of usability issues – Quantitative and qualitative data as evidence – Possible causes – Effects on user experience
• Assign importance/severity • Suggest solutions
31
Communicating Findings
Usability test report • Project overview • Participants • Testing environment • Tasks • Procedure • Key "ndings
– Issues – Recommendations
32
Communicating Findings
Presentation • Prepare the audience • Use supporting evidence (e.g., highlight videos) • Emphasize user perspectives • Use numbers carefully (because of small sample) • Leave enough time for questions
33
Communicating Findings
Common problems: • “This is not statistically signi"cant.” • “This user is our typical user.” “This user does not
have relevant experience.” • “User X did Y; therefore, everyone must do Y.” • “More than half of users did not like this design;
therefore, we should change it.”
34
What’s Essential?
• Develop a study plan • Find participants • Conduct the sessions • Debrief with observers and decide together • Present results
35
Discussion
Think of an existing design that you want to evaluate for effectiveness • What are my objectives? • What types of resources do I have? • Who am I presenting this to?
36
Other Types of Usability Studies
• Remote usability testing http://www.userzoom.com/
• Eye tracking
37
http://www.normalmodes.com/blog/2009/09/28/eye-tracking-heatmap-gallery-a-preview-discussion-of-ui-considerations/
Where to Find Out More
• 6 Must-See Usability Testing Videos http://blog.usabilla.com/7-must-see-usability-testing-videos/
• Moderating Usability Tests Joe Dumas & Beth Loring (2008)
• A Field Guide to Usability Testing Smashing eBook https://shop.smashingmagazine.com/guide-to-usability-testing-1.html
• Handbook of Usability Testing Rubin & Chisnell (2008)
• Don’t Make me Think by Steve Krug Chapter: “Usability Testing on 10 cents a Day”’
38
Learning Objectives
39
• Participants will be able to describe the usability concept and it's applications to libraries.
• Participants will be aware of the basic work!ows and techniques needed when doing usability tests.
• Participants will be aware of resources for usability studies and where to "nd out more.
Agenda
40
• What is a usability study? • Usability study process • Conducting the study • Analyzing Results • Communicating • Where to "nd out more
Works Cited
• 6 Must-See Usability Testing Videos - The Usabilla Blog. (n.d.). Retrieved from http://blog.usabilla.com/7-must-see-usability-testing-videos/
• Affairs, A. S. for P. (2013). Usability Testing. Retrieved from http://www.usability.gov/how-to-and-tools/methods/usability-testing.html
• Clear, Consistent, Capable, NICE» Blog Archive » Eye Tracking Heatmap Gallery: A Preview & Discussion of UI Considerations. (n.d.). Retrieved from http://www.normalmodes.com/blog/2009/09/28/eye-tracking-heatmap-gallery-a-preview-discussion-of-ui-considerations/
• Dumas. Joe, & Loring, B. (2008). Moderating Usability Tests. • DIY Mobile Usability Testing - SXSW Interactive 2012. (n.d.). Retrieved from
http://www.slideshare.net/beleniq/diy-mobile-usability-testing-sxsw-interactive-2012 • Morae usability testing software from TechSmith. (n.d.). Retrieved from
http://www.techsmith.com/morae.html • Krug, S. (2005). Don’t Make Me Think. • Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing. • UserZoom | zooming in on the user experience. (n.d.). Retrieved from http://www.userzoom.com/ • Silverback — guerrilla usability testing. (n.d.). Retrieved from http://silverbackapp.com/
41
Final Notes
Evaluation of this webinar Questions? Tao Zhang [email protected] Ilana Barnes [email protected]
42