introduction to team- based usability testing as companies design more for usability and...
TRANSCRIPT
![Page 1: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/1.jpg)
Introduction to Team-based Usability Testing
“As companies design more for usability and understanding, they will discover a competitive edge, for these principles save customers time
and money while increasing morale.” - Donald Norman
![Page 2: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/2.jpg)
Objectives
Usability overview Setting objectives Preparing the test plan Conducting the test
![Page 3: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/3.jpg)
Usability
Usability “means that the people who use the product can do so quickly and easily to accomplish their own tasks.”
- Redish and Dumas, A Practical Guide to Usability Testing
![Page 4: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/4.jpg)
Usability
Is not functionality- Functionality is how the system works
Is not beta testing- Beta testing is unfocused and at end of process
Is not market research- Market research is about customer demands
![Page 5: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/5.jpg)
When to test
Design phase- surveys and focus groups- low fidelity prototype testing- high fidelity prototype testing- heuristic evaluation
Development phase- usability evaluation
![Page 6: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/6.jpg)
Advantages of a usability lab
Lab is helpful for tightly controlling the variables
Lab helps capture low-level details - keyboard monitoring- videotaping for later analysis- eye movement tracking
Labs allow less intrusive observations
![Page 7: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/7.jpg)
Disadvantages of a usability lab
Labs are intimidating to users Labs are rarely portable so they cannot
go into the users’ environment Sometimes low-level information is not
as important as making high-level design decisions
Labs are expensive
![Page 8: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/8.jpg)
How long will the test take?
How much usability testing has already occurred?- Have users been profiled?- Has a task analysis occurred?- Have usability objectives been set?
How complex is the product? How much of the product are you going
to test?
![Page 9: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/9.jpg)
How long will the test take?
How many test participants do you need to get the information you want?
How much training or prior experience does the test team have with the product?
How many different usability objectives do you need to test?
![Page 10: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/10.jpg)
Who should run the test?
The most successful approach to usability testing is a cross-functional, team-based approach- People bring varying expertise to the team- All groups benefit from experience with
users- Interpretation of challenges and design
decisions are more creative with teams
![Page 11: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/11.jpg)
Who should run the test?
Team-based testing provides a great way to get support involved early
Developers learn to “hear” user questions when they design
Writers and trainers can help design communication into the product
Human factors experts and user interface designers can provide helpful principles about users and design
![Page 12: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/12.jpg)
Who should run the test?
More people can be involved in planning the test than in conducting the test
Target 3-5 people for conducting the test Sometimes best if the developers are not
actively involved in running the test, although they should observe if possible
![Page 13: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/13.jpg)
How do you ensure success?
You need the buy-in of upper management if the results are to be applied
You need the cooperation of development to ensure they don’t feel attacked
You need the support of marketing, sales, and product management for funding
You need the support of your immediate manager to spend time testing
![Page 14: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/14.jpg)
How do you ensure success?
Pick a manageable portion of the product to test
Select a high-profile project Select an area that will produce
significant results Advertise your successes (become part
of the sales story)
![Page 15: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/15.jpg)
Stages of a usability test
Setting objectives Creating the test plan Creating questionnaires Selecting test participants Training the team Conducting the pilot test Conducting the tests
![Page 16: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/16.jpg)
Usability objectives
Created during user/task analysis and product scoping
Must be measurable Should indicate:
- Type of user- Task to be performed- Specific performance criteria
![Page 17: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/17.jpg)
Defining usability objectives
Five attributes of usability (Nielsen, 1994):- Learnability: system is easy to learn so users can
get started quickly- Efficiency: system should be easy to use, resulting
in high productivity- Memorability: system should be easy to remember- Errors: system should have low error rate and allow
error recovery- Satisfaction: system should be pleasant to use
![Page 18: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/18.jpg)
Examples
Novice PC users can change the ink cartridge in their color inkjet printer in less than 5 minutes using the user manual
Users will be able to install and configure the default application in less than 15 minutes
Users should rate the product as either “easy to use” or “very easy to use” (4 or higher on a 5 point scale)
![Page 19: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/19.jpg)
Considerations
A single usability objective may result in multiple design decisions
Emphasizes importance of having cross-functional team
Focus on objective that are most important to the success of the product
![Page 20: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/20.jpg)
Test plan
Explains what you are testing and how you will conduct the test
Do not skip this step- Important to have a plan that everyone can
agree to and understand- Helps communicate your intentions to
others in the organization Do not overdo this step
![Page 21: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/21.jpg)
Test plan
Purpose- Why are you conducting the test?
Problem Statement/Test Objectives- What usability objectives are you testing?
User Profile- Who are your users and their defining
characteristics?
![Page 22: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/22.jpg)
Test plan
Method- Details how you will run the test
Task List- List of the tasks to be completed, projected
times for completion, required state of system to test tasks
Test Environment/Equipment- List what you need to conduct the test
![Page 23: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/23.jpg)
Test plan
Test Team Roles- Define what each person will do
Evaluation Measures- What data you will collect through the test
Conclusion- What will happen after the test - reports,
design meetings, presentations
![Page 24: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/24.jpg)
Pre-test questionnaire
Ensure that participant meets user profile Gather additional detail about the
participants’ knowledge and experience
![Page 25: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/25.jpg)
Post-task questionnaire
Asks questions about a task as soon as it is completed
Captures feedback when it is fresh Can help measure change in attitude
toward the task Keep it short
![Page 26: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/26.jpg)
Post-test questionnaire
Measure satisfaction and reaction to the product and the overall experience
Ask participants to elaborate on a few critical areas/issues
![Page 27: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/27.jpg)
Training the team
Make sure everyone understands the usability objectives and the testing procedure
Define everyone’s roles and discuss how they should behave during the test
![Page 28: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/28.jpg)
Conducting a pilot test
Conduct a practice test using the complete team, all prepared materials, and the actual facilities
Make sure at least one person observes the test to make recommendations
Refine any of the materials or facilities as necessary
![Page 29: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/29.jpg)
Selecting test participants
Number of participants depends on the number of user groups (differentiating characteristics)
Target 3-5 users from each user group Best if users come from the real user
population rather than internally
![Page 30: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/30.jpg)
Observers
Not actively involved in conducting the test, but can benefit from observing
Must not comment or react to the participant’s actions or comments
Should be separated from the actual room where the testing is going on
Should be removed if they cannot avoid feeling defensive or being intrusive
![Page 31: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/31.jpg)
Data logger
Be familiar with any data logging forms Establish standard codes ahead of time
for recording events and comments Remain open to unforeseen directions Record user comments that summarize
or emphasize good and bad points Keep track of time
![Page 32: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/32.jpg)
Facilitator
Main contact with the participants Conducts the briefing and debriefing Only person to talk to the participants
during the test
![Page 33: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/33.jpg)
Briefing
Thank them for participating in the product evaluation
Assure users that they are not being tested; instead, they are helping you evaluate the product
Explain to them what will happen during the test; they will be especially interested in any recording devices and observers
![Page 34: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/34.jpg)
Briefing
Ask them to sign a consent form and a non-disclosure agreement
Encourage them to think out loud so you can record their actions and the reasons behind their actions
Make sure you explain what they should do if they have questions or experience problems
![Page 35: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/35.jpg)
During the test
Be sensitive to excessive frustration Be ready to handle unplanned situations Remind participants to talk out loud Look and listen for the unexpected Avoid intervening unless necessary; try
using questions to redirect rather than take over as the expert
![Page 36: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/36.jpg)
Debriefing
Get users to elaborate on significant events or comments
Ask for reasons to specific behaviors from the test
Watch the video tape with them and ask them to explain what was going on in their mind
![Page 37: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/37.jpg)
Summary
Build a cross-functional team to:- take advantage of different skills- get “buy in” from different areas- build awareness of usability testing benefits
When you test:- Set measurable objectives- Test with the right people- Be prepared for the test- Share your results
![Page 38: Introduction to Team- based Usability Testing As companies design more for usability and understanding, they will discover a competitive edge, for these](https://reader033.vdocuments.us/reader033/viewer/2022061306/55148855550346d36e8b4d56/html5/thumbnails/38.jpg)
Questions?
Scott DeLoach
Founding Partner, User First Services, Inc.
Certified RoboHELP Instructor and Consultant
404.520.0003
For more information about this presentation and other training seminars, visit www.userfirst.net. This presentation is at www.userfirst.net/teamusability.ppt.