usability 2.0
DESCRIPTION
Talk for the Vancouver User Experience group on October 16, 2007 about the user experience of usability projects and how we've re-designed our process.TRANSCRIPT
Usability 2.0usability as a conversation
Analytic Design Group
Analytic Design Group
Suite 300 - 1300 RichardsVancouver British ColumbiaV6B 3G6 Canadawww.analyticdesigngroup.com
Karyn ZuidingaSenior Information Architectoffice: 604 669 7655mobile: 778 386 [email protected]
Suite 300 - 1300 RichardsVancouver British ColumbiaV6B 3G6 Canadawww.analyticdesigngroup.com
Greg BellUser Experience Designeroffice: 604 669 7655mobile: 778 995 [email protected]
usability as a conversationUsability 2.0
*disclaimerWork in progress.
Ever done a usability test, gone to the trouble of writing that killer report and seen the results go nowhere?
Ever have to read someone else’s usability test report and think you’ve found the cure for insomnia?
0
1
2
3
4
5
0 1 2 3 4 5
Num
ber
of U
sabi
lity
Repo
rts
to R
ead
Hours slept during office hours
Usability Reports vs Office Hours Slept
The user experience of usability.
Mobile Device OOBE Testing
Mobile Device OOBE Testing
Out of Box Experience
Post-test: - Getting Started Guide- Screen Clings- Other In-Box Materials- Device Changes
The Situation (6 months ago)
Limited client engagement.
Results were not getting to who they needed to.
Discussion ends after reporting.
So, we did what we do best.
User Experience Analysis of Usability Testing
Proj
ect
Kick
Off
Project Completion
Usability Project
Proj
ect
Kick
Off
Project Completion
Usability Project
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
Problem: How do we engage clients and get them excited about testing?(and get more people to show up)
Collaborative Planning with Client
Wiki space for Task List & Test Plan
Engage clients on a deeper level before beginning testing.
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
Problem: Difficult to get a recruiting company to engage in finding the RIGHT people.
Recruiting Screener
Personas for recruiters.
Recruiters engaged on a new level and the testing was more successful.
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
Problem: Difficult to record easy to analyze data
Problem: Difficult to get notes from the entire team who is attending testing.
Problem: Difficult to create video clips after testing.
1. The People2. The Notepad3. The Video System
The People1. The Facilitator2. The Observer3. The Schmoozer
Invite as many people from the client team as possible.
1. The People2. The Notepad3. The Video System
9 Send & Receive Email
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
Study: _____________________________
User ID: ___________
Notes By: ___________________________
1 Make a Phone Call
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
2 Set up Voicemail
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
3 Send a Text Message
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
4 Take a Photo & Send MMS
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
5 Make a Video Share Call
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
6 Set Up & Send an IM
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
7 Go to a Website
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
8 Set Up Email
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
10 Sync Calendar & Contacts
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
11 Set Up TeleNav
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
12 Bluetooth
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
The Notepad
9 Send & Receive Email
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
Study: _____________________________
User ID: ___________
Notes By: ___________________________
1 Make a Phone Call
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
2 Set up Voicemail
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
3 Send a Text Message
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
4 Take a Photo & Send MMS
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
5 Make a Video Share Call
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
6 Set Up & Send an IM
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
7 Go to a Website
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
8 Set Up Email
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
10 Sync Calendar & Contacts
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
11 Set Up TeleNav
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
12 Bluetooth
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
9 Send & Receive Email
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
Study: _____________________________
User ID: ___________
Notes By: ___________________________
1 Make a Phone Call
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
2 Set up Voicemail
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
3 Send a Text Message
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
4 Take a Photo & Send MMS
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
5 Make a Video Share Call
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
6 Set Up & Send an IM
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
7 Go to a Website
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
8 Set Up Email
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
10 Sync Calendar & Contacts
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
11 Set Up TeleNav
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
12 Bluetooth
Finished w/ Ease Completed w/ Difficulty Failed to Complete
Use GSG? NoYes
The Used Notepad
Get clients to use sheets and record data.
1. The People2. The Notepad3. The Video System
Our Setup
Participant
Our Setup
Participant
Facilitator
Our Setup
Participant
Facilitator
Video Camera
Our Setup
Participant
Facilitator
WebCam
Video Camera
Our Setup
Participant
Facilitator
WebCam
Video CameraTwo Way Glass
Observer
Our Setup
Participant
Facilitator
WebCam
Video CameraTwo Way Glass
Observer
Our Setup
Network / Router
Participant
Facilitator
WebCam
Video CameraTwo Way Glass
Client Team &Schmoozer
Observer
Our Setup
Network / Router
Using these tools, we were able to create better data that was easier to analyze.
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
Proj
ect
Kick
Off
Project Completion
Planning Recruiting Usability Testing Data Analysis Reporting
Usability Project
“I recently asked 258 usability practitioners which methods they use to communicate findings from their studies”
- Jakob Nielsen, 2005 http://www.useit.com/alertbox/20050425.html
“I recently asked 258 usability practitioners which methods they use to communicate findings from their studies”
- Jakob Nielsen, 2005 http://www.useit.com/alertbox/20050425.html
http://www.happywebbies.com/download
Formal Usability Reports vs. Quick Findings
Formal Usability Reports vs. Quick Findings
Formal Usability Reports vs. Quick Findings (2)
Formal Usability Reports vs. Quick Findings (2)
Formal Usability Reports vs. Quick Findings (2)
The typical usability report life-cycle:
http://www.flickr.com/photos/phooky/1390640732/
http://www.flickr.com/photos/seanstayte/660772151/
Do I really have to read all of this?
Maybe, a DVD arrives in the mail a few weeks later with video.
Problem: Usability reports are boring and nobody wants to read them.
Problem: Usability reports are not shared among the people who need them most.
Problem: There is a disconnect between video highlights and the findings in the report.
Test Debrief Session
http://www.flickr.com/photos/phooky/1390640732/
Findings Module
Easy to Parse Data
Utility Functions
Invite a Colleague via Email
Montages
Sortable Data
Demo
Clients engaged on report.
Delivery method improved reporting
While hours increased on creating reports, the value was well worth it for our clients.
What’s Next?
Discussion threads on findings
Assignable action items on all recommendations.
Web app to create reports
Takeaways
Get your reports online and interactive.
Engage with client’s product team as much as possible.
Re-design your process!
Thank you.
Analytic Design Group
Analytic Design Group
Suite 300 - 1300 RichardsVancouver British ColumbiaV6B 3G6 Canadawww.analyticdesigngroup.com
Karyn ZuidingaSenior Information Architectoffice: 604 669 7655mobile: 778 386 [email protected]
Suite 300 - 1300 RichardsVancouver British ColumbiaV6B 3G6 Canadawww.analyticdesigngroup.com
Greg BellUser Experience Designeroffice: 604 669 7655mobile: 778 995 [email protected]
http://creativecommons.org/licenses/by-nc/2.5/ca/