ux 101, usability testing

Download Ux 101, usability testing

Post on 17-Oct-2014




0 download

Embed Size (px)


Observing users interact with your product is hands down the best way to learn what is and what is not working and why. This workshop will teach you how to plan, facilitate, and analyze your own usability tests. We will cover the steps of planning, recruiting, writing tasks, facilitating, analyzing, and reporting the findings. We will discuss the steps then split into groups and put the learning into practice with a quick exercise.


UX 101Usability TestingCori Jones


UX 101


UX 101

AGENDAOverview of usability testingPlanningRecruitingWriting activitiesFacilitationAnalyzing results Sharing findings


UX 101


Is Working Is Not Working

UX 101


User Design Facilitator

UX 101

UX 101

BASICS OF USABILITY TESTING (IDI)Decide what to testDefine who the audience isRecruit 4-6 participants Write 3-5 realistic activities for the participants to completeFind a location and set a dateInvite observers Conduct the test sessionsGather, compare notes and discuss

UX 101

BENEFITS TO DESIGNERSBetter understanding of the users perspective (Mental Model)Uncover unknown or unexpected issuesFix problems early in the process when they are easy to fix Test assumptionsHighlight unnecessary featuresProvide objectivity help and solve opinion battlesSet baselines and measure improvementGet stakeholder buy-in

UX 101

BUSINESS BENEFITSFix problems early in the process when they are cheaper to fixImprove customer satisfaction and retentionIncrease conversion ratesReduce maintenance, training and support costsReduce project riskSet baselines and measure improvement

Video: The ROI of User Experience with Dr. Susan Weinschenk


UX 101


Briefly describe the projectOutline the objectives of the testPick a location and date for the testOutline what is being tested (version or section)

Profile your desired test subjects (Screener for recruiting)

Write activities and the goal of eachDefine what metrics will be measured (times, success rates)

UX 101

ADDRESS LOGISTICSDecide if (or when) the facilitator will be allowed to help usersTime limits on activities, follow up questionsList of needed materials (NDAs, questionnaires, pens & paper, water, incentives, etc.)State of system at start of each session (Cookies and history erased, start page (or place) for each user)What to do between tests


UX 101


Get participants who closely match your target usersCreate a table with key requirementsWrite the Screener (script for recruiting)

Cost per user:Pay a vendor to recruit for you: $100 - $250 + incentive ($5-$100)

Benefit = High quality participants

Recruit yourself (Customers, Craigslist, KSL, friends, family, coworkers): Your time + incentive

Benefit = Inexpensive


UX 101

CHOOSING ACTIVITIES What are the top user tasks?What are your priorities?What new features have never been tested?Competitive features?What areas do you think may be difficult?What should users be able to do?

UX 101

TYPE OF ACTIVITIESFirst impression questions

Good for learning about the image of the site (home pages)

Exploratory taskOpen-ended/ research oriented

Use the website and see if you would invest in this company

Find a cellular phone plan for yourself

Directed TasksSpecific/ answer oriented

Find the contact information for the PR department

How fast can a cheetah run?

UX 101

ACTIVITY-WRITING GUIDELINESConsider the goal of the activityMake it realisticAvoid humorous tasksGive minimal context, not overly scenario-basedKeep it neutral and unbiasedUse language people understandLease out clues or hints

Avoid using wording used in the design

Avoid having micro-steps

Involve team in writing and/or reviewing activities

UX 101

SESSION LOGISTICSPrioritize activities

Is the priority to get through many activities, or hit a select few deeply?

Consider the total session lengthAllow 15-25 minutes for introductions and debrief

Allow 10-20 minutes per activity (depends on the difficulty of the activity)

Make the first task easy Put essential activities early onPrepare additional activities in case of extra time

UX 101

EXERCISE: ACTIVITY CRITIQUE1. Sign up for email exclusives on target.com

Goal: See if they can join the email list

Go to spotify.com, sign up, create a new playlist, then add Ben Kwellers album: Go Fly a Kite to that playlist

Goal: See if people can create a playlist

Your mothers birthday is next week. Send her a bouquet of flowers.Goal: See how people browse the product offerings on your floral site

Add new events to the calendar feature on your phoneGoal: Study adding events

You are organizing a team of runners for an upcoming relay race. The team wants matching shirts for the event. The shirts need to be green, have a white logo on the front, and yellow racing stripes on the back. Go to www.customink.com and order 20 of these shirts.

Goal: See if users can customize a design

UX 101

EXERCISE: ACTIVITY WRITINGChoose an electronic device that you have with you todayWrite 3 activities that can be done using the deviceWrite the activity and the goal for each one



UX 101

UX 101

UX 101

WHILE FACILITATINGGive participants one activity at a time Monitor


Session time

User comfort



ObserveTake notesConsider whether to interrupt or ask questions

UX 101

EXAMPLE TEST SESSIONThe facilitator welcomes the participant and explains what the test is about. They then ask any pre-test or demographic questions.The facilitator explains thinking aloud and asks if the participant has any additional questions. The facilitator explains where to start.The participant reads the first activity aloud and begins completing the activity, vocalizing their thoughts as they go.The observers take notes of the participants behaviors, comments, errors and completion (success or failure) on each activity.The session continues until all task scenarios are completed or time allotted has elapsed.The facilitator either asks the end-of session subjective questions or sends them to an online survey, thanks the participant, gives the participant the agreed-on incentive, and escorts them from the testing environment.The facilitator them resets the materials and equipment, speaks briefly with the observers and waits for the next participant to arrive.

UX 101

INTERACTING WITH THE PARTICIPANT Give subtle acknowledgements

Uh-huh, OK, nodding

Refrain from being interruptive or chatty

Stay neutralDont ask leading questions

Avoid prompting

Avoid explaining of defending interface (dont test your own stuff)Avoid answering questions or helping out too early

Get clarificationEchoing technique

Trailing-off technique

UX 101



UX 101

EXERCISE: FACILITATIONBreak into groupsChoose roles

One facilitator

One participant

The rest will be observers

Use the activities you wrotePrepare to do a full usability sessionWhen groups are ready, participants move to another groupRun the session

Everyone should do their part (role)

Observers: note findings as well as the facilitation techniques


UX 101


Focus on frequency and team involvement over precision and depth

Same day meeting with all who participated

Discuss and agree on obvious usability issues

Side effect: User empathy throughout entire team and process

Traditional approachFocus on being thorough and precise

Reference the tests goals

Gather all notes and do affinity diagramming

UX 101

UX 101

GUIDE TO AFFINITY DIAGRAMMINGLow-tech method, flexible, easy to learnEveryone in the room participatesOne facilitatorIdentify all issues during test sessionsGroup issues into categoriesVote on most severe issuesGoal: To have all issues sorted in categories and assigned a priority rating

UX 101

AFFINITY DIAGRAMMING STEPSWrite issues from tests on post-it notes and place on the wall

one issue per note (good or bad)

Use a different color post-it for each user

Find two issues that go together, give category name and post elsewhere in room

Continue grouping and naming categories until all post-it notes are placed

No talking when doing initial groupings

Give time limit

Walk around room gallery style and review/refine categoriesDiscuss categories once you are refining them

Vote on severity on top issuesGive each attendee 2-3 votes to determine high priority issues


UX 101


Frequency and team involvement are key

Make and share video of sessions

Gather and share session notes

Keep a good thing going

leverage enthusiasm to help make usability testing a regular activity

Traditional approach (make a report)Helpful if the team cant or wont participate

Positions you as the usability expert and defender of the user needs

The report must make findings understandable and actionable

Be a historical document

Include screenshots

Be objective (dont get caught up in selling your recommendations)

UX 101

WHAT SHOULD BE IN YOUR REPORTLook for both positive and negative findingsDescribe what happened

Explain WHY, not just what

Distinguish between fact and interpretationsInclude simple quantitative data

Success (pass/fail) rates