1

Guerrilla Usability Testing

@johnwhalen #novaux

XU

Thursday, February 14, 13

2

Gorilla Usability Testing

@johnwhalen #novauxThursday, February 14, 13

Brilliant Experience

@johnwhalen #novaux

User Research Strategy User ExperienceMobile

@johnwhalen #novauxThursday, February 14, 13

User Research

SurveysAdwords

InterviewsEthnography

A/B TestingAnalytics

Usability TestingEye Tracking

Evaluative

Generative

Quantitative Qualitative

@johnwhalen #novauxThursday, February 14, 13

User Research

SurveysAdwords

InterviewsEthnography

A/B TestingAnalytics

Evaluative

Generative

Quantitative Qualitative

@johnwhalen #novauxThursday, February 14, 13

1. Why usability test?

2.When?

3.What?

4. How?

5. Quick Tips

XU

@johnwhalen #novauxThursday, February 14, 13

XU

Why usability test?

@johnwhalen #novauxThursday, February 14, 13

Why usability test?

@johnwhalen #novauxThursday, February 14, 13

XU• Reduce project risk

• Uncover unexpected issues

• Fix problems early when easier/cheaper to fix

• Test assumptions / solve opinion battles

• Set baselines & measure improvement

• Improve customer metrics (conversion, retention)

• “It’s so cheap you’d be stupid not to”

Why usability test?

@johnwhalen #novauxThursday, February 14, 13

XU

Why guerrilla usability testing?

@johnwhalen #novauxThursday, February 14, 13

A. Because you don’t have $30-60k to pay for a full-blown Usability Test

@johnwhalen #novauxThursday, February 14, 13

B. You want to do this quickly and often to refine your designs

@johnwhalen #novauxThursday, February 14, 13

C. Because Eric Ries says so!@johnwhalen #novaux

Thursday, February 14, 13

1. Why usability test?

2. When?

3. What?

4. How?

5. Quick Tips

XU

@johnwhalen #novauxThursday, February 14, 13

XU

When to usability test?

Just afterusers revolt?

@johnwhalen #novauxThursday, February 14, 13

XU

When to usability test?

@johnwhalen #novauxThursday, February 14, 13

XU

When to usability test?

• When you first get the idea and commit it to paper

• When you have several possible design solutions

• When you have a new/updated feature

• When there is just enough code to test (but not perfect)

• Ahead of a sprint

@johnwhalen #novauxThursday, February 14, 13

1. Why usability test?

2. When?

3. What?

4. How?

5. Quick Tips

XU

@johnwhalen #novauxThursday, February 14, 13

XU

What to test?

Clickable Prototype

Paper Prototype

Full Interaction

@johnwhalen #novauxThursday, February 14, 13

1. Why usability test?

2. When?

3. What?

4. How?

5. Quick Tips

XU

@johnwhalen #novauxThursday, February 14, 13

XU

$40,000 Eye Tracker? Probably not...

@johnwhalen #novauxThursday, February 14, 13

XU

Equipment

Silverback Camtasia

@johnwhalen #novauxThursday, February 14, 13

Test Plan

• Goals of the test

• Profile of desired test subjects

• Define tasks

• Write test script (intro, thank you)

• Design a post-test survey

@johnwhalen #novauxThursday, February 14, 13

Tasks

Bad: Search for a bookcase.

Good: You have more than 200 books in boxesin your living room. Find a way to organize them.

[Don’t lead the witness, let them answer the questions.]

@johnwhalen #novauxThursday, February 14, 13

Recruiting

Hi,can I film you with

my laptop?

@johnwhalen #novauxThursday, February 14, 13

Recruiting

• Bring a second person (opposite gender)

• Recruit friends of friends

• Consider posting adds in relevant groups online

• If needed, use recruiter

• Recruit 4-8 per test per user type

@johnwhalen #novauxThursday, February 14, 13

Record & Share Results

@johnwhalen #novauxThursday, February 14, 13

Record

• Record users demographics, goals

• Make post-it note for each major event

• Record what user expected to happen

• Write down other apps / tools they regularly use

@johnwhalen #novauxThursday, February 14, 13

1. Why usability test?

2. When?

3. What?

4. How?

5. Quick Tips

XU

@johnwhalen #novauxThursday, February 14, 13

Plan

• Define the goals of the test• Outline research questions

• Discuss user personas and tasks

@johnwhalen #novauxThursday, February 14, 13

Prepare your recruiting pitch

"Hi,

I'm from organization x

and we are doing some research

into how people use our website for y. I'm

just going around asking people to have a

quick look at our website, and recording what

they think about it. It is for internal use within

the company. Would you be interested in

taking part...it will literally only take a

couple of minutes..."

@johnwhalen #novauxThursday, February 14, 13

Bring along one of: developer, PM, designer, etc.

@johnwhalen #novauxThursday, February 14, 13

Listen actively

• Why?

• What were you expecting to happen?

• What are you thinking?

• Can you say more about that?

• What are you trying to do?

• If there was one thing you could change, what would it be?

@johnwhalen #novauxThursday, February 14, 13

Be a good moderator

• Be friendly and put participant at ease• Start with a simple task

• Don’t give hints or ask leading questions• Don’t rescue people too soon

• Don’t ask opinion-based questions• Give encouraging but non-committal feedback

@johnwhalen #novauxThursday, February 14, 13

Prioritize findings! (Top 5)

@johnwhalen #novauxThursday, February 14, 13

Test!

Revise Prototype!

Test!

Revise Prototype!

Immediately make changes

@johnwhalen #novauxThursday, February 14, 13

Revised as soon as a

problem is identified>

Test!

Revise Prototype!

Test!

Revise Prototype!

Immediately make changes

@johnwhalen #novauxThursday, February 14, 13

37

Guerrilla Usability Testing

@johnwhalen #novaux

XU

Thank you!

Thursday, February 14, 13

John Whalen

@johnwhalen

User Research Strategy User Experience

BrilliantExperience.com

jw@brilliantexperience.com

Mobile

Thursday, February 14, 13