which research when

You’re Doing The Wrong Research

Laura Klein@lauraklein

[email protected]://www.usersknow.com

Wednesday, December 5, 2012

mailto:[email protected]


http://usersknow.blogspot.com


GET OUT OF THE BUILDING!

@lauraklein


What You Expect

@lauraklein

1. Get out of the building2. Talk to “users”3. Profit


What Really Happens

@lauraklein

1. Get out of the building2. [A lot of really hard work]3. Profit????


@lauraklein

Let’s figure out that middle bit...


How Many Ways Are You Listening To Users?

@lauraklein

Did you know that there are dozens of ways to do this?


Here Are A Few Of Them

@lauraklein

Task Based Usability

Analytics

Observational Usability

NPS Surveys

New User Interviews A/B Testing

Landing Pages

Wizard of Oz

Product Stubs (Fake Doors)

Customer Development Interviews

Prototype Usability Sales

SurveysFocus Groups

Brain Imaging (yes, really!)

Guerilla User Tests

Unmoderated Testing

Click Tests


@lauraklein

I’m not going to teach you how to do all of those today.

(You’re welcome.)


@lauraklein

What you want to learn determines the type of research you need to do!


You Only Need To Know About Two Types Of Research*

@lauraklein

Generative & Evaluative

*for now


Generative Research

@lauraklein

You want to know what to do next.


You Don’t Have An Idea

@lauraklein

Find a great idea by learning about problems within specific markets.

Best Methods:•Contextual Inquiry•Customer Development Interviews


Like This

@lauraklein


You Have A Great Idea (maybe)

@lauraklein

Learn more about your potential customers and figure out what’s necessary for an MVP.

Best Methods:•Contextual Inquiry•Customer Development Interviews•Observational Studies of Competitive Products


Like This

@lauraklein


You Have An MVP & Want To Know How People Use It

@lauraklein

Understand more about your users and what they’re really doing with your product.

Best Methods:•Observational Studies•Diary Studies•Analytics


Like This

@lauraklein


Or This

@lauraklein


See A Pattern?

@lauraklein

Generative research is about Generating Ideas.


Evaluative Research

@lauraklein

You want to know if what you did was right.


You’re Doing A Redesign & Want To Make Sure It’s Better

@lauraklein

See whether users can perform key tasks in your product easily and without confusion.

Best Methods:•Task Based Usability Tests•Observation of Interactive Prototypes


Like This

@lauraklein


You Created A New Landing Page & Want To Test Messaging

@lauraklein

Understand whether users really understand your new tag line, and whether your messaging conveys what you think it should.

Best Methods:•Five Second Tests•Guerilla Usability•A/B Tests


Like This

@lauraklein


You’re Thinking Of Adding A Feature

@lauraklein

Validate whether or not it’s a good idea before spending time designing and building it.

Best Methods:•Feature Stubs (Fake Doors)•Wizard of Oz Features


Like This

@lauraklein

“This feature is coming soon!”


You Added A Feature & Want To Know The Impact

@lauraklein

Learn whether your changes improve both the user experience and your key business metrics.

Best Methods:•Customer Interviews•Observation Studies•A/B Test Against Control•Analytics


Like This

@lauraklein


See A Pattern?

@lauraklein

Evaluative research is about Validating Hypotheses.


@lauraklein

Did you notice that some methods are quantitative?

That’s because Qualitative Research sucks for some things.


@lauraklein

Quantitative Research tells you what.

Qualitative Research tells you why.


@lauraklein

Some Common Questions


What Are Some Good Research Methods That Don’t Involve

People Outside The Company?

@lauraklein

(This page intentionally left blank)


But What If I’m Solving My Own Problem?

@lauraklein

That depends. How many of you are there?


What If I Don’t Have Any Users?

@lauraklein

Get some.


But What If I Can’t Get Enough Users For Quantitative Research?

@lauraklein

Rely more heavily on Qualitative Research.


Ok, Your Turn


Exercise!

@lauraklein

Type Metric Plan

Fill this in with real life examples of some research.


Example!

@lauraklein

Type Metric Plan

GenerativeObservational

TestingRevenue

Observe 4 people who have never used the product before go through the payment flow to find pain points that are causing drop off in the funnel. Generate ideas for fixing those usability bugs.

Evaluative5 Second Test

Sign Up

Test whether potential users understand the new messaging on the home page by running 20 people who have never seen the site before through a 5 second test and asking the question, “What does this product do?”


More Questions?You know where to find me...

@[email protected]

http://www.usersknow.com

pssst...this will all be in the book! To be published by O’Reilly in early 2013.






which research when

Technology

best methods

lauraklein youre

good research methods

lauraklein im

lauraklein exercise

laurakleinlike thiswednesday

types of research

type of research