Collaborative Research

with Erika Hall (@mulegirl)

The Conference | Malmö

Hello!

I have a question...

Do you enjoy being right?

You are correct!

YESSS!

p0wned!

No.

?

>? !

Dogma

Flickr/Chris Voll

Ego!

Why?

People!

Collaboration!

TextAgenda

9–10:30

10:30–10:40

10:40–12:00

Introduction.

The research process

Questions and activities

Understanding your organization

Break

User research

Analysis

Models and reports

Finale

Barriers

Overcoming Objections

Laziness

Fear

Lunch

Following

Losing Control

Sharing

Shared Understanding

We don’t havethe time.

We don’t havethe money.

We don’t have the expertise.

We’re already A/B testing

Everyone wants better products, faster.

No one wants to read a report.

What is your experience?

Research + Collaboration

A design project isa series of decisions.

Data doesn’t change minds.

Whatis

What ought to be

Design-Led

Research-Led

ExpertMindset

ParticipatoryMindset

Users seen as subjects Users seen as partners

Design-ledwith

expert mindset

Design-ledwith

participatory mindset

Research-ledwith expert mindset

Research-ledwith participatory

mindset

Dubberly Design Office

Goal Driven

Skeptical Mindset

Increase chance of successReduce risk

Willing to question the value of any approach

Team + Goal + Reality = Good

One Simple

Process

Form Questions

AnalyzeData

GatherData

Form Questions

AnalyzeData

Think Critically

Form Questions

AnalyzeData

Observe

Form Questions

AnalyzeData

Interview

Form Questions

AnalyzeData

Read

Form Questions

AnalyzeData

Read

Experiment

Interview

Observe

Think

PersonalView

PersonalView

PersonalView

SharedReality

Researchis a Craft

Form Questions

AnalyzeData

GatherData

Questions determine results.

Questions give research meaning.

Research high-priority questions.

Good Questions

SpecificActionablePractical

A Bad Question

“What do people think about pets?”

A Better Question

“How do single urban adults choose and acquire a pet?”

A Bad Question

“What do people do around here all day?”

A Better Question

“How do editors and designers work together?”

The Best Question

The unknown with the most risk.

Bias

Bias: Something that causes an influence or prejudice

Confirmation Bias:

You selectively weight the information that confirms what you already believe.

Sampling Bias:

Your sample of research subjects isn’t sufficiently representative.

Interviewer Bias:

You insert your opinion into interviews.

Social Desirability Bias

People don’t say the true things that they worry will make them look bad.

Ease

Clear Display

Related

Primed Idea

Good Mood

Feels True

Feels Familiar

Feels Good

Feels Effortless

Daniel Kahneman

Feeling confident?It’s not a good sign.

You might have a bad case ofDunning-Kruger.

Critical Thinking

Critical Thinking

DisciplinedSelf-correctingClearLogical

Uncritical Thinking

“I hate yellow, so a yellow website won’t succeed.”

Critical Thinking

“I hate yellow, but based on the evidence, it might work for our audience.”

Critical Thinking

“I don’t know.”

Activities!

Form Questions

AnalyzeData

GatherData

Questions About

Users

ProductOrg

Competition

InterviewsInterviews

UsabilityTesting

A/BTesting

ContextualInquiry

LiteratureReview

SWOTAnalysis

BrandAudit

Usability Testing

CompetitiveAnalysis

HeuristicAnalysis

Descriptive

Evaluative

Evaluative

Evaluative

Analytic

Analytic

Generative

Descriptive

ResearchActivity

TopicPurpose

Time

Money

Phone InterviewsWhat do we

need to know about?

What kind of decision will it

inform?

How long do we have?

What is our budget?

Contextual Inquiry

In-Person Interviews

Usability Testing

Competitive Analysis

Why not just make a

prototype?

20

If we only test bottle openers, we may never realize customers prefer screw-top bottles.

– Victor Lombardi, Why We Fail

Topics

Organizational Research

Stakeholders

Stakeholders

Executives

Sales People

Customer Service

Editors

Production Team

Organizational research helps you with:

Requirements

Politics

Workflow

Capabilities

Goodwill

Requirements

What are the top business priorities for this project/product?

Politics

What does success mean to the individual stakeholders?

Workflow

Do we have to change how people are working together to be successful?

Workflow

How do we have to change how people are working together to be successful?

Workflow

How can we possibly change how people are working together?

Capabilities

What are the strengths and weaknesses of our team?

Capabilities

Where is the internal expertise?

Goodwill

How can this project make your job easier (or harder)?

Get them alone

Basic Stakeholder Questions

What is your title? How long have you been in this role?

What are your essential duties and responsibilities?

What does a typical day look like?

Who are the people you work most closely with? How is that going?

What does success mean from your perspective, what will have changed for the better once this project is complete?

Do you have any concerns about this project?

What do you think the greatest challenges to success are? Internal and external?

For each stakeholder, note the following:

What’s their general attitude toward this project?

What’s the goal as they describe it?

To what extent are this person’s incentives aligned with the project’s success?

How much and what type of influence do they have?

Who else do they communicate with on a regular basis?

To what extent does this stakeholder need to participate throughout the project, and in which role?

Is what you heard in harmony or in conflict with what you’ve heard from others throughout the organization?

Stakeholder power moves

“Why are you asking me this?”

“I don’t understand that question. It doesn’t make any sense.”

“I don’t feel comfortable talking to you about that.”

“No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.”

“How much more time is this going to take?”

Practice!

10 minutes practice. Find a partner. Take turns.

What is your title? How long have you had this job?

What are your essential duties and responsibilities?

What is a typical day like?

Who are the people you work most closely with? How is that going?

What do you think the greatest challenges to your success are? Internal and external?

Empathy

Break!

To Review

Team + Goal + Shared Reality = Good

Research is a simple process you can apply to however you work. You shouldn’t be dogmatic.

Even though this sounds obvious, some people will resist this because questions can feel threatening.

Facts will not change the minds of people who are threatened.

You need to appeal to what you know is important to them, and fit your facts into their story.

So, understanding what is important to your stakeholders is necessary for design and research to succeed.

UserResearch

Photo: Flickr/theloushe

Ethnography

How to do bad user research:Ask people what they want.

How to do bad user research:Ask people what they like.

Never ask users what they want or like.

The Four Ds of Design Ethnography

Deep DiveDaily Life

Data AnalysisDrama

“...true ethnography reveals not just what people say they do, but what they actually do.”

–PARC

Photo: Flickr/lintmachine

The Art ofThe Interview

Interviewing is not talking.

Interviewing is listening.

Good Interviewers:

Know Your Question

Warm Up

Shut Up

Interview Structure:

Introduction

Body

Conclusion

Introduction:

Smile

Express gratitude

Describe the process

Ask to record

Warm up questions

Body:

Ask open-ended questions

Probe for more

Allow silence

Use questions as checklist

Conclusion:

Transition to wrap-up

Ask if there is anything else

Thank for time

You are the hostYou are the student

Out of your comfort zone,and into theirs.

Interview ChecklistCreate a welcoming atmosphere to make participants feel at ease.

Always listen more than you speak.

Take responsibility to accurately convey the thoughts and behaviors of the people you are studying.

Start each interview with a general description of the goal, but be careful of focusing responses too narrowly.

Avoid leading questions and closed yes/no questions. Ask follow-up questions.

Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it.

Also note the exact phrases and vocabulary that participants use.

Look forGoals

Priorities

Tasks

Motivators

Barriers

Habits

Relationships

Tools

Environment

Roles

Interviewer

Notetaker

Observer

Practice!

Interview Scenario

You want to develop a new service to help people give gifts.

The goal of the research is to identify unmet needs people might have with regard to giving gifts.

Interview Practice

Break into groups of 3-4 people

1 interviewee, 1 interviewer , 1 notetaker, 1 observer (optional),

Switch in 15 minutes

2 rounds

Look forGoals

Priorities

Tasks

Motivators

Barriers

Habits

Relationships

Tools

Environment

Switch!

Get ready to

finish.

Time’s Up!

How did that go?

How about a focus group?

14

“Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.”

–Robert K. Merton, Sociologist,invented focus groups

Everybody Lies

Competitive Research

How else might your target customer solve

the same problem?

Competitive Review

What do they say they offer?

Who is their customer? How is this the same or different from your target audience or users?

What are the key differentiators—the factors that make them uniquely valuable to their target market, if any?

How do the user needs or wants they’re serving overlap or differ from those that you’re serving or desire to serve?

What do you notice that they’re doing particularly well or badly?

Based on this assessment, where do you see emerging or established conventions in how they do things, opportunities to offer something clearly superior, or good practices you’ll need to adopt or take into consideration to compete with them?

Your target customershave to love youmore than they

hate change.

(Usability) Testing

A good research activity:

•Answers a key question

•Addresses identified assumptions

•Informs specific decisions

•Involves your team

•Fits your level of expertise

•Fits your schedule and budget

•Fundamentally research is a simple process

•There are many activities and definitions

•No pressure!

•Select the methods that inform decisions

•Begin by understanding your organization

•Never ask what people like

•People are lazy, forgetful creatures of habit

•Keep each other honest

•Practice and learn

Research and Collaboration

Working together across disciplines and making decisions based on evidence shouldn’t be hard, but they can be.

Done right, research and working collaboratively reinforce each other through a shared understanding of reality.

Start with your goal in mind, not with any process or buzzword.

Asking questions and cutting across traditional roles can both be threatening to the established order.

Commit to clear communication and critical thinking.

Research questions follow from goals, assumptions, and risk.

Always have a framework and a plan.

Creating Meaning From Data

1. Compile data2. Analyze3. Identify Insights4. Create Model

Basic Analysis

Closely review the notes.

Look for interesting behaviors, emotions, actions, and verbatim quotes.

Write what you observed on a sticky note (coded to the source, the actual user, so you can trace it back).

Group the notes.

Watch the patterns emerge.

Rearrange the notes as you continue to assess the patterns.

Observation

Observation

Observation

Observation

Observation

Observation

Collaborates on purchases

Uses several devices

Needs affirmation

Observation

Observation

Observation

Observation

Observation

Observation

Observation

Ground rules

Acknowledge that the goal of this exercise is to better understand the context and needs of the user. Focus solely on that goal.

Respect the structure of the session. Refrain from identifying larger patterns before you’ve gone through the data.

Clearly differentiate observations from interpretations (what happened versus what it means).

No specific solutions until after you’ve gone through insights and principles. Solutions come next.

Practice!

15 minutes analysis.

Break into groups of 6-8 people

Each group work together to fill out one diagram with the strongest patterns.

Negotiate and advocate for your perspective.

Look for

Goals

Priorities

Tasks

Motivators

Barriers

Habits

Relationships

Tools

Environment

15 minutes analysis.

Break into groups of 3-4 people

Each group work together to fill out one diagram with the strongest patterns.

Negotiate and advocate for your perspective.

Get ready to

finish.

Time’s Up!

How did that go?

Models

Reporting

You are collaborating withyour future selves.

Research ReportStudy Title

Date Completed

Research Goal

Activities

Related Decisions

Key Insights

Supporting Observations

Recommended Actions

Questions for Further Study

A useful report supports

Clear goals

Shared values

Access to information

Clear decision-making

You decide if it’s important for the report to be

Informing?

Inspiring?

Focusing?

Remembering?

Recording?

Deciding?

Finale

In summary

Research creates a shared understanding of reality.

Asking questions is uncomfortable. Embrace that feeling.

A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun.

Clear goals and good questions are required.

Choose only the research activities that answer real questions and inform your top priority design and development decisions.

Practice! Observe and listen every day.

Document! Report! Share! It’s easy to lose what you learn.

Any questions?

Additional sources:Designing Together by Dan A. Brownhttp://www.designingtogetherbook.com/

LeanUX by Jeff Gothelfhttp://www.leanuxbook.com/

Remote Research by Nate Bolt & Tony Tony Tulathimuttehttp://rosenfeldmedia.com/books/remote-research/

Interviewing Users by Steve Portigalhttp://rosenfeldmedia.com/books/interviewing-users/

Google Ventures Library | Designhttp://www.gv.com/library/design/

Pacific Standard Magazine http://www.psmag.com/

Helsinki Design Lab (closed, but excellent publications still available)http://www.helsinkidesignlab.org/pages/publications

Brief books for people who make websites No.

9

JUST ENOUGHRESEARCH

Erika Hall

You might enjoy the book.

www.abookapart.comFor 15% off, use code: HALL-THE14(Good through November 2014)