ux london collaborative research workshop

UX LONDON

Research Together!

Hello!

I have a question…

Do you enjoy being right?

You are correct!

YESSS!

p0wned!

Flickr/Chris Voll

Where are you?

What We’re Doing Today Research & Collaboration

A Framework for Research Questions & Activities Biases & Objections

Understanding the Organization

User Research Analysis

Models and Reports Getting Buy-In at Every Stage

Wrap Up

Research & Collaboration

People

Collaboration!

True Collaboration

Chris Noessel

Collaboration requires behavior change

For Effective Collaboration

Have a plan Provide a rationale Define roles and responsibility Set expectations Communicate progress Reflect on performance

The 4 Virtues of Collaboration

Clarity & definition Accountability & ownership Awareness & respect Openness & honesty

Embrace Conflict

Why research?

Real World

Context

Organization Users

Things you

don’t control

Goals, assumptions &

resources

Needs & behaviors

Personal View

Shared Reality

Shared Understanding

A design project is a series of decisions.

What is

What ought to be

Research leads to evidence-based

decisions.

Data (alone) doesn’t change minds.

Objections!

6We don’t have the time.

$We don’t have the money.

$We don’t have the expertise.

We have a guy.

April May June July

We’ll just A/B test.

We have analytics.

!Let’s just make a prototype.

Steve Jobs

Any others?

Everyone wants better products, faster.

No one wants to read a report.

Research Advantages

¿Faster decisions

CLower risk

Saves money$58

JIncreased value

Continuing returnsC60

Everyone is more effective and

efficient

How to Do Research

Design-Led

Research-Led

Expert Mindset

Participatory Mindset

Users seen as subjects Users seen as partners

Design-led with

expert mindset

Design-led with

participatory mindset

Research-led with expert

mindset

Research-led with participatory

mindset

Dubberly Design Office

Goal Driven

Skeptical Mindset

Increase chance of success Reduce risk

Willing to question the value of any approach

One Simple Process

Form Questions Analyze Data

Gather Data

Form Questions Analyze DataThink Critically

Form Questions Analyze DataObserve

Form Questions Analyze DataInterview

Form Questions Analyze DataRead

Experiment

Interview

Observe

Your Process Success!!!!Insights

You need a plan!

Gather Data

Questions determine results.

Questions give research meaning.

Research high-priority questions.

Good Questions

Specific Actionable Practical

A Bad Question

“How do we get Millennials to like us?”

Better Question

“How do recent college graduates living in cities decide what to have for dinner?”

A Bad Question

“What do people do around here all day?”

A Better Question

“How do editors and designers work together?”

The Best Question

The unknown that carries the most risk.

Project Risks

•Target customers don’t value product

•Customers need something else

•Business model doesn’t support it

•Organization can’t produce it

•Someone else is doing it better

Practice!!88

Practice Exercise: Writing questions Time:10 minutes

• I will give you a project scenario • Discuss potential research questions • Write down the 3–5 highest priority

What research questions might serve this project?

A e-commerce startup wants to create an app to

help people give gifts.

Time is up!691

Let’s talk about your questions.

How to answer those questions

Gather Data

Research activities are simply ways to answer questions.

Questions About

ProductOrg

Competition

InterviewsInterviews

Usability Testing

A/B Testing

Contextual Inquiry

Literature Review

SWOT Analysis

Brand Audit

Usability Testing

Competitive Analysis

Heuristic Analysis

Descriptive

Evaluative

Analytic

Generative

Descriptive

Research Activity

Topic Purpose

Phone Interviews

What do we need to know

about?What kind of

decision will it inform?

How long do we have?

What is our budget?

In-Person Interviews Contextual

Inquiry

Usability Testing

Competitive Analysis

Why not just make a prototype?

“If we only test bottle openers, we may never realize customers prefer screw-top bottles.”

– Victor Lombardi, Why We Fail

There is no one right method or activity.

How to get the most out of any method •Clarify goals •Enumerate assumptions •Identify questions •Prioritize questions •Work collaboratively •Present strategically

Don’t focus on the method

Tell the story

Critical Thinking

•Disciplined •Self-correcting •Clear •Logical

Uncritical Thinking

“I hate yellow, so a yellow website won’t succeed.”

Critical Thinking

“I hate yellow, but based on the evidence, it might work for our audience.”

Uncritical Thinking

“The information on this website is too dumbed-down for me.”

Critical Thinking

“Our target audience needs clear, simple information.”

Critical Thinking

“I don’t know.”

Bias: Something that causes an influence or

prejudice

Confirmation Bias: You selectively weight the information that confirms what you already believe.

Sampling Bias: Your sample of research subjects isn’t sufficiently

representative.

Interviewer Bias: You insert your opinion

into interviews.

Social Desirability Bias People don’t say the true things that they worry will

make them look bad.

Hawthorne Effect Observation changes the behavior being observed.

Clear Display

Related Experience

Primed Idea

Good Mood

Feels True

Feels Familiar

Feels Good

Feels Effortless

Daniel Kahneman

Feeling confident? It’s not a good sign.

You might have a bad case of Dunning-Kruger.

Research Topics

Organizational Research

Real World

Context

Organization Users

All organizations have baggage

Organizational research helps you understand

•Requirements •Politics •Workflow •Capabilities •Goodwill

Requirements

What are the top business priorities for this project/product?

Politics

What does success mean for our individual roles?

Workflow

(How) do we have to change how we work together to be successful?

Capabilities

What are the strengths and weaknesses of our team?

Capabilities

Where is the internal expertise?

Goodwill

How might product decisions make someone’s job easier (or harder)?

Stakeholders

How to extract knowledge from these people?

Get them alone.

Butter them up.

Basic Stakeholder Questions What is your title? How long have you been in this role?

What are your essential duties and responsibilities?

What does a typical day look like?

Who are the people you work most closely with?

How is that going?

Project Specific Questions What does success mean from your perspective?

What will have changed for the better once this project is complete?

Do you have any concerns about this project?

What do you think the greatest challenges to success are?

Internal and external?

Stakeholder Power Moves

“Why are you asking me this?”

“I don’t understand that question. It doesn’t make any sense.”

“I don’t feel comfortable talking to you about that.”

“No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.”

“How much more time is this going to take?”

Practice!!141

Practice Exercise: Stakeholder interviews Time:10 minutes x 2

• Find a partner. This is a pair exercise. • We will show you a set of questions • Interview your partner • Really listen • No need to take notes

Stakeholder Questions

•What is your title? How long have you had this job?

•What are your essential duties and responsibilities?

•What is a typical day like?

•Who are the people you work most closely with?

•How is that going?

•What do you think the greatest challenges to your success are?

•Internal and external?

Switch!∞144

Stakeholder Questions

•What is your title? How long have you had this job?

•What are your essential duties and responsibilities?

•What is a typical day like?

•Who are the people you work most closely with?

•How is that going?

•What do you think the greatest challenges to your success are?

•Internal and external?

Time is up!6146

What did you learn?

Empathy

In summary

• A shared reality is as important as “new” facts. • Research is a simple process. • Be goal-oriented, not dogmatic. • Questions threaten authority. • Facts alone won’t convince those who feel threatened. • Use what you learn to fit your findings into the project story.

User Research

Real World

Context

UsersOrganization

Photo: Flickr/theloushe

Ethnography

How to do bad user research: Ask people what they want.

How to do bad user research: Ask people what they like.

Never ask users what they want or like.

The Four Ds of Design

Ethnography

Deep Dive Daily Life

Data Analysis Drama

“...true ethnography reveals not just what people say they do, but what they actually do.”

–PARC

Photo: Flickr/lintmachine

The Art of The Interview

Interviewing is not talking.

Interviewing is listening.

Good Interviewers 1.Know Your Question 2.Warm Up 3.Shut Up

Interview Structure

1 Intro

2 Body

3 Conclusion

Introduction:

Express gratitude

Describe the process

Ask to record

Warm up questions

Ask open-ended questions

Probe for more

Allow silence

Use questions as checklist

Conclusion:

Transition to wrap-up

Ask if there is anything else

Thank for time

Interview Structure

Introduction:

Express gratitude

Ask to record

Warm up questions

Probe for more

Allow silence

Conclusion:

Thank for time

Interview Structure

Introduction:

Express gratitude

Ask to record

Warm up questions

Probe for more

Allow silence

Conclusion:

Thank for time

Interview Structure

You are the host You are the student

Out of your comfort zone,

and into theirs.

Interview Checklist • Create a welcoming atmosphere.

• Always listen more than you speak.

• Take responsibility to accurately convey the thoughts and behaviors of the people you are studying.

• Start each interview with a general description of the goal, but be careful of focusing responses too narrowly.

• Avoid leading questions and closed yes/no questions. Ask follow-up questions.

• Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it.

• Also note the exact phrases and vocabulary.

Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment

Make a note of

RolesInterviewer Notetaker Observer Participant

Practice!!174

Exercise: User research interview Time: 15 min x 3

•We will give you a research scenario and an interview script

•In your groups, assign roles of participant, interviewer, note taker, and observer (optional)

•Conduct the interview in 15 min, then switch roles. •Hang on to your notes. We’ll need them.

Interview Scenario

You are working on a new service to help people give gifts.

The goal of the research is to identify unmet needs people might have with regard to giving gifts.

Listen for:

!Ready? Go!

Switch!∞179

Remember: This is a research interview, not a friendly conversation.

Listen for:

Switch!∞181

Clear your mind.

Listen for:

Time is up!6183

How did that go?

How about a focus group?

“Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.” –Robert K. Merton, Sociologist, invented focus groups

Everybody Lies

Anything else about activities?

Analysis

Analysis turns data into useful insights

Analysis isn’t complicated

Compile data Look for patterns

Identify insights

Create models

…but it is challenging

You need to create meaning from data

Data > Meaning > Actionable Inputs

Observation Observation

Observation

Observations are

Verbatim quotes Reported behaviors

Observed behavior

Observation

Collaborates on purchases

Uses several devices Needs affirmation

Make product information sharable

Save state in purchase process

Users collaborate on purchases

Insight Mandate/Action

Ground rules The goal of this exercise is to better understand the context and needs of the user.

Wait to identify larger patterns until you’ve gone through the data.

Clearly differentiate observations from interpretations (what happened versus what it means).

No specific solutions until after you’ve gone through insights and principles. Solutions come next.

Practice!!201

Practice Exercise: Analysis Part 1 Time: 15 minutes.

1.Gather all notes 2.Scan notes for interesting observations 3.Write observations on notes 4.Put the notes in a pile

Look for quotes and observations that indicate

Goals (what the

participant wants to

accomplish)

Priorities (what is most important to

the participant)

Tasks (actions the participant

takes to meet their goal)

Barriers (what prevents accomplishing

the goal)

Motivators (the situation or

event that starts down the

task path)

Habits (things the participant does on a

regular basis)

Relationships (who the

participant interacts with

doing the tasks)

You have 15 minutes.6

Get ready to finish!6

Time is up!6206

Practice Exercise: Analysis Part 2 Time: 15 minutes.

1.Put notes on the board 2.Group notes into patterns 3.Label the group 4.Negotiate and advocate for your perspective

Do this part last

You have 15 minutes.6

Get ready to finish!6

Time is up!6213

How did that go?

Creating a Model

Personal View

Shared Reality

Thinking is not useful until shared

A model is thinking made visible

Some examples

In summary •A model simplifies and clarifies complex ideas.

•Thinking isn’t useful until it’s out where people can see it.

• An effective model makes it easy to incorporate new information.

•A model diagram is a tool, not an end in itself.

•Avoid the temptation to make diagrams more pretty than useful.

•An effective model communicates without needing an audio guide.

Personas

A persona is “the user” made visible

Busy mom is the ur-persona.

I’ve never seen a persona called “Married woman, no kids, with pristine hardwood.” God, how I aspire to see that persona.

-Steve Portigal

They can still be useful

Personas

• Distill ethnographic research • Document representative groups of needs and

behaviors as archetypes • Represent relationships among user types • Allow team to advocate for user needs • Act as a reference point for decision-making • Maintain empathy throughout design process

Personas must be based on actual research data

Fun, but not useful

A good model represents and simplifies knowledge

Practice!!241

MAKE A SIMPLE JOURNEY

Experience Journey Map | Task: Eating Lunch

Practice Exercise: User Journey Time: 15 minutes.

1.As a group, review your user interview notes to identify steps in gift purchase journey

2.Decide whether that step is positive or negative 3.Decide how much of a positive or negative 4.Fill in diagram

Time is up!6244

How did that go?

Reporting

2A useful report supports

Clear goals Shared values Access to information Confident decisions

2You decide the purpose

Informing? Inspiring? Focusing? Remembering? Recording? Deciding?

Research ReportStudy Title

Date Completed

Research Goal

Activities

Related Decisions

Key Insights

Supporting Observations

Recommended Actions

Questions for Further Study

Keep it as brief as you can.

Building a research-driven

culture250

Capture the value of research

Use your report to tell a story

How most people do itMethods

(what we did, usually in vast

detail)

Findings (what we found,

often disconnected

from biz goals)

Meaning (the

implications for our design

Framing (How it

connects to the project story…

maybe)

How you shouldMethods (a brief

summary up front, most as an appendix)

Findings (what we found, leading with the interesting bits)

Meaning (the

implications for your business)

Framing (Setting the stage with

context and a good story)

Finish line!

In summary • Research creates a shared understanding of reality.

• Asking questions is uncomfortable. Embrace that feeling.

• A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun.

• Clear goals and good questions are required.

• Choose only the research activities that answer real questions and inform your top priority design and development decisions.

• Practice! Observe and listen every day.

• Document! Report! Share! It’s easy to lose what you learn.

Any questions?

Thank you!

ux london collaborative research workshop

Design

playdate! app - ux di london 2014 - week1 project

collaborative sketching for ux - nyu 02/19/14

designing addicted products @ ux london 13

design research for everyday projects - ux london

collaborative brainstorming for better ux workshop

html5 ux london

ux camp london - ux in the enterprise

ux is team work, front-end london, august 2015

ux london 2013 - notes and key themes

ux principles at marketing week live london 2014

reiss on e service ux london

design patterns for fantabulous collaborations [ux london,...

getting ux done (for interact london)

ux for lean startups london

collaborative ux: pairing user & business goals

taxonomy for app makers workshop – ux london

interact london 2015: vanessa kirby - the fables of ux

open for open questions - ux london 2014

beyond measure | ux london

gamestorming ux london