collaborative research | uxlx 2014

Collaborative Research with Erika Hall (@mulegirl) UXLX

Hello!

I have a question...

Do you enjoy being right?

You are correct!

YESSS!

p0wned!

Flickr/Chris Voll

People!

Collaboration!

Text Agenda

Text Agenda 910:30 10:3011 1112:30 Introduction. The research process Questions and activities Understanding your organization Break User research Analysis Models and reports Finale

Barriers

Overcoming Objections

Laziness

Laziness Fear

Laziness Fear Lunch

Laziness Fear Lunch Following

Laziness Fear Lunch Following Losing Control

Laziness Fear Lunch Following Losing Control Sharing

Shared Understanding

We dont have the time.

We dont have the money.

We dont have the expertise.

Were already A/B testing

Everyone wants better products, faster.

No one wants to read a report.

What is your experience?

Research + Collaboration

A design project is a series of decisions.

Data doesnt change minds.

What is What ought to be

Design-Led Research-Led Expert Mindset Participatory Mindset Users seen as subjects Users seen as partners Design-led with expert mindset Design-led with participatory mindset Research-led with expert mindset Research-led with participatory mindset Dubberly Design Office

Goal Driven Skeptical Mindset Increase chance of success Reduce risk Willing to question the value of any approach

Team + Goal + Reality = Good

One Simple Process

Form Questions Analyze Data Gather Data

Form Questions Analyze Data Think Critically

Form Questions Analyze Data Observe

Form Questions Analyze Data Interview

Form Questions Analyze Data Read

Form Questions Analyze Data Read Experiment Interview Observe Think

Personal View Personal View Personal View

Shared Reality

Research is a Craft

Questions determine results.

Questions give research meaning.

Research high-priority questions.

Good Questions Specific Actionable Practical

A Bad Question What do people think about pets?

A Better Question How do single urban adults choose and acquire a pet?

A Bad Question What do people do around here all day?

A Better Question How do editors and designers work together?

The Best Question The unknown with the most risk.

Bias: Something that causes an influence or prejudice

Confirmation Bias: You selectively weight the information that confirms what you already believe.

Sampling Bias: Your sample of research subjects isnt sufficiently representative.

Interviewer Bias: You insert your opinion into interviews.

Social Desirability Bias People dont say the true things that they worry will make them look bad.

Ease Clear Display Related Primed Idea Good Mood Feels True Feels Familiar Feels Good Feels Effortless Daniel Kahneman

Feeling confident? Its not a good sign.

You might have a bad case of Dunning-Kruger.

Critical Thinking

Critical Thinking Disciplined Self-correcting Clear Logical

Uncritical Thinking I hate yellow, so a yellow website wont succeed.

Critical Thinking I hate yellow, but based on the evidence, it might work for our audience.

Critical Thinking I dont know.

Activities!

Questions About Users ProductOrg Competition InterviewsInterviews Usability Testing A/B Testing Contextual Inquiry Literature Review SWOT Analysis Brand Audit Usability Testing Competitive Analysis Heuristic Analysis Descriptive Evaluative Evaluative Evaluative Analytic Analytic Generative Descriptive

Research Activity Topic Purpose Time Money

Phone Interviews What do we need to know about? What kind of decision will it inform? How long do we have? What is our budget? Contextual Inquiry In-Person Interviews Usability Testing Competitive Analysis

Why not just make a prototype?

20 If we only test bottle openers, we may never realize customers prefer screw-top bottles. Victor Lombardi, Why We Fail

Topics

Organizational Research

Stakeholders

Executives

Sales People

Customer Service

Editors

Production Team

Organizational research helps you with: Requirements Politics Workflow Capabilities Goodwill

Requirements What are the top business priorities for this project/ product?

Politics What does success mean to the individual stakeholders?

Workflow Do we have to change how people are working together to be successful?

Workflow How do we have to change how people are working together to be successful?

Workflow How can we possibly change how people are working together?

Capabilities What are the strengths and weaknesses of our team?

Capabilities Where is the internal expertise?

Goodwill How can this project make your job easier (or harder)?

Get them alone

Basic Stakeholder Questions What is your title? How long have you been in this role? What are your essential duties and responsibilities? What does a typical day look like? Who are the people you work most closely with? How is that going? What does success mean from your perspective, what will have changed for the better once this project is complete? Do you have any concerns about this project? What do you think the greatest challenges to success are? Internal and external?

For each stakeholder, note the following: Whats their general attitude toward this project? Whats the goal as they describe it? To what extent are this persons incentives aligned with the projects success? How much and what type of influence do they have? Who else do they communicate with on a regular basis? To what extent does this stakeholder need to participate throughout the project, and in which role? Is what you heard in harmony or in conflict with what youve heard from others throughout the organization?

Stakeholder power moves Why are you asking me this? I dont understand that question. It doesnt make any sense. I dont feel comfortable talking to you about that. No one pays attention to anything I have to say, so I dont know why I should bother talking to you. How much more time is this going to take?

Practice!

10 minutes practice. Find a partner. Take turns. What is your title? How long have you had this job? What are your essential duties and responsibilities? What is a typical day like? Who are the people you work most closely with? How is that going? What do you think the greatest challenges to your success are? Internal and external?

Empathy

Break!

To Review Team + Goal + Shared Reality = Good Research is a simple process you can apply to however you work. You shouldnt be dogmatic. Even though this sounds obvious, some people will resist this because questions can feel threatening. Facts will not change the minds of people who are threatened. You need to appeal to what you know is important to them, and fit your facts into their story. So, understanding what is important to your stakeholders is necessary for design and research to succeed.

User Research

Photo: Flickr/theloushe Ethnography

How to do bad user research: Ask people what they want.

How to do bad user research: Ask people what they like.

Never ask users what they want or like.

The Four Ds of Design Ethnography

Deep Dive Daily Life Data Analysis Drama

...true ethnography reveals not just what people say they do, but what they actually do. PARC

Photo: Flickr/lintmachine

The Art of The Interview

Interviewing is not talking.

Interviewing is listening.

Good Interviewers: Know Your Question Warm Up Shut Up

Interview Structure: Introduction Body Conclusion

Introduction: Smile Express gratitude Describe the process Ask to record Warm up questions

Body: Ask open-ended questions Probe for more Allow silence Use questions as checklist

Conclusion: Transition to wrap-up Ask if there is anything else Thank for time

You are the host You are the student

Out of your comfort zone, and into theirs.

Interview Checklist Create a welcoming atmosphere to make participants feel at ease. Always listen more than you speak. Take responsibility to accurately convey the thoughts and behaviors of the people you are studying. Start each interview with a general description of the goal, but be careful of focusing responses too narrowly. Avoid leading questions and closed yes/no questions. Ask follow-up questions. Prepare an outline of your interview questions in advance, but dont be afraid to stray from it. Also note the exact phrases and vocabulary that participants use.

Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment

Roles Interviewer Notetaker Observer

Practice!

Interview Scenario You work for an e-commerce site that wants to develop a new service to help people give gifts. The goal of the research is to identify unmet needs people might have with regard to giving gifts.

Interview Practice Break into groups of 3-4 people 1 interviewee, 1 interviewer , 1 notetaker, 1 observer (optional), Switch in 15 minutes 2 rounds

How did that go?

How about a focus group?

14 Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched. Robert K. Merton, Sociologist, invented focus groups

Everybody Lies

Competitive Research

How else might your target customer solve the same problem?

Competitive Review What do they say they offer? Who is their customer? How is this the same or different from your target audience or users? What are the key differentiatorsthe factors that make them uniquely valuable to their target market, if any? How do the user needs or wants theyre serving overlap or differ from those that youre serving or desire to serve? What do you notice that theyre doing particularly well or badly? Based on this assessment, where do you see emerging or established conventions in how they do things, opportunities to offer something clearly superior, or good practices youll need to adopt or take into consideration to compete with them?

Your target customers have to love you more than they hate change.

(Usability) Testing

A good research activity: Answers a key question Addresses identified assumptions Informs specific decisions Involves your team Fits your level of expertise Fits your schedule and budget

Fundamentally research is a simple process There are many activities and definitions No pressure! Select the methods that inform decisions Begin by understanding your organization Never ask what people like People are lazy, forgetful creatures of habit Keep each other honest Practice and learn

Research and Collaboration Working together across disciplines and making decisions based on evidence shouldnt be hard, but they can be. Done right, research and working collaboratively reinforce each other through a shared understanding of reality. Start with your goal in mind, not with any process or buzzword. Asking questions and cutting across traditional roles can both be threatening to the established order. Commit to clear communication and critical thinking. Research questions follow from goals, assumptions, and risk. Always have a framework and a plan.

Creating Meaning From Data

1. Compile data 2. Analyze 3. Identify Insights 4. Create Model

Basic Analysis Closely review the notes. Look for interesting behaviors, emotions, actions, and verbatim quotes. Write what you observed on a sticky note (coded to the source, the actual user, so you can trace it back). Group the notes. Watch the patterns emerge. Rearrange the notes as you continue to assess the patterns.

Observation Observation Observation Observation Observation Observation

Observation Observation Observation Observation Observation Observation Observation

Collaborates on purchases Observation Observation Observation Observation Observation Observation Observation

Collaborates on purchases Uses several devices Observation Observation Observation Observation Observation Observation Observation

Collaborates on purchases Uses several devices Needs affirmation Observation Observation Observation Observation Observation Observation Observation

Ground rules Acknowledge that the goal of this exercise is to better understand the context and needs of the user. Focus solely on that goal. Respect the structure of the session. Refrain from identifying larger patterns before youve gone through the data. Clearly differentiate observations from interpretations (what happened versus what it means). No specific solutions until after youve gone through insights and principles. Solutions come next.

Practice!

20 minutes analysis. Break into groups of 6-8 people Each group work together to fill out one diagram with the strongest patterns. Negotiate and advocate for your perspective.

How did that go?

Models

Reporting

You are collaborating with your future selves.

Research ReportStudy Title Date Completed Research Goal Activities Related Decisions Key Insights Supporting Observations Recommended Actions

A useful report supports Clear goals Shared values Access to information Clear decision-making

You decide if its important for the report to be Informing? Inspiring? Focusing? Remembering? Recording? Deciding?

Finale

In summary Research creates a shared understanding of reality. Asking questions is uncomfortable. Embrace that feeling. A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun. Clear goals and good questions are required. Choose only the research activities that answer real questions and inform your top priority design and development decisions. Practice! Observe and listen every day. Document! Report! Share! Its easy to lose what you learn.

Any questions?

Additional sources: Designing Together by Dan A. Brown http://www.designingtogetherbook.com/ LeanUX by Jeff Gothelf http://www.leanuxbook.com/ Remote Research by Nate Bolt & Tony Tony Tulathimutte http://rosenfeldmedia.com/books/remote-research/ Interviewing Users by Steve Portigal http://rosenfeldmedia.com/books/interviewing-users/ Google Ventures Library | Design http://www.gv.com/library/design/ Pacific Standard Magazine http://www.psmag.com/ Helsinki Design Lab (closed, but excellent publications still available) http://www.helsinkidesignlab.org/pages/publications

Brief books for people who make websites No. 9 JUSTENOUGH RESEARCH Erika Hall You might enjoy the book. www.abookapart.com For 15% off, use code: UXLXJER14