Qualitative Research Methods in User Experience Design

UX Tokyo Talk February 2014

What is Qualitative Research? The data is not numeric – Photos – Videos – Direct observation – Comments on message boards – Recordings of phone transactions –  Interview transcripts

What is Qualitative Research? Although this data can be converted into

quantitative data, doing so is not always useful or appropriate. – Data of this type is usually very difficult to collect

consistently across a representative sample – Meaningful valuable outcomes can be achieved

without using numbers –  This is especially good at providing you insights into

things you didn’t already know

What is Qualitative Research? Analysis, interpretation, and presentation often are done

using very different techniques from quantitative analysis.

Examples: –  personas - A "typical" user presented in a way that a

development team can focus on –  process maps - An understanding of the actual steps in a

process

What is Qualitative Research? Some (mostly) qualitative research traditions from

the social sciences: –  Ethnography –  Biography –  Phenomenology – Grounded Theory – Case Study –  Ethno-methodology

Source:  Qualita've  Inquiry  and  Research  Design:  Choosing  Among    Five  Tradi'ons  by  John  Creswell  

Why do Qualitative Research? It is sometimes the best way to address research needs:

–  Discovering issues that you don't know exist –  Sorting out messy information ( ie the critical knowledge only

exists in the heads of a privileged few) –  To complement quantitative research.  –  Some projects do not allow for easy collection of quantitative

data (ie a B2B with a limited number of customers, hardware development, or “sensitive” markets)

How to Start Research should be designed carefully.

You need to consider: –  Budget –  Goals –  Available resources –  Schedule –  Etc...

How to Start Good research starts with questions Examples:

–  How can we improve app X? •  What pain-points do current users have? •  What else would users like to do with the app?

–  Which of these 5 innovation projects should we productize? •  What are the potential market, competitors, technical risks,

etc...? •  Which ones excite people?

How to Start Know why your doing it Often, you just want to bring back something for the

design team to make brain-storming more productive.

But there can be other strategies

How much data to collect How many visits to end users’ homes/offices is

enough? It depends… –  Until you seen so much that you aren’t learning anymore –  Until you run out of time or budget –  Until you meet the objectives of the study

Some of the Tools Semi-structured interviews

User observation

Photo & video “ethnography”

Semi-structured interviewing Goldilocks interviewing style Structured interviews tend to miss out: –  What you don’t know to ask –  Getting good detail without consuming too much time

Unstructured interviews tend to gather data that is –  Inconsistent –  Off-topic

Semi-structured interviewing Usually requires more planning & refining Create a research protocol

–  An outline of what you want to talk about –  Format it so that you can be flexible while talking –  Pay attention to how you print it and use it in the field

Update it as you research

Like many other skills, it requires talent and practice.

Semi-structured interviewing Recording Where possible, audio-record Get good at note-taking too Working in pairs is also recommended, when possible

Semi-structured interviewing Steering the conversation Because it’s an interview, you are allowed to drive it in the

direction you want to. But you also want to get them talking as openly, freely, &

much as possible.

Always be very aware of the time, but don’t make it too obvious!!!

Semi-structured interviewing Tricks: Use pauses and interjections (相槌) effectively Get them to tell stories Get them to show you things

Observation "What people say, what people do, and what they say

they do are entirely different things." Margaret Mead

Observation Depending on your topic, ability to observe and

video or photograph end users varies greatly

Strategies typically follow these constraints

You should always try to collect as much intelligible data as you can, including what you store in your head

Observation

When you can’t use a camera….

Observation

Note what you can & re-sketch before you forget details

Observation Frameworks Typically useful for novice researchers or when

including engineers/designers in the research process –  AEIOU (Actions, Environments, Interactions, Objects,

Users) –  POEMS (People, Objects, Environments, Message,

Services) –  Who, What, Where, Why, When

What to do with the data Process it while it is fresh

If possible, everyone that participated should sit down immediately following the interviewing/observation, to discuss what was seen.

Copious notes should be taken during this session.

What to do with the data If the goal of the research is deeper

understanding… Deep analysis is needed. Methods vary by researcher. –  Post-It sorts –  Video tagging – Conversation analysis

Various, sometimes lengthy, reports result

What to do with the data If the goal is quick fodder for brain-storming…

Make the data as clear and visual as possible Try to bring the design team into the user’s

shoes