collaborative analysis techniques for user reseaerch
DESCRIPTION
You've done the research. You've gathered data. Piles of it. Now what do you do to keep the experiences of the people you observed in mind? These techniques will help your team agree, buy in, and prioritize all the way to a smart design direction.TRANSCRIPT
Making smart design decisions
Collaborative analysis techniques
Dana ChisnellUX Lx - May 2010
1
2
3
Telling Stories
4
5
6
7
Wiki or Blog
8
Rolling Issues Lists
9
Rolling issues listsObservations in real time
Observer participation = buy-in
Low-fi reporting
10
Observations in real time
Large whiteboard
Colored markers
Don’t worry about order
Be clear enough to remember what was meant
11
Rolling issues
Rolling issuesAngela Coulter
Observer participation
After 1-3 participants, longer break
You start
Invite observers to add and track items
14
Weighting
Number of incidents
Who’s who
15
16
17
Consensus: Observers buy in
observers contribute to identifying issuesyou learn constraintsinstant reporting
Big ideas, so far
18
What obstacles do teams face in
implementing user experience design
practices?
19
20
KJ Analysis
Priorities, democratically
reach consensus from subjective data
similar to affinity diagramming
invented by Jiro Kawakita
objective, quick
8 simple steps
21
1. Focus questionWhat needs to be fixed in Product X to improve the user experience?(observations, data)
What obstacles do teams face in implementing UE practices? (opinion)
22
2. Organize the groupCall together everyone concerned
For user research, only those who observed
Typically takes an hour
23
3. Put opinions or data on notesFor a usability test, ask for observations
(not inferences, not opinion)
No discussion
24
4. Put notes on a wallRandom
Read others’
Add items
No discussion
25
5. Group similar items In another part of the room
Start with 2 items that seem like they belong together
Place them together, one below the other
Move all stickies
Review and move among groups
Every item in a group
No discussion
26
6. Name each groupUse a different color
Each person gives each group a name
Names must be noun clusters
Split groups
Join groups
Everyone must give every group a name
No discussion
27
7. Vote for the most important groupsDemocratic sharing of opinions
Independent of influence or coercion
Each person writes their top 3
Rank the choices from most to least important
Record votes on the group sticky
No discussion
28
8. Rank the groupsPull notes with votes
Order by the number of votes
Read off the groups
Discuss combining groups
Agreement must be unanimous
Add combined groups’ votes together
Stop at 3-5 clear top priorities
29
30
Observation to Direction
31
Observations to direction
Observation
Inference
Opinion
Theory: Direction
32
Observations
33
Participants typed in the top chat area rather than the bottom area.
Observations
Sources:
usability testing
user research
sales feedback
support calls and emails
training
34
What you saw
What you heard
Inferences
35
- Participants are drawn to open areas when they are trying to communicate with other attendees - Participants are drawn to the first open area they see
Inferences
Judgements
Conclusions
Guesses
Intuition
36
Opinions
37
- Participants are invited to click there because it looks clickable - It’s the first open area in the widget
- Participants did not see the typing area below
Opinions
Review the inferences
What are the causes?
How likely is this inference to be the
cause?
How often did the observation happen?
Are there any patterns in what kinds of
users had issues?
38
Direction
39
- Make the response area smaller until it has content
Direction
What’s the evidence for a design change?
What does the strength of the cause
suggest about a solution?
Test theories
40
41
42
DirectionOpinionInferenceObservation
A theory about what to do about it
Why you think it’s happening
Gap between UI and user behavior
What happened
43
Big idea: Making sense of the data
44
45
Big idea: Collaborative analysisshare experience: stories
each person shares their experience with the user
everyone hears/sees rich users stories
easy to visualize people using designs
build consensus in real time: rolling issues
observers contribute to identifying issues
you learn constraints
instant reporting
identify priorities: KJ analysis
quick, democratic
make sense of the data, together
what you heard, saw
gap between the UI and the behavior
What you think is happening
Where to learn more
46
Dana’s blog: http://usabilitytestinghowto.blogspot.com/
Download templates, examples, and links to other resources from www.wiley.com/go/usabilitytesting
