agile australia 2014 | ux: how to measure more than a gut feel by amir ansari
Post on 16-Sep-2014
231 views
DESCRIPTION
How does UX design and research fit into Agile development so you can ensure designs are heading in the right direction? How do you measure the success of your lean designs? How often do you test? How do the numbers direct or shape the design and development process? And finally, how do the numbers and measurements impact business outcomes for clients I’ve worked for? Discover: -The benefits of measuring through UX activities - Approaches for incorporating UX measurements and learning outcomes into Agile projects -Tools and techniques to help provide frequent and repeatable UX insights and applicability within Agile projects -How clients have used the findings and measurements to drive their business goals and positively impact organsisation outcomesTRANSCRIPT
@amir_ansari @dius_au1
@amir_ansari @dius_au2
6’3”
@amir_ansari @dius_au3
@amir_ansari @dius_au4
@amir_ansari @dius_au5
product and technology strategy
application-specific devices
web applications
mobile applications
@amir_ansari @dius_au6
What is UX?
@amir_ansari @dius_au7
Study and Design of ALL aspects of a person’s EXPERIENCE with a product or service
@amir_ansari @dius_au8
@amir_ansari @dius_au9
http://www.uxisnotui.com/
@amir_ansari @dius_au10
Pain points
Usability
Likelihood to recommend
Customer loyalty
Current behaviours Right direction
Alternate designs
Mental model
Why measure?
@amir_ansari @dius_au11 www.theimprovegroup.com
@amir_ansari @dius_au12
@amir_ansari @dius_au13
What?What? Why?Why?
@amir_ansari @dius_au14
@amir_ansari @dius_au15
@amir_ansari @dius_au16 http://lyndabuitrago.com/tag/leaky-gut/
@amir_ansari @dius_au17
• Picture of me measuring UX
@amir_ansari @dius_au18
@amir_ansari @dius_au19
@amir_ansari @dius_au20www.r-66.info
Completion rate
Task time
Task level satisfactionConversions
Page viewsSystem Usability Scale (SUS)
Net Promoter Score (NPS)
@amir_ansari @dius_au21
Why?Why?
@amir_ansari @dius_au22http://dralexkent.files.wordpress.com/2011/12/gut.jpg
We feel we know…
• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea
@amir_ansari @dius_au23
How do we measure and
validate these? And When?
@amir_ansari @dius_au24
Visioning Iterations Iterations Iterations Iterations MVPLaunch
Inception & Planning
1 2 3
@amir_ansari @dius_au25
Technique 1
@amir_ansari @dius_au26
The sooner we can find which features are worth investing in, the sooner we can focus our limited resources on the best solution to our business problems.
Jeff Gothelf – LEAN UX
“
@amir_ansari @dius_au27
@amir_ansari @dius_au28
Visioning Iterations Iterations Iterations Iterations MVPLaunch
Inception & Planning
1 2 3
@amir_ansari @dius_au29
30 – 60 mins 3 One on one interviews
@amir_ansari @dius_au30
• Concerns• Fears• Pain points• Ideal experience• Ideal channel
@amir_ansari @dius_au31
• Stakeholders observing• Call centre Staff felt engaged
and included • More insights than we had
anticipated
@amir_ansari @dius_au32
@amir_ansari @dius_au33
@amir_ansari @dius_au34
@amir_ansari @dius_au35
Visioning Inception & Planning
1 2 3
Iterations Iterations Iterations Iterations MVPLaunch
@amir_ansari @dius_au36
Focus group 3
2 hours
Focus group 1
Focus group 224
@amir_ansari @dius_au37
@amir_ansari @dius_au38
Feature 1 Feature 2 Feature 3
@amir_ansari @dius_au39
• Stakeholders buy-in through observation• Validate assumptions• Client was anonymous
@amir_ansari @dius_au40
• 2 hours wasn’t quite enough• Running sessions back to back
in the same evening!
@amir_ansari @dius_au41
@amir_ansari @dius_au42
Visioning Inception & Planning
1 2 3
Iterations Iterations Iterations Iterations MVPLaunch
@amir_ansari @dius_au43
1 hour One on one testing
6
@amir_ansari @dius_au44
http://www.nngroup.com/articles/how-many-test-users/
@amir_ansari @dius_au45
@amir_ansari @dius_au46
Lean Stack framework by Spark 59(www.spark59.com/leanstack)
@amir_ansari @dius_au47
1. Background
2. Falsifiable Hypotheses
3. Details
6. Next Actions
5. Validated Learning
4. Results
User 1 User 2 User 3 etc.
Hypothesis 1
Hypothesis 2
Etc.
• What feature or assumption to test• What questions to ask• How well does X perform• Etc. • What to test next
• What hypothesis do we want to validate
1. That majority of users think X2. That majority of users perform Y 3. That majority of users need Z4. Etc.
• Previous experiment taught us …• We want to find out…• We want to test our solutions…
1. We can proceed with X2. We need to tweak Y3. We need to pivot and get rid of Z
Title: Experiment 2: XXX Author: Created:
@amir_ansari @dius_au48
1. Background
2. Falsifiable Hypotheses
3. Details
6. Next Actions
5. Validated Learning
4. Results
User 1 User 2 User 3 etc.
Hypothesis 1
Hypothesis 2
Etc.
• What feature or assumption to test• What questions to ask• How well does X perform• Etc. • What to test next
• What hypothesis do we want to validate
1. That majority of users think X2. That majority of users perform Y 3. That majority of users need Z4. Etc.
• Previous experiment taught us …• We want to find out…• We want to test our solutions…
1. We can proceed with X2. We need to tweak Y3. We need to pivot and get rid of Z
Title: Experiment 2: XXX Author: Created:
• From focus groups, it was clear that one barrier with other online providers has been the lack of trainer support
• That providing access to trainers in the classroom and telling students upfront about trainer availability will eliminate barriers and increase conversion
• Show users the marketing pages that talk about trainer availability
• Show users the classroom with the trainer being online
• Users indeed look for trainer availability and are comforted by knowing OTI provides access to them
• Test something else! ; )
@amir_ansari @dius_au49
bit.ly/rapid-debrief
@amir_ansari @dius_au50
@amir_ansari @dius_au51
@amir_ansari @dius_au52
@amir_ansari @dius_au53
@amir_ansari @dius_au54
• Speed of testing• The feedback loop• Stakeholder buy-in
(live stream)• Rapid de-brief• One designs and one tests
@amir_ansari @dius_au55
@amir_ansari @dius_au56
@amir_ansari @dius_au57
@amir_ansari @dius_au58
5
One on one testingFocus groups
= $800
2 FTEs
X $160
X $375*
* Daily gross equivalent based on a gross annual salary of $90k
= $750
$1,550$1,550
16 = $3,200
3 FTEs
X $200
X $375*= $1,125
$4,325$4,325
If doing this internally…
@amir_ansari @dius_au59
@amir_ansari @dius_au60http://stamfordinteractive.com.au/conversations/benefits-of-guerrilla-testing/
@amir_ansari @dius_au61
You can quantify qualitative data!
bit.ly/quantifying-qualitative bit.ly/t-distribution
@amir_ansari @dius_au62
Completion rateCompletion rate
Task timeTask time
Task level satisfactionTask level satisfaction
ConversionsConversions
Page viewsPage views
System Usability Scale (SUS)System Usability Scale (SUS)
Net Promoter Score (NPS)Net Promoter Score (NPS)
{Visioning Inception & Planning Iterations Iterations Iterations Iterations MVP
Launch
@amir_ansari @dius_au63
@amir_ansari @dius_au64
Define why, what, when who, where how, how often
1
@amir_ansari @dius_au65
Become friends with the PM (or the person with the budget)
2
@amir_ansari @dius_au66
Keep it lean and measure often
3
@amir_ansari @dius_au67
Get stakeholder buy-in(Get them to observe)
4
@amir_ansari @dius_au68
Be consistentin how you measure and capture measurements
5
@amir_ansari @dius_au69
Define when to Quant and when to Qual
6
@amir_ansari @dius_au70
https://www.measuringusability.com/topics/UX
@amir_ansari @dius_au71
Thanks to
@amir_ansari @dius_au72http://dralexkent.files.wordpress.com/2011/12/gut.jpg
We feel we know…
• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea
@amir_ansari @dius_au73
Thank you
Qualitative data can in fact be converted into quantitative measures even if it doesn't come from an experiment or from a large sample size.
Jeff Sauro – Measuring Usability“