qual vs. quant: using data to build better products
TRANSCRIPT
Who’s Tired Of Hearing About Data?*
*be honest
A/B Testing
A/B Testing
Funnel Analysis
A/B Testing
Cohorts
Funnel Analysis
A/B Testing
Cohorts
Funnel Analysis
Blah
A/B Testing
Cohorts
Funnel Analysis
Blah
Blah
A/B Testing
Cohorts
Funnel Analysis
Blah
Blah
Blah
Sorry. I’m Going To Talk About Data Some More.
Better Data==
Better Products
Well…Sometimes
What Data Can’t DoReplace Design
Replace Listening to Users
Tell Us What We Should Be Building
What Data Can DoInform Design
Inform Research
Tell Us What We Should Be Investigating
How About An Example?
A Funnel
A Checkout Funnel
Create Account
Select Plan
Payment Info
Address
ConfirmThanks & Up Sell
A Leaky Checkout Funnel
100%
Create Account
Select Plan
Payment Info
Address
ConfirmThanks & Up Sell
-20%
-10%
-50%
-33%
-25%== ???
What That Means to You
100 Buyers
Create Account
Select Plan
Payment Info
Address
ConfirmThanks & Up Sell
-20 Buyers
-8 Buyers
-36 Buyers
-12 Buyers
-6 Buyers== 18 Buyers
100 To 18????How Do We Fix This?
Identifying a Friction Point
100 Buyers
Create Account
Select Plan
Payment Info
Address
ConfirmThanks & Up Sell
-20 Buyers
-8 Buyers
-36 Buyers
-12 Buyers
-6 Buyers== 18 Buyers
Stop It
Quantitative Data Qualitative Data
Understanding the Friction Point
Quantitative Data
WHAT
Qualitative Data
WHY
How Can You Use Them Together?
Step 1: Identify the Biggest Problem
Step 1: Identify the Biggest Problem
Use Funnel Metrics
The Biggest Problem
100 Buyers
Create Account
Select Plan
Payment Info
Address
ConfirmThanks & Up Sell
-20 Buyers
-8 Buyers
-36 Buyers
-12 Buyers
-6 Buyers== 18 Buyers
Step 1: Identify the Biggest Problem
Use Funnel Metrics Quant
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability Qual
Problem Possibilities
Create Account
Select Plan
Payment Info
Address
ConfirmThanks & Up Sell
-36 People• Didn’t have payment info handy• Confused by specific question• Looked for promo code
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability
Step 3: Propose Solutions
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability
Step 3: Propose Solutions
Create Solution Hypotheses
Solution Hypothesis
Problem
People looked for a promo code.
Solution Hypothesis
Possible Solution
Remove promo code entirely.
Solution Hypothesis
What Do We Think Would Happen?
An increase in conversion.
Solution Hypothesis
What Might Go Wrong?
A decrease in acquisition or revenue.
Step 4: Learn & Iterate*
*The last step is always learn & iterate!
Quant & Qual
Learn…
How Would We Know We Were Right?
A/B test of the checkout flow with and without the promo code.
What Really Happened?
Create Account
Select Plan
Payment Info
Address
ConfirmThanks & Up Sell
Why Did It Happen?
Qual!
…And Iterate
Another Possible Solution
Create custom landing pages rather than promo codes.
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability
Step 3: Propose Solutions
Create Solution Hypotheses
Step 4: Learn & Iterate!
Or…
Step 1: Identify the Biggest Problem
Step 1: Identify the Biggest Problem
Interview Current UsersQual
Step 1: Identify the Biggest Problem
Interview Current Users
Step 2: Understand What
Step 1: Identify the Biggest Problem
Interview Current Users
Step 2: Understand What
Study Relevant MetricsQuant
But Why?
Step 1: Identify the Biggest Problem
Interview Current Users
Step 2: Understand What
Study Relevant Metrics
Step 3: Propose Solutions
Create Solution Hypotheses
Step 1: Identify the Biggest Problem
Look at Your Sales Pipeline
Step 2: Understand Why
Interview People Who Said No
Step 3: Propose Solutions
Create Solution Hypotheses
Step 4: Learn & Iterate*
*The last step is always learn & iterate!
Once Again…
Quantitative Data
WHAT
Qualitative Data
WHY
Better Data ==
Better Products