growth models: the science of converting users …...@sjtousley q1 “zoom out” w1 w13 w3 w5 w8 w9...
TRANSCRIPT
with Scott Tousley
Growth Models: The Science of Converting Users into Customers
WORKSHOP
@sjtousley
@sjtousley
@sjtousley
Process > Model > Experiments
GROWTH PROCESS
GROWTH MODEL
EXPERIMENTS
1
2
3
@sjtousley
Q1
Q2Q3
Q4
W1
W13
W3
W5
W8W9
W11
Annual
W12
W10
W7W6
W2W4
W13
W1
W11
W9
W6W5
W3
W2
W4
W7W8
W12W10
W1W2
W3W4
W5W6
W7W8
W9W10
W11W12W13
W13
W12
W11
W10W9
W8W7
W6W5
W4W3 W2 W1
@sjtousley
Q1“zoom out”
W1
W13
W3
W5
W8W9
W11
W12
W10
W7W6
W2W4
W13
W1
W11
W9
W6W5
W3
W2
W4
W7W8
W12W10
W1W2
W3W4
W5W6
W7W8
W9W10
W11W12W13
W13
W12
W11
W10W9
W8W7
W6W5
W4W3 W2 W1
Q2“zoom out”
Q3“zoom out”
Q4“zoom out”
North Star Metric
@sjtousley
W1
W13
W3
W5
W8W9
W11
Weekly Active Team
W12
W10
W7W6
W2W4
W13
W1
W11
W9
W6W5
W3
W2
W4
W7W8
W12W10
W1W2
W3W4
W5W6
W7W8
W9W10
W11W12W13
W13
W12
W11
W10W9
W8W7
W6W5
W4W3 W2 W1
User Activation Rate
Team Activation Rate
Week 12 Retention New Users
Acquired
@sjtousley
W1
W13
W3
W5
W8W9
W11
Weekly Active Team
W12
W10
W7W6
W2W4
W13
W1
W11
W9
W6W5
W3
W2
W4
W7W8
W12W10
W1W2
W3W4
W5W6
W7W8
W9W10
W11W12W13
W13
W12
W11
W10W9
W8W7
W6W5
W4W3 W2 W1
User Activation Rate
Team Activation Rate
Week 12 Retention New Users
Acquired
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
ExperimentExperiment
Experiment
Experiment
Experiment
@sjtousley
W1
W13
W3
W5
W8W9
W11
Weekly Active Team
W12
W10
W7W6
W2W4
W13
W1
W11
W9
W6W5
W3
W2
W4
W7W8
W12W10
W1W2
W3W4
W5W6
W7W8
W9W10
W11W12W13
W13
W12
W11
W10W9
W8W7
W6W5
W4W3 W2 W1
User Activation Rate
Team Activation Rate
Week 12 Retention New Users
Acquired
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
ExperimentExperiment
Experiment
Experiment
Experiment
@sjtousley
AARRR
@sjtousley
Pirate Metrics
Activation
Retention
Referral
Revenue
Acquisition …. people sign up for your product/service
…. people experience “aha moment”
…. people continually come back to you
…. people tell your friends about you
…. people pay you money
@sjtousley
Activation
Retention
Referral
Revenue
Acquisition …. Create Facebook account
…. Add 7 friends in 10 days
…. Sign in once per day
…. Email Facebook invite to a friend
…. Browse news feed, see ads, FB makes $$$
North Star Metric
Case Study: Facebook
@sjtousley
Activation
Retention
Referral
Revenue
Acquisition …. Create Spotify account
…. Listen to 100 songs in 5 days
…. Listen for at least 1 hour per week
…. Share song with your friend
…. Upgrade to Spotify Premium for $10/month
North Star Metric
Case Study: Spotify
@sjtousley
Activation
Retention
Referral
Revenue
Acquisition …. Create free HubSpot CRM account
…. 2 people view 1 contact record
…. 2 people view 1 contact record per week
…. Tag coworker on contact record (via @dannos)
…. Upgrade to paid HubSpot products
North Star Metric
Case Study: HubSpot CRM
@sjtousley
Case Study: HubSpot CRM
Weekly Active Team = New Teams Activated + Existing Teams Retained - Existing Teams Lost
New Teams Activated = New Teams Acquired * Team Activation Rate
New Teams Acquired = Teams from SEO + Teams from Paid + Teams from Affiliate + ……..
Teams from Paid = Number of impressions * ad conversion rate * landing page conversion rate
Landing page conversion rate = Copywriting + Visual Design + User Experience
North Star
Metric
Zoom out metrics
Zoom in experiments
@sjtousley
Weekly Active Team = New Teams Activated + Existing Teams Retained - Existing Teams Lost
New Teams Activated = New Teams Acquired * Team Activation Rate
New Teams Acquired = Teams from SEO + Teams from Paid + Teams from Affiliate + ……..
10,000 = 5,000 + ???? + 1,500
Case Study: HubSpot CRMNorth Star
Metric
Zoom out metrics
@sjtousley
@sjtousley
W1
W13
W3
W5
W8W9
W11
Weekly Active Team
W12
W10
W7W6
W2W4
W13
W1
W11
W9
W6W5
W3
W2
W4
W7W8
W12W10
W1W2
W3W4
W5W6
W7W8
W9W10
W11W12W13
W13
W12
W11
W10W9
W8W7
W6W5
W4W3 W2 W1
User Activation Rate
Team Activation Rate
Week 12 Retention New Users
Acquired
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
Experiment
ExperimentExperiment
Experiment
Experiment
Experiment
@sjtousley
How do you run experiments?
1
2
3
4
5
6
Identify
Ideate
PrioritizeDocument
Analyze
Share1. Identify the metric
2. Ideate list of experiment ideas to improve metric
3. Prioritize ideas with P.I.E.
4. Write experiment document & turn experiment live
5. Analyze results in experiment document
6. Share results with team
Get the templates!
@sjtousley
Take out your laptops
@sjtousley
scotttousley.com/blog/athens
Experiment Spreadsheet Template
@sjtousley
@sjtousley
@sjtousley
@sjtousley
@sjtousley
@sjtousley
@sjtousley
@sjtousley
@sjtousley
Experiment Document Template
@sjtousley
@sjtousley
@sjtousley
Experiment Presentation Template
@sjtousley
@sjtousley
Insert your text
Problem
@sjtousley
Insert your text
Problem: Today’s UX
@sjtousley
Insert your text
Experiment Design: Proposed UX
@sjtousley
Insert your text (this is optional)
Minimum Viable Experiment (MVE)
@sjtousley
Insert your text
Results
@sjtousley
Next Steps
Insert your text
Case StudyHubSpot CRM: Upgrade Intent
Disclaimer
All data is fake! However, all learnings are very real.
@sjtousley
Inside the product, users have one option if they want to talk to sales -- book a meeting in the future. However, what if users want to talk to sales now?
In addition, we have a poor PQL-to-purchase funnel of only 2%.
Problem
@sjtousley
Problem: Today’s UX
@sjtousley
Problem: Today’s UX
@sjtousley
Problem: Today’s UX
@sjtousley
Problem: Today’s UX
@sjtousley
Problem: Today’s UX
@sjtousley
Problem: Today’s UX
@sjtousley
Problem: Today’s UX
@sjtousley
Problem: Today’s UX
@sjtousley
Experiment Design: Proposed UX
@sjtousley
Experiment Design: Proposed UX
@sjtousley
Experiment Design: Proposed UX
@sjtousley
Experiment Design: Proposed UX
@sjtousley
Experiment Design: Proposed UX
@sjtousley
Experiment Design: Proposed UX
@sjtousley
Experiment Design: Proposed UX
@sjtousley
Experiment Design: Proposed UX
@sjtousley
MVE (Minimum Viable Experiment)
Control
@sjtousley
MVE (Minimum Viable Experiment)
Variant
@sjtousley
ResultsTotal PQLs: 219 Total Conversations: 37
17%
CONTROL
@sjtousley
ResultsTotal PQLs: 160 Total Conversations: 74
46%
VARIANT
@sjtousley
ResultsTotal PQLs: 160 Total Conversations: 74
46%
NEW
`
TL;DR
We’ve nearly tripled the amount of sales conversations.
17% → 46%
@sjtousley
Results
Link to data - https://docs.google.com/spreadsheets/d/1Tb8SRferibKjBZ8T-ORE569NN5VTLr530dO3sKK0VY0/edit#gid=0
@sjtousley
Next Steps
We have a few easy next experiments we could attempt:
● Embed Meetings into the app (versus booking outside of app)● Embed live chat into the app (versus chatting outside of app)● Track the entire funnel per action ● Turn follow-up email into plain text and see how that impacts email conversion rates● Implement these same options on the website … not just in the app
In addition, we have a few more challenging experiments we could tackle:
● When user clicks “Call now” it uses the Calling tool instead of them connecting via their own cell phone
● We use our own live chat product instead of an external live chat tool. This might help with product virality as well.
Exercises
@sjtousley
@sjtousley
@sjtousley
@sjtousley
Step 1) Divide into teams of 10
@sjtousley
Step 2) Choose a company for your team
@sjtousley
Step 3) Fill out the AARRR funnel
- How would you describe the acquisition stage?
- How would you describe the activation stage?
- How would you describe the retention stage?
- How would you describe the referral stage?
- How would you describe the revenue stage?
@sjtousley
Step 4) Brainstorm what you think is their high level growth model
- Use your A.A.R.R.R. funnel as a guide to build your model
- Keep it simple! (for now)
- Everything is a math equation
- Raise your hand if you feel stuck
- ***Break it down until you get to something “actionable”***
@sjtousley
Step 5) Choose a metric you can run an experiment on
- Break down your equation enough to where you can test the copy, ad impressions, etc.
@sjtousley
Step 6) Play around with predictive model to get an idea of different metrics you can
influence!
@sjtousley
Step 7) Remember the metric you identified
1. Identify the metric
2. Ideate list of experiment ideas to improve metric
3. Prioritize ideas with P.I.E.
4. Write experiment document & turn experiment live
5. Analyze results in experiment document
6. Share results with team
1
2
3
4
5
6
Identify
Ideate
PrioritizeDocument
Analyze
Share
@sjtousley
Step 8) Create list of ideas together
@sjtousley
Step 9) Pick your favorite idea you want to prioritize
@sjtousley
1. Identify the metric
2. Ideate list of experiment ideas to improve metric
3. Prioritize ideas with P.I.E.
4. Write experiment document & turn experiment live
5. Analyze results in experiment document
6. Share results with team
1
2
3
4
5
6
Identify
Ideate
PrioritizeDocument
Analyze
Share
@sjtousley
Step 10) Write experiment document
1. Identify the metric
2. Ideate list of experiment ideas to improve metric
3. Prioritize ideas with P.I.E.
4. Write experiment document & turn experiment live
5. Analyze results in experiment document
6. Share results with team
1
2
3
4
5
6
Identify
Ideate
PrioritizeDocument
Analyze
Share
@sjtousley
@sjtousley
Step 7) Remember the metric you identified
1. Identify the metric
2. Ideate list of experiment ideas to improve metric
3. Prioritize ideas with P.I.E.
4. Write experiment document & turn experiment live
5. Analyze results in experiment document
6. Share results with team
1
2
3
4
5
6
Identify
Ideate
PrioritizeDocument
Analyze
Share
@sjtousley
Step 11) Analyze (fake) results
1. Identify the metric
2. Ideate list of experiment ideas to improve metric
3. Prioritize ideas with P.I.E.
4. Write experiment document & turn experiment live
5. Analyze results in experiment document
6. Share results with team
1
2
3
4
5
6
Identify
Ideate
PrioritizeDocument
Analyze
Share
@sjtousley
@sjtousley
@sjtousley
Step 12) Share results
1. Identify the metric
2. Ideate list of experiment ideas to improve metric
3. Prioritize ideas with P.I.E.
4. Write experiment document & turn experiment live
5. Analyze results in experiment document
6. Share results with team
1
2
3
4
5
6
Identify
Ideate
PrioritizeDocument
Analyze
Share
@sjtousley
@sjtousley
@sjtousley
Summary
1. Identify the metric
2. Ideate list of experiment ideas to improve metric
3. Prioritize ideas with P.I.E.
4. Write experiment document & turn experiment live
5. Analyze results in experiment document
6. Share results with team
1
2
3
4
5
6
Identify
Ideate
PrioritizeDocument
Analyze
Share
BONUS: Real-world experiment results
@sjtousley
7 Lessons From Experiments that Grew Blog Traffic by 241%
@sjtousley
Lesson 1: Giveways 2x blog post shares
Version A Version B
SUCCESS
@sjtousley
Lesson 2: Inserting “Click-to-Tweets” boosts shares
SUCCESS
@sjtousley
Lesson 3: Twitter was our most valuable “share” source
@sjtousley
Lesson 4: Our “share retention” lowers over time
@sjtousley
Lesson 5: Influencer shares old articles spikes traffic
SUCCESS
@sjtousley
Lesson 6: Headline formulas can scale blog traffic
SUCCESS
@sjtousley
Lesson 7: Social traffic is not correlated with user signups
FAIL
@sjtousley
8 Lessons from Email A/B Experiments
@sjtousley
A/B Test #1: Zeigarnik Effect
2x 4.4%8.5%CTR CTR
SUCCESS
@sjtousley
A/B Test #2: Scarcity Principle
+120%
SUCCESS
@sjtousley
A/B Test(s) #3: Customizing Email Format SUCCESS
Customizing subscriber emails drove an over 50% increase in reader engagement.
● horizontal image (versus mini thumbnail left-aligned) led to 5% higher CTR
● changing the hyperlinked blog title to a visualized email header led to 21% higher CTR
● using “read more” button over hyperlinked text led to 6% greater CTR
● including author headshot in email led to 6% greater CTR
● shorter subject lines (1-3 words) generated 16% greater open rate than longer ones (full titles, more than 5 words, etc)
@sjtousley
A/B Test #4: Holiday Christmas Poem Email
FAIL
14 sales meetings
@sjtousley
A/B Test #5: A.I.D.A. copy had 25% higher CTR
SUCCESS
Attention
Interest
Desire
Action
@sjtousley
A/B Test #6a: Writing references to popular culture
7 sales meetings
FAIL
@sjtousley
A/B Test #6b: Writing references to popular culture
SUCCESS
407 sales meetings!
@sjtousley
FAIL
A/B Test #7: Nurture healthy user (added 30 contacts) to meeting
@sjtousley
A/B Test #8: Sending automated plain-text emails from sales reps
SUCCESS
@sjtousley
New User Onboarding A/B Test @ Duolingo
@sjtousley
New User Onboarding A/B Test @ Duolingo
@sjtousley
New User Onboarding A/B Test @ Duolingo
@sjtousley
New User Onboarding A/B Test @ Duolingo
SUCCESS
+20% Daily Active Users (DAU)
@sjtousley
A/B Tests of Landing Page Scarcity
@sjtousley
A/B Tests of Landing Page Scarcity
@sjtousley
A/B Tests of Landing Page Scarcity
@sjtousley
A/B Tests of Landing Page Scarcity
@sjtousley
Hypothesis: Users aren’t importing their contacts because they don’t realize they CAN import their contacts. It’s a discoverability problem.
Objective: Increase signup-to-import conversion rate.
Results: This button resulted in a +4% increase in new users importing contacts, which in turn resulted in a +3% increase in new user retention.
Next steps: Determine how much more opportunity there is to increase import CVR. Are we nearing a local maximum of CVR for new user -> import success? Also nail down alternative paths for users to get their data into their CRM. How are the existing paths performing? Are there any additional paths that we can add?
Product A/B Test: Adding “Import” button increases retention by 3%
SUCCESS
@sjtousley
“Smoke Test”
Hint: Use this method to learn something new when there isn’t enough volume for statistically significant A/B test!
User Intent Research: People want videos to learn how to use CRM
@sjtousley
Landing Page A/B Test: Increased # of PQLs to generate
Hypothesis: Adding a Smart CTA to the HubSpot Sales Free homepage, targeting current CRM/Sales Free users, will generate new PQLs.
Objective: Increase number of PQLs
Results: In the past 30 days, this generated 63 new PQLs, closing 5 new business deals at average MRR value of $72, totalling $4,320 in ARR. Avg. PQL-to-close rate is ~8%
Lessons Learned: Adding smart content on product pages is an easy, low-effort way to generate new PQLs from current users.
Next steps: Use predictive model to calculate potential future impact scaling this across other product page.
@sjtousley
● https://blog.hubspot.com/sales/100k-email-templates-follow-up
● https://blog.bufferapp.com/social-media-experiments
● https://patreonhq.com/thats-not-a-hypothesis-25666b01d5b4?gi=c819bff3b75d
● https://darius.com/increase-funnel-conversion-with-psych-7378d51c4caf
● https://www.widerfunnel.com/ab-testing-for-long-term-success/
● https://apptimize.com/blog/2016/02/this-is-how-you-find-your-apps-aha-moment/
● https://www.appcues.com/blog/this-one-in-app-message-increased-sales-demos-by-50
● https://www.nickkolenda.com/copywriting-tips
● https://www.groovehq.com/blog/conversion-rate-optimization-wins-of-2015
● https://okdork.com/why-content-goes-viral-what-analyzing-100-millions-articles-taught-us/
More Examples on Growth Experiments
@sjtousley
The tools you need
● Google Spreadsheets to analyze North Star and “zoom out” metrics growth
● Airtable or Trello as project management tool (experiment backlog)
● Google Presentations to visualize your experiments
● Google Optimize to run A/B tests on your website
● Google Analytics to analyze results on your website
● Atlassian’s Wiki as central place to share “What did we learn this week?”
● Statistical significance calculator to double check results
● HubSpot (or similar tool) to run email A/B tests with marketing automation
● Hotjar for heatmaps, clickmaps, and visitor recordings
● Marketing tools like SEMRush, Buzzsumo, Hunter.io, Ahrefs, etc
Q&A