a/b testing essentials unwebinar with michael aagaard
Post on 08-May-2015
1.669 Views
Preview:
DESCRIPTION
TRANSCRIPT
Thank you for joining us! We’ll be starting in just a few minutes.
Ryan Engley
Ryan is Unbounce’s Director of Customer Success who along with
his team, helps Unbounce customers boost their conversion rates. He is our resident webinar
host and consummate office vegan.
About the Hosts
Michael Aagaard
Michael is a self-confessed split test junkie and copywriting fanatic who’s obsessed with finding out what works in online marketing. He’s Danish and hails from the fair city of Copenhagen.
Join our chat on Twitter
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Recording and Slides Will Be Emailed This Week
Join our chat on Twitter
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Join our chat on Twitter
@Unbounce @ContentVerve #Unwebinar
Ryan EngleyDirector of Customer SuccessUnbounce
@ryan_engley
Tuesday, 23 April, 13
Join our chat on Twitter
@Unbounce @ContentVerve #Unwebinar
Michael AagaardSplit Testing JunkieContentVerve.com
@ContentVerve
Tuesday, 23 April, 13
Join our chat on Twitter
@Unbounce @ContentVerve #Unwebinar
1. Top 5 Questions About A/B Testing
2. Q&A
3. How to Test with Unbounce
Outline
Tuesday, 23 April, 13
1. Why do we need to test?
2. How do we get started?
3. What should we test?
4. How do we set up an A/B split test?
5. When can we stop the test?
The Top 5 Questions about A/B Testing:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
1. Why do we need to test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
1. Why do we need to test?
“Because relying on guesswork and gut feeling is a dangerous business
strategy.”
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
1. Why do we need to test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
1. Why do we need to test?
Control: Treatment:
100% privacy - we will never spam you!
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
1. Why do we need to test?
Control: Treatment:
18.70% less sign-upsStatistical confidence 96%
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
1. Why do we need to test?
Control: Treatment:
We guarantee100% privacy. Your information will not be shared.
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
1. Why do we need to test?
Control: Treatment:
19.74% more sign-ups
Statistical confidence 96%
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
1. Why do we need to test?
“Testing is the only way to make sure your optimization efforts work.”
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
2. How do we get started?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
“Start small and simple and get your footing.”
2. How do we get started?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
2. How do we get started?
Get off to an easy start for free with:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
“Start simple, but use your energy, where it counts.”
2. How do we get started?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
2. How do we get started?
31.54% more sign-upsStatistical confidence 99%
Control: Treatment:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
“Show initial results and get the company buy-in.”
2. How do we get started?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Website
Landing Page
Test Environment
Landing Page
Test Environment
Landing Page
Test Environment
Landing Page
Test Environment
CRO based on findings from
LPO
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
“Pick all the low-hanging fruits.”
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
1. Messaging
2. Forms
3. Call-to-Action Buttons
3 low-hanging fruits on your landing pages:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
Messaging - clarity & relevance are king
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
YOU WORK OUT SMARTER AT FITNESS WORLD
Control:
3. What should we test?
Tuesday, 23 April, 13
YOU WORK OUT SMARTER AT FITNESS WORLD
GROUP TRAINING & FITNESS AT YOUR LOCAL GYM
Control:
Treatment:
3. What should we test?
Tuesday, 23 April, 13
YOU WORK OUT SMARTER AT FITNESS WORLD
GROUP TRAINING & FITNESS AT YOUR LOCAL GYM
38.46% more sign-upsDuring the campaign, the treatment
sold 38.46% more gym
memberships.
Control:
Treatment:
3. What should we test?
Tuesday, 23 April, 13
Why Trade Forex with Forextrading.com?
Control:
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Why Trade Forex with Forextrading.com?
Get Your Free, No Risk, No Obligation$100,000 Forex Trading Demo Account
Control:
Treatment:
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Why Trade Forex with Forextrading.com?
Get Your Free, No Risk, No Obligation$100,000 Forex Trading Demo Account
Control:
Treatment:
99.4% more conversions
Statistical Confidence 99%
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
1. Use the word, “get” when you are drafting your headlines
2. Ask yourself, “why”, “what”, “why”
2 tips for writing high-impact landing page copy
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
Forms - give me a good reason to fill out your form
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
31.54% more sign-upsStatistical confidence 99%
Control: Treatment:
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Control: Treatment:
83.75% more conversionsStatistical Confidence 97%
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
CTA Buttons - relevant copy and “stand-out” design
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Control: Treatment:
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Control: Treatment:
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
90% higher CTRDuring the campaign, the
treatment got 90% more clicks.
Control: Treatment:
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Control: Treatment:
31.03% more payments
Statistical Confidence 98%
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
Control: Treatment:
35.81% more payments
Statistical Confidence 98%
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
CTA
ConversionBounce
The tipping point between bounce and conversion
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
1. Design (color, shape, size)
2. Copy (text in the button)
CTAs consist of two overall elements:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
1. Design answers the question, “Where should I click?”
2. Copy, answers the question, “Why should I click?”
CTAs consist of two overall elements:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
1. Design (color, shape, size)
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
2. Copy (text in the button)
Value and relevance = clicks
Focus on what the prospect gets by clicking the CTA
3. What should we test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
Negative:
Button Copy Landing Page Gym Membership:
The prospects’ motivation is to get a membership at a local gym (the actual location of the gym is crucial)
When they click the button, they’ll get the opportunity to find a gym and buy a membership
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
Negative: Ad value:
Button Copy Landing Page Gym Membership:
The prospects’ motivation is to get a membership at a local gym (the actual location of the gym is crucial)
When they click the button, they’ll get the opportunity to find a gym and buy a membership
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
3. What should we test?
Negative: Ad value: Ad relevance:
Button Copy Landing Page Gym Membership:
The prospects’ motivation is to get a membership at a local gym (the actual location of the gym is crucial)
When they click the button, they’ll get the opportunity to find a gym and buy a membership
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
4. How do we set up an A/B test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
4. How do we set up an A/B test?
A (control) B (treatment)
A/B-test
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
4. How do we set up an A/B test?
“Start with simple A/B tests - 1 control vs. 1 treatment”
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
4. How do we set up an A/B test?
Control: Treatment:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
4. How do we set up an A/B test?
Video “How to set up an A/B test in 2 minutes”
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
“The only thing that’s worse than not testing is relying on bad data.”
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
1. Confidence level
2. Conversion range
3. Sample size
Statistical Significance - 3 Basic Factors:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
Confidence level
+
Conversion range
+
Sample size
=
Statistical Significance
Statistical Significance - 3 Basic Factors:
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
Is A in fact better than B - or are the results random?
A (Control): B (Treatment):
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
How likely is it that A is better than B?
A (Control): B (Treatment):
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
Confidence Level
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
99% Confidence Level=
1% chance that numbers are off
60% Confidence Level=
40% chance that numbers are off
5. When can we stop the test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
Conversion Range
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
Conversion Range
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
Conversion Range
Conversion range (A) 0.91 - 2.91%
Conversion range (A) 1.58 - 3.58%
1%
1%
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
Sample Size
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Sam
ple
Size
Reliability Level
Low High
Larg
eSm
all
zSa
mpl
e Si
ze
Reliability Level
Low High
Larg
eSm
all
5. When can we stop the test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
A lot can happen within the first 100 conversions
5. When can we stop the test?
155 conversions / 1977 visitors
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
A lot can happen within the first 100 conversions
5. When can we stop the test?
155 conversions / 1977 visitors
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
A lot can happen within the first 100 conversions
5. When can we stop the test?
155 conversions / 1977 visitors
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
5. When can we stop the test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Get as close to a 99% Confidence Level as possible
Sample Size large enough to allow at least 100 conversions
Conversion Range of <±1%
Summary Guidelines:
Look for fluctuations (diamond shapes)
5. When can we stop the test?
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Q&A
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Additional Resources from Michael
ContentVerve.com
Twitter.com/ContentVerve
Facebook.com/ContentVerve
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
Additional Resources from Unbounce
twitter.com/unbounce
unbounce.com/blog
The Ultimate Guide to A/B Testing
(we’ll send you these in the follow up email)
@Unbounce @ContentVerve #Unwebinar
Tuesday, 23 April, 13
@Unbounce @ContentVerve #Unwebinar
Stick Around
Learn to Set Up YourFirst Landing Page & Test with Unbounce
See How to Test and Optimize Buttons for Free with buttonoptimizer.com
Tuesday, 23 April, 13
top related