google analytics and cro: deeper conversion insights from google analytics by ayat shukairy
TRANSCRIPT
#SMX #14B @ayat
Deeper Conversion Insights from Google Analytics
Google Analytics and CRO
#SMX #14B @ayat
Deeper Conversion Insights from Google Analytics
Google Analytics and CRO
#SMX #14B @ayat
• 3,000+ successful tests
• 400+ CRO projects
• 11 different countries
• 65 years of combined experience
INVESP IN NUMBERS
#SMX #14B @ayat
Analytics tells you what is going on your website; It does not tell why it is happening
#SMX #14B @ayat
You should be able to analyze the data and:1. identify a particular problem in visitor behavior2. why the problem happening3. the impact of that particular problem on user behavior4. the potential revenue gain by fixing that problem
#SMX #14B @ayat
Standard content reports generated by Google analytics describe metrics/performance of a single page.
#SMX #14B @ayat
1. Setup goals to track how visitors navigate around the website
#SMX #14B @ayat
Ecommerce goals1. Homepage to Category pages2. Category pages’ flow through rate (to product page) 3. Search to product4. Search to conversions5. Pre-product page abandonment rate6. Product to category page flow back7. Product page effectiveness rate8. Cart to order confirmation [cart abandonment rate] 9. Checkout completion[checkout abandonment rate]
#SMX #14B @ayat
SaaS goals1. Homepage to pricing page2. Features pages to pricing page3. Comparison pages to pricing page4. Pricing page to subscription confirmation5. Pre-pricing page abandonment rate6. Checkout completion[checkout abandonment rate]
#SMX #14B @ayat
Lead generation goals1. Homepage to services pages2. Services pages to contact “thank you” page3. Content pages to services pages
#SMX #14B @ayat
Evaluate funnels/goals conversion rate before and after conducting AB tests
#SMX #14B @ayat
2. When determining which pages to optimize, translate numbers into dollar value
#SMX #14B @ayat
Potential dollar gainPotential dollar gain = page bounces * AOV * 10%
#SMX #14B @ayat
3. Segment before you test to understand visitor behavior and create hypothesis
#SMX #14B @ayat
Segment based
• New vs. returning• Source type• Logged vs. not • Technology• Geographical regions • Men vs. women• Age range• Content viewed• Action taken
#SMX #14B @ayat
Segment test variation
• Does a particular variation perform better for a specific segment?• Did we capture enough data
to make the assertion?• Does the ROI justify the cost
of personalization?
#SMX #14B @ayat
All visitors
Direct
New
Mobile Desktop
Returning
Mobile Desktop
Organic
New
Mobile Desktop
Returning
Mobile Desktop
CPC
New
Mobile Desktop
Returning
Mobile Desktop
New
Mobile Desktop
Returning
Mobile Desktop
#SMX #14B @ayat
Potential converter segmentsEcommerce:• Navigated to product pages AND• Clicked on “Add to cart button” AND• Never reached “order confirmation” page
SaaS• Navigated to pricing page AND• Clicked on “subscribe” button AND• Never reached “subscription confirmation” page
Category to Product Page Ratio
EMAIL CPC ORGANIC DIRECT
1.11 0.94 1.12 0.83
Category to Product Page Ratio
NEW RETURNING
1.02 0.93
#SMX #14B @ayat
Insights & Recommendations
Category to product page ration shows that category pages are not engaging enough for visitors. This may be caused because:
• Category pages don’t persuade visitors to check out more product pages
• Majority of visitors are focused on purchasing a single product based on an email/ad
#SMX #14B @ayat
When you are starting out, segment the test results to understand visitor behavior
and create hypothesis
#SMX #14B @ayat
After the first 6 months, create AB tests for different visitors segments and
monitor results
#SMX #14B @ayat
The Essentials of Multivariate and AB Testing
ü The Basics of AB Testing
ü The Basics of Multivariate Testing
ü What Elements Should You Test In An A/B Test?
ü AB Testing Best Practices
ü 14 Beginner Mistakes That Will Kill Your A/B
Testing (And What You Can Do About Them)invespcro.com/ab
#SMX #14B @ayat
4. Post test segmentation provides deep insights
#SMX #14B @ayat
A segment must receive enough visitors and conversions to justify testing by itself
#SMX #14B @ayat
Statistical analysis is even more critical when segmenting
#SMX #14B @ayat
Unless you have large website, large numbers of visitors, large number of transactions – do not mix segments
#SMX #14B @ayat
Important data points• Fact of life: A test will generate
a false positive due to random chance. Run your tests with 95% confidence level (5% significance level)
• Data from Google suggest that a new variant of a website is generally only 10% likely to cause a true uplift.
#SMX #14B @ayat
Statistical power• The probability that a statistical test
will detect a difference between two values when there is an underlying difference
• if you don’t calculate the sample size required up-front you might not run your experiment for long enough. Even if there is an uplift you won’t have enough data to be able to detect it.
#SMX #14B @ayat
If you run 100 tests (95% confidence & 80% power)
1. let the tests run long enough to achieve confidence2. 10 out of our 100 variations will be effective and we expect to
detect 80%= 8 tests3. If we use a significance level cutoff of 5% we also expect to see 5
false positives. 4. So, on average, we will see 8 + 5 = 13 winning results from 100
A/B tests.5. 38% of your winning tests are false positives
#SMX #14B @ayat
If you run 100 tests (95% confidence & 30% power)
1. let the tests run long enough to achieve confidence2. 10 out of our 100 variations will be effective and we expect to
detect 30%= 3 tests3. If we use a significance level cutoff of 5% we also expect to see 5
false positives. 4. So, on average, we will see 3 + 5 = 8 winning results from 100 A/B
tests.5. 63% of your winning tests are false positives
#SMX #14B @ayat
When segmenting test results in GA
• Run your tests until you achieve an 80% to 90% power (well powered tests)• Run your tests until you’ve reached validity (beyond significance)• Each variation/segment must have statistical significance, otherwise you
might identify false positives
#SMX #14B @ayat
Major testing tools send data to GA
#SMX #14B @ayat
Some data lends itself to analysis
#SMX #14B @ayat
Others…not so much!
#SMX #14B @ayat
5. Run baseline campaigns
#SMX #14B @ayat
Baseline (monitoring) campaigns• Experiment that doesn’t have any
variations. • The goal is simply to determine
the baseline conversion rate for a certain goal.
• Baseline campaigns should be run for different types of pages
#SMX #14B @ayat
Baseline (monitoring) campaigns• Run baseline campaigns before
and after running an AB test to validate and assess the impact
#SMX #14B @ayat
6. Beware of the winner curse
#SMX #14B @ayat
The winner curse1. Winning designs loose when
deployed to production or on follow up validation tests.
2. This could be due to “novelty affect”
3. Most likely due to false positive
#SMX #14B @ayat
The winner revenue impact• There is no 1 to 1 relation between conversion rate uplift and overall
website revenue.• You are optimizing a portion of the website traffic
• False positives
• The closer you test to the bottom of the funnel, the higher the revenue impact is (homepage test vs. checkout process).
