google analytics and cro: deeper conversion insights from google analytics by ayat shukairy

#SMX #14B @ayat

Deeper Conversion Insights from Google Analytics

Google Analytics and CRO

#SMX #14B @ayat

Deeper Conversion Insights from Google Analytics

Google Analytics and CRO

#SMX #14B @ayat

• 3,000+ successful tests

• 400+ CRO projects

• 11 different countries

• 65 years of combined experience

INVESP IN NUMBERS

#SMX #14B @ayat

Analytics tells you what is going on your website; It does not tell why it is happening

#SMX #14B @ayat

You should be able to analyze the data and:1. identify a particular problem in visitor behavior2. why the problem happening3. the impact of that particular problem on user behavior4. the potential revenue gain by fixing that problem

#SMX #14B @ayat

Standard content reports generated by Google analytics describe metrics/performance of a single page.

#SMX #14B @ayat

1. Setup goals to track how visitors navigate around the website

#SMX #14B @ayat

Ecommerce goals1. Homepage to Category pages2. Category pages’ flow through rate (to product page) 3. Search to product4. Search to conversions5. Pre-product page abandonment rate6. Product to category page flow back7. Product page effectiveness rate8. Cart to order confirmation [cart abandonment rate] 9. Checkout completion[checkout abandonment rate]

#SMX #14B @ayat

SaaS goals1. Homepage to pricing page2. Features pages to pricing page3. Comparison pages to pricing page4. Pricing page to subscription confirmation5. Pre-pricing page abandonment rate6. Checkout completion[checkout abandonment rate]

#SMX #14B @ayat

Lead generation goals1. Homepage to services pages2. Services pages to contact “thank you” page3. Content pages to services pages

#SMX #14B @ayat

Evaluate funnels/goals conversion rate before and after conducting AB tests

#SMX #14B @ayat

2. When determining which pages to optimize, translate numbers into dollar value

#SMX #14B @ayat

Potential dollar gainPotential dollar gain = page bounces * AOV * 10%

#SMX #14B @ayat

3. Segment before you test to understand visitor behavior and create hypothesis

#SMX #14B @ayat

Segment based

• New vs. returning• Source type• Logged vs. not • Technology• Geographical regions • Men vs. women• Age range• Content viewed• Action taken

#SMX #14B @ayat

Segment test variation

• Does a particular variation perform better for a specific segment?• Did we capture enough data

to make the assertion?• Does the ROI justify the cost

of personalization?

#SMX #14B @ayat

All visitors

Direct

New

Mobile Desktop

Returning

Mobile Desktop

Organic

New

Mobile Desktop

Returning

Mobile Desktop

CPC

New

Mobile Desktop

Returning

Mobile Desktop

Email

New

Mobile Desktop

Returning

Mobile Desktop

#SMX #14B @ayat

Potential converter segmentsEcommerce:• Navigated to product pages AND• Clicked on “Add to cart button” AND• Never reached “order confirmation” page

SaaS• Navigated to pricing page AND• Clicked on “subscribe” button AND• Never reached “subscription confirmation” page

Category to Product Page Ratio

EMAIL CPC ORGANIC DIRECT

1.11 0.94 1.12 0.83

Category to Product Page Ratio

NEW RETURNING

1.02 0.93

#SMX #14B @ayat

Insights & Recommendations

Category to product page ration shows that category pages are not engaging enough for visitors. This may be caused because:

• Category pages don’t persuade visitors to check out more product pages

• Majority of visitors are focused on purchasing a single product based on an email/ad

#SMX #14B @ayat

When you are starting out, segment the test results to understand visitor behavior

and create hypothesis

#SMX #14B @ayat

After the first 6 months, create AB tests for different visitors segments and

monitor results

#SMX #14B @ayat

The Essentials of Multivariate and AB Testing

ü The Basics of AB Testing

ü The Basics of Multivariate Testing

ü What Elements Should You Test In An A/B Test?

ü AB Testing Best Practices

ü 14 Beginner Mistakes That Will Kill Your A/B

Testing (And What You Can Do About Them)invespcro.com/ab

#SMX #14B @ayat

4. Post test segmentation provides deep insights

#SMX #14B @ayat

A segment must receive enough visitors and conversions to justify testing by itself

#SMX #14B @ayat

Statistical analysis is even more critical when segmenting

#SMX #14B @ayat

Unless you have large website, large numbers of visitors, large number of transactions – do not mix segments

#SMX #14B @ayat

Important data points• Fact of life: A test will generate

a false positive due to random chance. Run your tests with 95% confidence level (5% significance level)

• Data from Google suggest that a new variant of a website is generally only 10% likely to cause a true uplift.

#SMX #14B @ayat

Statistical power• The probability that a statistical test

will detect a difference between two values when there is an underlying difference

• if you don’t calculate the sample size required up-front you might not run your experiment for long enough. Even if there is an uplift you won’t have enough data to be able to detect it.

#SMX #14B @ayat

If you run 100 tests (95% confidence & 80% power)

1. let the tests run long enough to achieve confidence2. 10 out of our 100 variations will be effective and we expect to

detect 80%= 8 tests3. If we use a significance level cutoff of 5% we also expect to see 5

false positives. 4. So, on average, we will see 8 + 5 = 13 winning results from 100

A/B tests.5. 38% of your winning tests are false positives

#SMX #14B @ayat

If you run 100 tests (95% confidence & 30% power)

1. let the tests run long enough to achieve confidence2. 10 out of our 100 variations will be effective and we expect to

detect 30%= 3 tests3. If we use a significance level cutoff of 5% we also expect to see 5

false positives. 4. So, on average, we will see 3 + 5 = 8 winning results from 100 A/B

tests.5. 63% of your winning tests are false positives

#SMX #14B @ayat

When segmenting test results in GA

• Run your tests until you achieve an 80% to 90% power (well powered tests)• Run your tests until you’ve reached validity (beyond significance)• Each variation/segment must have statistical significance, otherwise you

might identify false positives

#SMX #14B @ayat

Major testing tools send data to GA

#SMX #14B @ayat

Some data lends itself to analysis

#SMX #14B @ayat

Others…not so much!

#SMX #14B @ayat

5. Run baseline campaigns

#SMX #14B @ayat

Baseline (monitoring) campaigns• Experiment that doesn’t have any

variations. • The goal is simply to determine

the baseline conversion rate for a certain goal.

• Baseline campaigns should be run for different types of pages

#SMX #14B @ayat

Baseline (monitoring) campaigns• Run baseline campaigns before

and after running an AB test to validate and assess the impact

#SMX #14B @ayat

6. Beware of the winner curse

#SMX #14B @ayat

The winner curse1. Winning designs loose when

deployed to production or on follow up validation tests.

2. This could be due to “novelty affect”

3. Most likely due to false positive

#SMX #14B @ayat

The winner revenue impact• There is no 1 to 1 relation between conversion rate uplift and overall

website revenue.• You are optimizing a portion of the website traffic

• False positives

• The closer you test to the bottom of the funnel, the higher the revenue impact is (homepage test vs. checkout process).

#SMX #14B @ayat

How can I help you?

@ayatayat@invesp.comwww.invespcro.com

@ayat