seo split-testing - why and how
TRANSCRIPT
![Page 1: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/1.jpg)
SEO Split-TestingWhy you should. How you can. What we’ve learnt.
@TomAnthonySEO
![Page 2: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/2.jpg)
![Page 3: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/3.jpg)
![Page 4: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/4.jpg)
But I wore the juice!
McArthur Wheeler
![Page 5: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/5.jpg)
?
Do you think lemon juice makes you
invisible?
![Page 6: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/6.jpg)
Do you think HTTP is better than HTTPS?
![Page 7: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/7.jpg)
Are multiple schema.org entries a
good idea?
![Page 8: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/8.jpg)
Are 302’s really as good as 301’s?
![Page 9: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/9.jpg)
Be careful with untested recommendations.
Don’t blindly believe Google.
!
![Page 10: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/10.jpg)
So today I want to discuss 3 things
How good are we at ranking factors?
How to run tests & pitfalls to avoid
Some of the tests we’ve been running
![Page 11: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/11.jpg)
RANKING FACTORS
![Page 12: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/12.jpg)
Search: future of tv advertising
http://adage.com/article/media/future-tv/303508/
![Page 13: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/13.jpg)
Search: future of tv advertising
http://contentmarketinginstitute.com/2016/04/television-advertising-native/
![Page 14: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/14.jpg)
Search: future of tv advertising
DA PA LRDs Rank
http://dis.tl/rankq-1a
91 47 2 ?
http://dis.tl/rankq-1b
86 57 9 ?
![Page 15: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/15.jpg)
Search: future of tv advertising
DA PA LRDs Rank
http://dis.tl/rankq-1a
91 47 2 3
http://dis.tl/rankq-1b
86 57 9 8
![Page 16: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/16.jpg)
https://www.jensinkler.com/fix-your-kettlebell-swing/
Search: kettlebell technique
![Page 17: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/17.jpg)
http://www.mensfitness.com/training/pro-tips/perfect-kettlebell-swing
Search: kettlebell technique
![Page 18: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/18.jpg)
Search: kettlebell technique
DA PA LRDs Rank
http://dis.tl/rankq-2a
41 42 12 ?
http://dis.tl/rankq-2b
76 44 20 ?
![Page 19: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/19.jpg)
Search: kettlebell technique
DA PA LRDs Rank
http://dis.tl/rankq-2a
41 42 12 4
http://dis.tl/rankq-2b
76 44 20 6
![Page 20: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/20.jpg)
http://www.macworld.co.uk/how-to/iphone/how-to-back-up-iphone-ipad-backup-3595444/
Search: do I need to backup my iphone
![Page 21: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/21.jpg)
http://www.idownloadblog.com/2013/08/28/not-enough-icloud-storage-iphone-cannot-be-backed-up/
Search: do I need to backup my iphone
![Page 22: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/22.jpg)
Search: do I need to backup my iphone
DA PA LRDs Rank
http://dis.tl/rankq-3a
77 59 16 ?
http://dis.tl/rankq-3b
67 34 3 ?
![Page 23: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/23.jpg)
Search: do I need to backup my iphone
DA PA LRDs Rank
http://dis.tl/rankq-3a
77 59 16 7
http://dis.tl/rankq-3b
67 34 3 5
![Page 24: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/24.jpg)
How did we do compared to others?
![Page 25: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/25.jpg)
We surveyed 2000 people for 10 queries
![Page 26: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/26.jpg)
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople
![Page 27: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/27.jpg)
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO
![Page 28: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/28.jpg)
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO
![Page 29: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/29.jpg)
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip
![Page 30: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/30.jpg)
Surprisingly hard.
![Page 31: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/31.jpg)
DeepRank
![Page 32: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/32.jpg)
![Page 33: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/33.jpg)
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip
![Page 34: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/34.jpg)
% of UK search result pairs predicted correctly
30%
40%
50%
60%
70%
Laypeople <= 3 yrs SEO > 3 yrs SEO Coin flip DeepRank
![Page 35: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/35.jpg)
Our instincts aren’t great!
We need data and ways to validate.
![Page 36: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/36.jpg)
CRO Testing
![Page 37: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/37.jpg)
All CRO teams run tests.
If they didn’t you wouldn’t take them seriously.
![Page 38: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/38.jpg)
Why is that not true for SEO teams?
![Page 39: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/39.jpg)
SEO TESTING
![Page 40: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/40.jpg)
IMEC Labs
IMEC Labs
Run experiments against the SERPs to understand aspects of Google’s algorithm.
![Page 43: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/43.jpg)
You can do proper testing too!
![Page 44: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/44.jpg)
RUN YOUR OWN TESTS
![Page 45: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/45.jpg)
Blog Post - http://dis.tl/mark-post
![Page 46: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/46.jpg)
‘Traditional’ CRO A/B Testing is User Oriented
![Page 47: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/47.jpg)
SEO A/B Testing is Page Oriented
![Page 48: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/48.jpg)
SEO A/B Testing includes Googlebot
![Page 51: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/51.jpg)
3 Steps to DIY SEO Split-Tests
1. Create two buckets of pages.
2. Make a change to all pages in one bucket.
3. Analyse which bucket performs better.
![Page 52: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/52.jpg)
1. Create two buckets of pages.
A good DIY approach is using GA segments. Easy to make (see link).
Suggestion: Create Segments by category (e.g. blog tag, product category).
![Page 53: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/53.jpg)
2. Make a change to all pages in one bucket.
Control Variant
This will be your variant bucket.
Utilise your CMS where possible. Ask Dev for scalable approach.
![Page 54: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/54.jpg)
3. Analyse which bucket performs better.
A naive approach can just compare absolute traffic, if you had a close match before.
But needs a big uplift and hard to create such Segments.
Better approach is using Google's Causal Impact library — read this great Lunametrics post.
Variant
Control
![Page 55: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/55.jpg)
DistilledODN
Your Server Your Home Page
CDN
Distilled ODN
Your Optimized Home Page
![Page 56: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/56.jpg)
SOME TESTS WE’VE RUN
![Page 57: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/57.jpg)
ConcertHotels.com
![Page 58: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/58.jpg)
ConcertHotels.com: Test Setup
~20,000 location category pages
Pages
Title: <<Location>> Hotels, NY | ConcertHotels.com H1: <<Location>> Hotels
Before (Control)
Title: Hotels near <<Location>>, NY | ConcertHotels.com H1: Hotels near <<Location>>
After (Variant)
![Page 59: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/59.jpg)
ConcertHotels.com: Results
2.5 weeks to get to significance. Gradual improvement in organic performance leading to steady amount of higher traffic.
Results
![Page 60: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/60.jpg)
SmokyMountains.com
![Page 61: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/61.jpg)
SmokyMountains.com: Test Setup
~100 lodging pages
Pages
schema.org markup: @type “WebSite”. Generic on all pages.
Before (Control)
schema.org markup: @type “LodgingBusiness”. Customised to each page.
After (Variant)
![Page 62: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/62.jpg)
SmokyMountains.com: Results
Fewer test pages than previous test, so it is less smooth but is detected much quicker. Traffic uplift here is estimated to be ~5%.
Results
![Page 63: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/63.jpg)
iCanvas.com
![Page 64: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/64.jpg)
iCanvas.com: Test Setup
~3200 artist category pages
Pages
meta description: Shop our selection of canvas prints by Banksy, each hand-stretched over museum-quality bars and printed with brilliant, fade-resistant inks. Free shipping and returns. internal links: ~50 self referential links
Before (Control)
meta description: Banksy Prints on Canvas, including There Is Always Hope Balloon Girl, Life Is Beautiful and others. Free shipping and returns. internal links: <removed>
After (Variant)
![Page 65: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/65.jpg)
iCanvas.com: Results
2 weeks to get significance. Number of pages is between the previous tests: a few days to be noticed but quite smooth.
Results
![Page 66: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/66.jpg)
SOME OTHER TEST IDEAS
![Page 67: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/67.jpg)
Tabbed vs Flat
![Page 68: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/68.jpg)
Breadcrumbs
!
!
!
![Page 69: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/69.jpg)
Remove Interstitials
![Page 70: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/70.jpg)
We removed all the ads from pages.
Immediate uplift. Ad revenue was 1%. Clear ROI.
![Page 71: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/71.jpg)
Build a library of tests
![Page 72: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/72.jpg)
POTENTIAL PITFALLS
![Page 73: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/73.jpg)
You can’t assume traffic equality between “buckets” of pages
This is why we build a counterfactual comparison using control pages. Use Google’s Causal Impact library to do it yourself.
![Page 74: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/74.jpg)
Pay attention to: Amount of traffic
& number of pages
These two factors will determine how quickly you can test and what size uplifts you can detect.
![Page 75: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/75.jpg)
New pages that appear during tests
The simplest approach is to just ignore all new pages that didn’t exist before the test started.
![Page 76: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/76.jpg)
Different pages can have different seasonality
For example, “roses” pages on valentine’s day. You need to cut outliers.
![Page 77: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/77.jpg)
You could damage conversions, so pay attention to those metrics too.
Tag people in your analytics depending if they arrive on Control or Variant
![Page 78: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/78.jpg)
Run Null Tests
Verify your test framework. Check for false positives.
![Page 79: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/79.jpg)
RECAP
![Page 80: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/80.jpg)
CROs are testing, we should too! I love testing!
![Page 81: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/81.jpg)
You make me so happy, Tom
Boss Friendly! Get Attribution! Get Dev Time!
![Page 82: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/82.jpg)
Small site? Use Mark Edmondson Tool
Bigger site? Use an A/B Approach
![Page 83: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/83.jpg)
Thanks!
@TomAnthonySEO
![Page 84: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/84.jpg)
www.distilledodn.com
DistilledODN allows you to test exactly which changes to your website will result in an uplift in traffic from search engines.
Our new SEO A/B Testing platform is available.
@TomAnthonySEO
![Page 85: SEO Split-Testing - Why and How](https://reader033.vdocuments.us/reader033/viewer/2022052705/587150981a28ab55588b7b59/html5/thumbnails/85.jpg)
Image Credits
• Buckets - mamarazzi
• Rose - alicelingching
• Girders - JFB119
• Rubiks - Wallpaperzone