chris dayley searchlove boston - powerful a/b testing user insights
TRANSCRIPT
#Searchlove @ChrisdayleyPowerful A/B Testing User Insights
dayleyconversion.com/searchlove
#Searchlove @Chrisdayley
So What?
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
What todo?
#Searchlove @Chrisdayley
2013 Non-Keyword Ranking Factors
2015 Non-Keyword Ranking Factors
#Searchlove @ChrisdayleyNot my problem, my designersjust suck
Designers
#Searchlove @ChrisdayleyOnsite metrics can no longer be considered SOMEONE ELSES PROBLEM
Bad or even mediocre design is becoming less & less tolerable.
#Searchlove @Chrisdayley
Wasting Traffic?
#Searchlove @Chrisdayley
So do you needto start doingCRO now?
#Searchlove @Chrisdayley
1Gain InsightsAbout YourAudience
2Move theNeedle
3Easy ContentTest
4TestResults
#Searchlove @Chrisdayley
Gaining InsightsAbout Your Audience
#Searchlove @Chrisdayley
Getting analytical
#Searchlove @Chrisdayley
Christopher Dayley (CD) - I don't really get this animation. Maybe we can change it for something else?
Focus groups
User recordingsessions
User testingfeedback
Eye-trackingsessions
Polls / surveys
Heatmaps
Analytics
#Searchlove @Chrisdayley
Surveys & Polls
#Searchlove @Chrisdayley
For UserExperienceIt`s a load-o-crap
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
#Searchlove @Chrisdayleyhttp://hbs.me/1QleHnI90-95% of purchasedecisions occur in the subconsciousmind
Surveys & polls get therationalization out of people
#Searchlove @Chrisdayley
Focusgroups?
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
http://bit.ly/1qD4oX2
#Searchlove @ChrisdayleyThe Hawthorne Effect:People behave differently because they are being observed
WASTE OFTIME!! Focus groups
Polls
Surveys
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
85%15%
#Searchlove @Chrisdayley
$$ & time
#Searchlove @ChrisdayleyUser recordingsessions
User Testing Feedback
Eye-trackingsessions
Crazy eggClick taleHotjarLucky orange
Heatmaps
#Searchlove @Chrisdayley
Christopher Dayley (CD) - kolby, get these product logos instead of having text
#Searchlove @Chrisdayley
Where are people clicking?
How far are people scrolling?
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
410%410%
Bookings+31%
Moving the Needle
#Searchlove @Chrisdayley
Bounce rates are only badfor certain pages.
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
Bad BounceRates
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
410%410%
Bounce Rates48%
Product Views410%
#Searchlove @Chrisdayley
Dont overload users with content
#Searchlove @Chrisdayley
#Searchlove @Chrisdayley
Content & UX shouldbe given equalplanning time.
#Searchlove @Chrisdayley
BigChanges=BigImpact
#Searchlove @Chrisdayley
Minor changes
#Searchlove @Chrisdayley
Christopher Dayley (CD) - Kolby, let's come up with some kind of "illustration" for big tests and smaller tests. Like "big tests are your tractors, and small tests are your ferrari's" or something, so that we can use an image
Who do you optimize for?
#Searchlove @Chrisdayley
@chrisdayley
@chrisdayley
@chrisdayley
8.1%Funnel Entrance
#Searchlove @Chrisdayley
Christopher Dayley (CD) - Kolby - i thought we were using a different image and/or GIF for fail?
Failed tests are equally as important as successful tests. You just learned what your audience doesn't like.
#Searchlove @Chrisdayley
@chrisdayley
8.1%Funnel Entrance
72%
@chrisdayley
12.3%Funnel Entrance
@chrisdayley
Content Test
#Searchlove @Chrisdayley
Christopher Dayley (CD) - Or your toilet paper tests, cause everyone needs to wipe their buttChristopher Dayley (CD) - Similar to the big/small tests, let's come up with a comparison for content tests. i.e. these are your bread and butter tests, or something everyone should do, like brushing your teeth
How much contentdoes the audiencewant from an article?
#Searchlove @Chrisdayley
Is it different on mobile & desktop?
#Searchlove @Chrisdayley
=?
Tools
#Searchlove @Chrisdayley
Christopher Dayley (CD) - Get the optimizely logo
3 Variations:
Short (< 500 words)Medium (500-1,000 words)Long (>1,000 words)
#Searchlove @Chrisdayley
64.9%
1750 Words1055 Words542 Words76.7%45.5%TOP1:30TOP3:49TOP4:47
#Searchlove @Chrisdayley
542 Words1055 Words1750 Words76.9%62.2%44.4%TOP3:36TOP4:37TOP2:01
#Searchlove @Chrisdayley
Mobile Winner83.1%
Desktop Winner
DONATIONS28.4%
DONATIONS
YOUVE BEEN LIED TO
Test Results
#Searchlove @Chrisdayley
Original
V1V2 V3
146.2%
-46.6%
Form CompletionV3 Winner!
#Searchlove @Chrisdayley
So What?
#Searchlove @Chrisdayley