SUBTITLE BELOW!
What gets you excited?!LOOKING AS A WEB ANALYST TO CONVERSION OPTIMIZATION!
[email protected]
Evidence based growth
[email protected]
Conversionrateop-miza-on
Analytics Psychology
Lots of Experiments
[email protected]
F!FIND
T!TELL
A!ANALYZE
C!COMBINE
C!CREATE
A!ANALYZE
T!TEST
F!FIND
T!TELL
A!ANALYZE
C!COMBINE
C!CREATE
A!ANALYZE
T!TEST
SUBTITLE BELOW!
What gets you excited?!
Ü ANALYZING HOW USER BEHAVIOUR IS CHANGING!!
@AM_Klaassen / @tomvdberg1
We started with them
7 years ago
Flow optimization
[email protected]
B A
Winner
Flow optimization
Dialogue optimization
[email protected]
Winner
B A
Flow optimization
Customer Journey Optimization
Dialogue optimization
[email protected]
#ConvConOnline Dialogue & Testing Agency
[email protected]
Did you already decided which hotel you want to stay in?
Are you currently comparing the price of the rooms in this hotel at other websites?
[email protected]
Book with your Valk account and get a €5 bonus
Easily control your booking with your Valk account
SUBTITLE BELOW!
What gets you excited?!
Ü INJECTING FEEDBACK FORM RESPONSES INTO ANALYTICS!
[email protected]
Toolsforrunningexperiments
[email protected]
Frontenddevelopment
[email protected]
WemeasureresultsinGoogleAnaly-cs
[email protected]
Theydohaveanaly-cs–butnotenough
[email protected]
Eventtrackingvaria-onbehaviour
[email protected]
WhynotuseGoogleTagManager?
[email protected]
Jorrin Quest
hDp://gtmtes-ng.com
Soweopenedupourlibraryforyou
[email protected]
ü Easy setup
ü Free
ü Preview mode
hDp://gtmtes-ng.com
[email protected]
Resultsspreadsheetthroughtheanaly-csAPI
[email protected]
DownloadourExcelforfree:hDp://ondi.me/vis
[email protected]
DownloadourExcelforfree:hDp://ondi.me/vis
SUBTITLE BELOW!
What gets you excited?!
Ü AUTOMATED GTM / GA / EXCEL RESULTS SET-UP !
[email protected]
• Exampletest:Orderflow
[email protected]
• Exampletest:Setcomple-on
A B
[email protected]
• Testresult!
[email protected]
• No effect on conversion à Don’t implement à Re-test with higher volumes
• Conclusion !
[email protected]
• What’sthealterna-ve?
Frequentist statistics
Bayesian statistics
[email protected]
• Exampletest:Setcomple-onA B
[email protected]
• BayesianTestevalua-on
[email protected]
• Focusonriskassessment
88,4%
A test result is the probability that B outperforms A: ranging from 0% - 100%
[email protected]
!
IMPLEMENT B! PROBABILITY! * EFFECT ON REVENU!
Expected risk 11,6% - € 69.791
Expected uplift 88,4% € 213.530
Contribution € 180.733
* Based on 6 months and an average order value of € 175
• Makeariskassessment
[email protected]
• ImplementBB
[email protected]
abtestguide.com/bayesian/!
Roy Schieving
Annemarie Klaassen
[email protected]
• ConclusionBayesian
ü Easier to understand and communicate
ü Better suits the business ü Don’t throw away good ideas
(indicatively significant)
ü Higher implementation rate and revenue
€0
€1.000.000
€2.000.000
€3.000.000
€4.000.000
€5.000.000
€6.000.000
1 3 5 7 9 111315171921232527293133353739
Expected impact on revenue in 1 year
€Frequen-st €Bayesian
SUBTITLE BELOW!
What gets you excited?!
Ü BAYESIAN STATISTICS APPLIED ON EXPERIMENT RESULTS!
[email protected]
Book with your Valk account and get a €5 bonus
Easily control your booking with your Valk account
[email protected]
50%
50%
A
B
• A/B-test
[email protected]
• Trafficalloca-onover-me
A
B
Traffic
Time
Profit or loss?
[email protected]
Themul-armbanditproblem
[email protected]
• Sothiswillhappeninyourexperiment:
A
B
Traffic
Time
More Profit!
[email protected]
Explore Exploit ?
• Explore/Exploitdilemma
[email protected]
Limited time frame Automation
• WhenuseBandits?
Earn over learn
[email protected]
Website
GTM testing
A
B
Google Tag Manager
User has Test cookie
No
Yes
Traffic Random
50/50
Show A or B
• A/Btest
[email protected]
Website
GTM testing
A
B
Google Tag Manager
User has Test cookie
No
Yes
Traffic Random
50/50
Show A or B
Must be dynamic! • Bandit
[email protected]
Website
GTM testing
A
B
Google Tag Manager
User has Test cookie
No
Yes
Get traffic allocation
Show A or B
Hourly check GA data for test
Algorithm
Write Traffic allocation,js
• Bandit
[email protected]
Get traffic allocation
Hourly check GA data for test
Algorithm
Write Traffic allocation,js
[email protected]
Get traffic allocation
Hourly check GA data for test
Algorithm
Write Traffic allocation,js
[email protected]
Get traffic allocation
Hourly check GA data for test
Algorithm
Write Traffic allocation,js
[email protected]
Get traffic allocation
Hourly check GA data for test
Algorithm
Write Traffic allocation,js
[email protected]
Get traffic allocation
Hourly check GA data for test
Algorithm
Write Traffic allocation,js
[email protected]
Website
GTM testing
A
B
Google Tag Manager
User has Test cookie
No
Yes
Get traffic allocation
Show A or B
Hourly check GA data for test
Algorithm
Write Traffic allocation,js
• Bandit
[email protected]
• Examplecase
[email protected]
• Examplecase
ü Older target group
ü Comparing price on other sites
ü Behavior of others
[email protected]
• Example1
Control! No message
Variation B! Copy default
Variaton C!
Variation D!
[email protected]
11,1%
10,6%
10,4%
10,5%
[email protected]
• Example1–Effectonrevenue
A
B
Traffic
Time
+ € 6.637
[email protected]
• ConclusionBandits
ü Less regret during testing
ü No implementation costs ü It never stops!
[email protected]
Lay-overno-fica-ons
[email protected]
www.smartno-fica-ons.com
SUBTITLE BELOW!
What gets you excited?!
Ü AUTOMATED PERSUASIVE MESSAGING FOR YOUR WEBSITE!
[email protected]
www.smartno-fica-ons.com
SUBTITLE BELOW!
Ü ANALYSING HOW USER BEHAVIOUR IS CHANGING!!
Ü INJECTING FEEDBACK FORM RESPONSES INTO ANALYTICS!
Ü AUTOMATED GTM / GA / EXCEL RESULTS SET-UP !
Ü BAYESIAN STATISTICS APPLIED ON EXPERIMENT RESULTS!
Ü AUTOMATED PERSUASIVE MESSAGING FOR YOUR WEBSITE!
SUBTITLE BELOW!
Thank you!!TWITTER: @TONW – EMAIL: [email protected]!