meetup 16 dec data driven process to optimization
TRANSCRIPT
The data driven process to identify, prioritize, launch and monitor your CRO experiments
What is conversion optimization?
Conversions – Actions you want your visitors to take on your site◦ Signs up to your newsletter◦ Buys a product ◦ Downloads a file◦ Etc.
Conversion optimization – understand your customers, provide them with a better experience on your site and simplify everything so they can convert more.
The Big PictureConversion optimization is about doing better
marketing
Assuming KnowingData
The Big Picture
“Data are just summaries of thousands of stories – tell a few of those stories to
help make the data meaningful.” – Chip & Dan Heath
Data Information Insight
Don’t just assumeIt makes an ass
out of uand me
Mindset of the Optimizer
Rule #1: There is no one size fits all
Mindset of the Optimizer There is no “this always works”
Ego and Bias – not your friends when it comes to optimization
We need to know, not assume
Typical CRO Process
If you can’t describe what you are doing as a process, you don’t know what you’re doing.
– W. Edwards Deming
Determine Business Objective – KPIs,
Metrics to increaseSetup Data Gathering
Insight Phase – Sources of traffic,
patterns, personas
Define Hypotheses
DesignTechnical IntegrationTesting
Learning and Improving Customer
TheoryBack to Step 2
Prioritize Hypotheses – Testing Plan
Conversion Research 101 Experience based assessment
◦ Relevance
◦ Clarity
◦ Value
◦ Friction
◦ Distraction
Conversion Research 101
-Online surveys
-Web traffic surveys
-Live chat transcripts
-Customer support insights
-User testing
Qualitative Research
-Web analytics research
-Mouse tracking analytics
Quantitative Research
Developing a hypothesis With each hypothesis we want to understand:
◦Which problem we’re solving
◦What’s the solution
◦Which metric we are trying to improve with this
Hypothesis Formula
By changing [problem] into [solution] we will increase [metric ]
All hypotheses should come as a result of conversion research: KNOW, don’t GUESS
Prioritizing Experiments Optimizing the optimization process is often just as important as the tests themselves
WiderFunnel uses this PIE Framework:
PIE Framework:
PotentialImportanceEase
T.I.R. Method◦ Time 1 – longest time 5 – shortest time
◦ Impact 1 – lowest impact 5 – highest impact
◦ Resources 1 – most resources 5 – least resources
Multiply these scores together and start working on the projects that have the highest scores as those will have the most lift with the least amount of time and the least amount of resources.
Test It Right Wait until statistical significance
Make sure your sample size is big enough
Sample size calculator: http://www.evanmiller.org/ab-testing/sample-size.html
Don’t call tests too early
Validity Threats History effect
Instrumentation effect
Selection effect
Compatibility issues – broken code
Questions?