analytical optimization technologies for games & apps analytics, a/b testing, segmentation &...
TRANSCRIPT
Analytical Optimization Technologies for Games & AppsAnalytics, A/B Testing, Segmentation &
Dynamic Best-Fit
Alan Avidan, Exec. Director & Chief BeezzzDev
Points We’ll Cover
What is optimizationWhat can be measured and optimized
Optimization technologies for games and apps AnalyticsA/B TestingUser SegmentationDynamic Best-Fit
Let’s get started!
Optimization Family Tree
What is Optimization?
Data-driven efforts formulated and designedto maximize Key Performance Indicators (KPI)by enhancing in-game/app conversions
Max Z {f(x)} ≡ f(Engagement, Retention, Monetization,
Virality) X
s.t. g(x)=0, h(x)<0
Which Key Performance Indicatorsshould you target for optimization?
Monetization Engagement Retention Virality
Analytics and Optimization Companies
Optimization Results
88.9% improvement on landing page
Which Game Elements Can Be Optimized?
New Features
Arts (Creative)
Message Wordings
Game Mechanics
Game Flow Landing Pages
Promotions
Optimization Technologies We Use
Analytics A/B Testing (Split Testing)
User Segmentation Dynamic Best-Fit
The process of developing optimal or realistic decision recommendations based on insights derived throughthe application of statistical models and analysisagainst existing and/or simulated future data - Wikipedia
Typical uses of Analytics
Engagement Tracking Funnel Analysis
Measure, Display, Analyze, Change, Repeat
Analytics
Analytics - Bottom Line
Upside• Monitor, record, & display Key Performance
Indicators (KPI) • Measure effectiveness of game mechanics and
monetization Efforts • Access and display data to understand how
users interact with game/app; decide where improvements are needed
Downside The capture and storage of data, followed by
analytics and visualization is tedious, provides retroactive information about the “Average User.”
A/B Testing
Credit: Steve Collins, Swrve
A/B Testing Uses
New features are introduced to a selection of users, and their reactions measured. Features remain only if users engage with them - Wooga
Photo: Spencer Higgins; Illustration: Si Scott
Q: A/B Testing: What are the most unexpected things people have learned from A/B tests?
Answer Wiki
1.Make sure that the test is statistically significant - run it for long enough, and with enough traffic to make it count2.I have learned how dramatically, and ridiculously wrong my most basic assumptions were3.It's empirically proven that you should let the data tell you what works or not and you should constantly be testing4.That the devil is in the detail - a minor change can generate a significant result
A/B Testing – Bottom line
Upside
Simple; understandable; can achieve very good results
Downside:
• One size fit all
User-Base Segmentation
A Priori Segmentation:• Geographic - states, regions, countries• Demographic - age, gender, education• Psychographic - lifestyle, personality, values• Positive - similar wants or needs
Clustering Segmentation:• Behavioral - similarities of behavioral patterns and like-
properties
Segmentation - Uses
Cohort Analysis – Track over time users with common reference feature
Targeting - Serve different treatments for each segment to maximize KPIs
Segmentation – the bottom line
Upside
Can be effective especially reaching out to groups identifiable by known attributes
Downside:
– Clusters are predefined and thus remain unchanged during the analysis
– Requires storage of terabytes of data– privacy issues
Dynamic Best-FitReal-Time Automated Action Optimization
A predictive algorithmic technology used to serve each user the page option they are most likely to convert on at any feature point
DNA Signature Attributes
Geo-Demographic attributes: age, gender, education, countryGeo-Demographic attributes: age, gender, education, country
Facebook attributes: Friends, Likes, Interests, Posts, Events
Behavioral attributes: level, spending, score, progress, custom
Session attributes: time of day, day, durationSession attributes: time of day, day, duration
Proprietary attributes: novice, high-bidder, risk-averseProprietary attributes: novice, high-bidder, risk-averse
3rd Party attributes: income level, education3rd Party attributes: income level, education
How Dynamic Best-Fit Works
Advanced statistical algorithms find strong correlations between user DNA data
and past conversions
Best-Fit Wording
Best-Fit Options
Payment Pages: Different Ranges
Best-Fit Options
Payment Pages: Different Incentives
Best-Fit: Game Flows
Option 1 Option 2Open page
Full tutorial
Stage 1
Open page
Short tutorial
Stage 1
Option 3Open page
No tutorial
Stage 2
Best-Fit: Payouts
Only large and less frequent
winnings*
*The sum of all winnings are the same
Mostly small but more
Frequent winnings*
Best-Fit:
Invite Friends - Different Layouts
Best-Fit: Promotions
Go VIP
Buy VIP card for 5 EURand enjoy 30% more coins
for all future buys
Buy 1 Get 1 FREE
Receive twice theamount of goldfor regular price
Triple your money
Buy 100 GoldGet 200 Free
Best-Fit Arts
Dynamic Best-Fit: Results
“Total Friends” attribute as conversion indicator in payment pageInsight: users with less than 100 friends more readily reach the payment page, and moreover convert better
“Like” attribute as conversion indicator in payment pageInsight: users with more than 25% of Likes associated with apps monetize much better, and moreover clearly prefer Layout 2
Increases conversions and KPIs
Gain Valuable new insights to improve app design and user targeting
Review
• Optimization is vital to your game/app’s success • Retrofit existing games and plan for future
games• Match objectives with technologies: Different
technologies have different uses; Require a different level of involvement; and produce different Uplift results
• Future? -- Lots and lots moredata. Those that will learn toharness it will succeed
