iscool entertainment @big data paris
DESCRIPTION
"Big Data + social + games @IsCool"A presentation by IsCool Entertainment at the Big Data Congress Paris.TRANSCRIPT
Big Data + Social + Games @Is Cool
3/24/12 TITRE DOCUMENT
IsCool Entertainment
! Social game publisher based in Paris, France
! #1 French publisher in terms of audience (450k Daily AcKve Users) & revenue
! 2.8 Million Fans ! 80 employees ! €9.1 million revenue in 2010 ! 4 live applicaKons on Facebook
Florian DoueTeau CTO @fdoue?eau
Agenda • What do we do Social Gaming • What kind of (Big) AnalyKcs we do Lots • How we do it Hadoop, Python, R, Tableau, Gephi and stuff…
Is Cool Games
Is Cool, Delirious CollecKble
Game
Absolute Solitaire, The best solitaire game available online
Temple Of Mahjong, Collect, Play, Exchange
Belote MulKjoueur, Play, Win, Meet
Games & Virtual Goods
! Play the Game & Gain some virtual goods
! Play again & Gain more ! Collaborate with other players
& Gain More ! …. ! Possibly buy § To grow quicker § To help others
Virtual Goods Virtual Economy
! Virtual Goods Must not be too easy to get § The game would not be fun! § No moneKzaKon
! Virtual Goods must not be hard to get § People would churn because of
frustraKon! ! Virtual Goods can usually be
traded between players ! Virtual and actual “Price” of a
good
Let’s Trade 1 Watch against 3 Hammers
Why is this Big Data ?
! Number of object transacKons per day § NYSE 3,600,000,000 § 1,600,000,000
§ 1,500,000,000
§ IsCool 1,400,000,000
§ 860,000,000
§ CAC 40 142,500,000
9,8 TB Data to analyze
18 Million user-‐generated acKons per day 7 Billion per year.
The Real Big Data Challenge Collaborate for collecQve insights
data scienKst?
what metrics?
Real-‐Kme?
Game Designer PerspecKve : Nice Charts ?
Programmers’ PerspecKve : Log Files & Work ?
Business Guy PerspecKve: Revenue Forecast ?
BI Veteran: Schema DefiniKon ?
Specifics of Game AnalyQcs
! Virtual Goods § We are the Factory AND the
Shop, and most of the products are free.
! Social Networks § Network effects are key
! Games § The product changes EVERY day ! § Sudden wage of unexpected
players from Guatemala ! § People try to cheat !
Use Case 1: Engagement Drivers
! StaKsKcal Mesaure of Engagement § Visit Frequency, DAU /
MAU ! Analyze Engagement Drivers § Use of Features ? § Demographics ? § How does it relate in Kme
with moneKzaKon ? § ….
3/24/12
Understanding Engagement -‐ Results
! Establish class of users with different engagement profile and use of features
Understanding Engagement – Benefits
! Adapt the features correlated with strong engagement and interesKng profile so that they can easily be accessed by other players
3/24/12
Use Case 2 : Understanding Users as a whole
! 10 Million Nodes ! Around 1 Billion Edges
! How does the graph evolve in Kme ?
! What are the communiKes? ! Leaders ? ! CorrelaKon with engagement,
virality , etc.. ?
Understanding Users as a Whole – Clusters and Graphs
A very large community
Some mid-‐size communi6es Lots of small clusters mostly 2 players)
! Specific communiKes in the graph
! CorrelaKon between community size and engagement / virality
! DetecKon of paTerns § 2 players paTerns § Family play § Group Play § Open Play (language
community)
Understanding Users as a Whole -‐ Benefits
3/24/12
! Cluster-‐oriented Community-‐Management § Engage with a community as
a whole as much as possible
! Nurture communiKes § Make communiKes grow unKl
they reach a criKcal mass § Reduce language barrier to
help community aggregaKons ! DetecKon of “opinion leaders”
Use Case 3 : Long Terms effects of a feature
3/24/12 TITRE DOCUMENT
! Are players using the new feature… § Happy with it ? § More engaged? § Generate more virality ? § etc….
! A/B Tests § Some features can be A/B tested § …and some cannot ! § How to measure the uplio ?
! Complexity § MulKple variable to observe (other
features, history, and 20 more ….) § Long term non local effect (game
economics)
Long Terms Effects of a feature -‐ Results
3/24/12
! Adapt game rules to fit most of the players § No InflaKon § But maintain Growth !!
How did we do that ?
In the past 4 years …. • Tools changed • Scale changed • Focus Changed
Technological Offering • Commercial / Open Source ETL • Commercial BI VisualizaKon Sooware
• Commercial / Open Source databases (column stores)
• …
• Big Data Approach
2010-‐2011
• BI Approach
2009-‐2010
• Basic Approach
2008-‐2009
What we learned
Diversity
RelaKvity
CollaboraKon
No Hadoop+R Magic
Windows / Linux ?
What is real budget ?
VisualizaQon is more important than precision
Do you have data mining experts (yes/no) ?
Do you have scalability experts ?
Cloud or on-‐premise ?
Do you want anybody to play with the data ?
No XYZ Magical Product
! Ad-‐hoc -‐ Datamining tools § To Discover new trends § Ad-‐hoc analyKcs § Graph AnalyKcs
! Week-‐to-‐Week -‐ Datawarehousing § Detailed Business Metrics § Virtual Economy Modeling § Long-‐term behaviours § Business Level Visibility
AdapQve AnalyQcs
! Day-‐To-‐Day -‐ SaaS AnalyKcs Plarorms § For common, business metrics
(virality, traffic, engagement) § Corporate Level Visibility § Day-‐to-‐day
Internal Data Warehousing
• +Direct connecKon to the database
• +Excel fan biz guy can use it with no training !
VisualizaKon (Tableau Sooware)
• Free (as beer) • Good performance for analyKcs tasks on a few hundreds million lines ( SELECT … GROUP BY … ORDER … )
• Featured and limited performance compared to commercial Column Stores
Columnar Database (Infinidb)
• Pure Python ETL • Good integraKon with AWS/ S3
• Easy to integrate in our development environment
ETL (PyBabe)
• Used to reduce the amount of informaKon : 10 GB a day => 1GB a day
• High cost of development for business-‐related processing
MapReduce (Hadoop/Hive)
?