discovery-driven design in social games: techniques, processes, and problems
DESCRIPTION
Heather Stark Kinran. Discovery-driven design in social games: techniques, processes, and problems. Strata Conference London October 1 2012. Market dynamics. Service access, delivery, and pricing. Market dynamics. Service access, delivery, and pricing. Social, mobile, and ‘free’. F2P. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/1.jpg)
1
@HA
Star
k
Discovery-driven design in social games: techniques, processes, and problems
Heather StarkKinran
Strata ConferenceLondonOctober 1 2012
![Page 2: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/2.jpg)
2
@HA
Star
k
Market dynamicsService access, delivery, and pricing
![Page 3: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/3.jpg)
3
@HA
Star
k
Market dynamicsService access, delivery, and pricing
![Page 4: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/4.jpg)
@HA
Star
k
4
Social, mobile, and ‘free’
+ + F2P
![Page 5: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/5.jpg)
@HA
Star
k
5
“The biggest gold rush in the history of games” – Venturebeat
![Page 6: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/6.jpg)
@HA
Star
k
6
http://venturebeat.com/2012/01/06/deanbeat-game-companies-raised-a-record-breaking-1-55b-in-2011/
$3.75 billion
![Page 7: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/7.jpg)
@HA
Star
k
7
But.....
http://www.google.com/finance?cid=481720736332929, Sept 19 2012
Zynga Facebook
-76%
-40%IPO
IPO
![Page 8: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/8.jpg)
@HA
Star
k
8
Social [vs, plus] mobile
A lot of the development and the energy in the eco-system is not going towards building desktop stuff anymore, it's going towards building mobile stuff
Mark ZuckerbergSept 11 2012http://www.reuters.com/article/video/idUSBRE88A1F220120912?videoId=237677561
...in July, Facebook sent people to the Apple App Store and Google play more than 170 million times...
https://developers.facebook.com/blog/post/2012/09/14/facebook---gdc-europe--developer-day-recap/
![Page 9: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/9.jpg)
@HA
Star
k
9
Has the world ended? Facebook
nearly 1 bn Facebook users 25% play games regularly virtual goods spend on FB estimated at $1.2 billion 2011
iOS and Android games revenue 2011 = 2x traditional portable console
(reverse of 2009) in-app purchases 87% of revenue of top 25 grossing apps total worldwide game revenue $1.6 bn 2011 68% of sessions from ‘indie’ developers (i.e. new market
entrants)
Sources: Facebook, Flurry, InsideNetworks
![Page 10: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/10.jpg)
@HA
Star
k
10
Market concentration has changed
Source: Velo Partners aggregation of AppData datahttp://www.slideshare.net/evanvrs/end-of-the-bull-market
![Page 11: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/11.jpg)
11
@HA
Star
k
Analytics is everywhere
![Page 12: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/12.jpg)
@HA
Star
k
12
Everyone’s doing itDevelopment really starts with
launch
Playfish
King.com
10-15% of staff on analytics
one release every day
more daily transactions than
the FTSE
IsCool (formerly WEKA)
Analytical creativity
extends the product lifecycle
hired Ocadoanalytics lead
after exec search
http://www.gamesbrief.com/2010/04/playfishs-advice-for-building-social-games-development-really-starts-with-launch/ Philip Reiseberger, Chief Games Officer, BigPoint, Evolve in London conference December 2011http://www.renovatapartners.com/news/kingcom-appoints-director-bihttp://www.facebookgarage.org.uk/talk/big-games-vs-big-datahttp://www.slideshare.net/IsCoolEnt/iscool-entertainment-big-data-paris
Playfish
![Page 13: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/13.jpg)
@HA
Star
k
13
Doing what?
![Page 14: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/14.jpg)
14
@HA
Star
k
Why?
Design = Change
![Page 15: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/15.jpg)
15
@HA
Star
k
ProcessesHow?
![Page 16: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/16.jpg)
@HA
Star
k
16
Capturing ‘everything’
![Page 17: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/17.jpg)
@HA
Star
k
17
While testing concurrent versions
We do hundreds
and hundred of
experiments every
quarter.
Most fail.
Ken Rudin, General Manager, Analytics2010
Lots of experiments... ~3-5K active at any time...
Manyfail.
http://www-conf.slac.stanford.edu/xldb2012/talks/xldb2012_wed_1125_DanielMccaffrey.pdf
http://tdwi.org/videos/2010/08/actionable-analytics-at-zynga-leveraging-big-data-to-make-online-games-more-fun-and-social.aspx
Daniel McCaffrey, General Manager, Platform and Analytics Engineering2012
![Page 18: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/18.jpg)
@HA
Star
k
18
And.... adapting the design
Any designelement...
(almost)any
designelement
And... adapting the design variations
Superficial
Fundamental
![Page 19: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/19.jpg)
19
@HA
Star
k
(this slide intentionally left blank)
![Page 20: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/20.jpg)
@HA
Star
k
20
Basic metrics: BI (with a bit of UX)
BI
Acquisition cost(service delivery
costs)
Customer lifetime value
UX
Funnel tracking Cohort
segmentation
EngagementMonetisation
Retention
![Page 21: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/21.jpg)
@HA
Star
k
21
Some popular metrics are bad
What is the right metric for ‘engagement’?
Which is the ‘better’ pattern?
Facebook resell per app summary data about DAU and MAU, and leaderboard info is widely freely available
people think DAU/MAU is about retention and engagement
but... it ain’t
![Page 22: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/22.jpg)
@HA
Star
k
22
No KPI is good but thinking makes it so
RetentionEngage-
ment
Virality
Monetis-ation
LTCVCustomeracquisitio
ncost
![Page 23: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/23.jpg)
@HA
Star
k
23
Virality, retention, and value
Diffusion dynamics of games on online social networks, Wei, Yang, Adamic, de Araújo and Rehkihttp://www-personal.umich.edu/~ladamic/papers/FBgames/FBgameDiffusion.pdf
Recruited by friend
Other
Play after day 1?No Yes
10% of inviters responsible for 50%of successful invitations
Retention
Value
![Page 24: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/24.jpg)
@HA
Star
k
24
Analytics
Experience
types
KPI
LTCV
Out-comes
Actions
Playertypes
is about discovering these relationships
Context
Options
![Page 25: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/25.jpg)
@HA
Star
k
25
Context is complex
Concurrent
Sequential
Imputed
goal
loss
risk
loss
choicereward
Factual
unease momentum suspenseresource level friend presentaction
or
event
Filter
![Page 26: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/26.jpg)
26
@HA
Star
k
Toolsand techniques
![Page 27: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/27.jpg)
@HA
Star
k
27
Tool types
Logging Report/query Exploratory Model testing
App tracking
BI frontend
Data science
Activities
![Page 28: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/28.jpg)
@HA
Star
k
28
The tools market is busy
GENERAL dozens of app tracking
solutions some platform specific some more gamey than others lots of new market entrants
BI front-ends are generic visual exploration important
Data science tools can be so heavily wrapped in
services you can’t see themor
come as a blinking cursor – DIY – open sourceor
as a bolt-on to std stats tools
ACTIVE EUROPEAN PLAYERS
Pingflux Playful Honeytracks Geosophy Fireteam Qlikview ..
Data-mining services Game analytics - UK Games analytics – Danish ..
![Page 29: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/29.jpg)
@HA
Star
k
29
Tool trends
Logging Report/query Exploratory Model testing
App tracking
BI frontend
Data science
Activities
graph dbpredictiv
escloud
services
Testing+Servin
g
![Page 30: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/30.jpg)
@HA
Star
k
30
Data mining services focus
Sonamine’s Player Lifecycle Management ™
ConvertSoon™ChurnSoon™PurchaseMoreSoon™InfluenceSoon™
Services focus on predicting behaviours with direct immediate revenue impact
Source: sonamine.com
e.g.
![Page 31: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/31.jpg)
@HA
Star
k
31
Remedies are not always obvious
“We have tested over 60 individual and game-specific metrics. None of them are critical enough to cause churn. None of them! We haven't found a silver bullet -- that magic barrier preventing players from enjoying the game.”
http://gamasutra.com/view/feature/170472/predicting_churn_datamining_your_.php
Dmitry Nozhnin, Head of analytics and monetisation, Innova:
![Page 32: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/32.jpg)
@HA
Star
k
32
Meaningful segments are juicier
Revenue Potential
Vira
lity
Pote
ntial
31%0.89%22%
$1.75
%Volume%Paying7Day Ret
CAC25%
1.30%26%
$2.21
5%0.19
9%$2.38
14%0.9721%$1.9
4
Early Enthusiasts
Confident Completers
Social Involver
Sporadic SemiEngaged
Losing Momentum
Need Guidance
Borderline Incompetent
6%2.34
%57%$4.4
0
12%0.8659%$3.5
7
7%0.55
%36%$0.7
5
Source: Games Analytics
![Page 33: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/33.jpg)
@HA
Star
k
33
SaaS VAS segment wrapping
Playnomics/Naked Communications
SaaS
VAS
PlayRM™ Messaging, individualised based on segmentation/scoring
• apply unsupervised learning to id patterns in the type, frequency and sequence of player actions
• predict P(return, engage, invite, monetize) using supervised learning
PlayRM™ Marketplace, target players by their historical and predicted game behaviour
Source: Playnomics.com
![Page 34: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/34.jpg)
@HA
Star
k
34
Is home-made juice tastier? useful open source tools readily available
r + libraries WEKA Python + libraries
data volumes quickly lead practitioners to become interested in:
sampling but... ‘interesting’ events are usually long-tailed
efficiency e.g. simplex volume maximisation for convex hull constrained matrix factorisation in n
▪ Christian Thureau, IT University of Denmark, Data Mining in Games, http://vimeo.com/14390303 parallelisation
Hadoop map/reduce in particular some ML algorithms more suitable for Map/Reduce style parallelisation than others
▪ KDD2011, Vijay Narayanan (Yahoo!) and Milind Bhandarkar (Greenplum Labs, EMC), Algorithms in modelling with Hadoop http://www.slideshare.net/hadoop/modeling-with-hadoop-kdd2011
![Page 35: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/35.jpg)
35
@HA
Star
k
Problems
BoredomBafflement
![Page 36: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/36.jpg)
@HA
Star
k
36
Boredom
![Page 37: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/37.jpg)
@HA
Star
k
37
And that data eventuallybecomes a crutch for every decision,paralyzing the company and preventing it from making any daring design decisions.”
“When a company is filled with engineers, it turns to engineering tosolve problems. Reduce each decision to a simple logic problem.Remove all subjectivity and just look at the data..
http://stopdesign.com/archive/2009/03/20/goodbye-google.html
For a more recent and enthusiastic exposition of Google’s split testing approach, see ex-Googler Josh Wills’ take on it:Experimenting at Scale, http://www.stanford.edu/group/mmds/slides2012/s-wills.pdfand http://berkeleydatascience.files.wordpress.com/2011/03/20110301berkeley.pdf
![Page 38: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/38.jpg)
@HA
Star
k
38
"when people start building their games to a spreadsheet its like putting cars through a wind tunnel-they all come out
looking like bullets"
http://www.gamesindustry.biz/articles/2012-09-05-remedy-entertainment-real-artists-shipMatias Myllyrinne, CEO, Remedy Entertainment
![Page 39: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/39.jpg)
@HA
Star
k
39
Tadgh Kellyhttp://www.slideshare.net/tadhgk/the-four-lenses-of-game-making-and-social-gameshttp://www.whatgamesare.com/2011/12/the-four-lenses-of-game-making.html
![Page 40: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/40.jpg)
@HA
Star
k
40
?
?
Bafflement
![Page 41: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/41.jpg)
@HA
Star
k
41
We don’t understand what’s going on. All we know is we’re going to keep running these experiments to try and understand better what it is that our customers are telling us. And there are clearly things that we don’t understand because a simple analysis ...implies very contradictory yet reproducible results. So clearly there are things that we don’t understand, and we’re trying to develop theories for them. It’s... an exciting time but also a very troubling time.
Gabe Newell, Founder, ValveOct 23 2011http://www.geekwire.com/2011/experiments-video-game-economics-valves-gabe-newell/
June
23
2012
Val
ue h
ires
Gree
k ec
onom
ist in
re
siden
ce to
look
at s
hare
d cu
rrenc
y iss
ues.
![Page 42: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/42.jpg)
@HA
Star
k
42
Designer = bean-counter, orBEAN-COUNTER DESIGNER?
![Page 43: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/43.jpg)
@HA
Star
k
43
Matt FalcusProduct ManagerTeam 17 Software
As someone who wants to analyse our games, but with no detailed knowledge of how to query databases or program, I need the easy-to-use solutions with the pretty presentation, and I need our web guys to set them up. I suspect a lot of
companies who are new to analytics will be the same - the staff who need to
analyse are not qualified to deep data mine themselves.
Product manager = data-miner?
![Page 44: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/44.jpg)
@HA
Star
k
44
Industry context
4th producer globally
£1b to GDP
~4000 people ~500 companies
Skillset (2008)
9,000 developers down 11% from
2008
TIGA (2011)
http://www.creativeskillset.org/uploads/pdf/asset_16891.pdf, 19/9/2012http://www.tiga.org/about-us-and-uk-games/uk-video-games-industry, 19/9/2012
Less than 55 to 10
11 to 2021 to 50
51 to 200over 200
0% 5% 10% 15% 20% 25% 30%
Firm size distribution
Percentage of firms
Num
ber
of e
mpl
oyee
s
![Page 45: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/45.jpg)
45
@HA
Star
k
Discovery-drivennot data-driven
![Page 46: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/46.jpg)
@HA
Star
k
46
How to be discovery-drivenFOCUS ON QUESTIONS THAT are important are interesting are open, not closed
although may need to be operationalised as yes/no
connect to concerns and curiosities about your design, and how people use it, and how it will evolve
emerge from answers
SEEK ANSWERS THAT
are actionable but also
lead to more questions
![Page 47: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/47.jpg)
@HA
Star
k
47
There’s no shortage of theories
Image of Sigmund Freud, Library of Congress, Prints & Photographs Division, Sigmund Freud Collection, LC-DIG-ppmsca-23761.
Fogg Behaviour model reproduced with permission from B J Fogg http://www.behaviormodel.org/index_files/pasted-graphic.jpgImage of Amos Tversky, Stanford News Network, http://psycnet.apa.org/journals/amp/58/9/images/amp_58_9_723_fig1a.jpg
Csikszentmihalyi Flow model, http://upload.wikimedia.org/wikipedia/commons/thumb/f/f6/Challenge_vs_skill.svg/300px-Challenge_vs_skill.svg.png
Depthpsychology
Behaviouraleconomics
+ “personality”
Csikszentmihalyi Flow model
![Page 48: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/48.jpg)
@HA
Star
k
48
Games have lots, too
Chris Batemanbrainhex.comTadgh Kelly
http://www.whatgamesare.com/2011/12/the-four-lenses-of-game-making.html
Game types Player types
![Page 49: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/49.jpg)
@HA
Star
k
49
Think
Look
Change
Core mechanic: design quest
A question bigger than itself
![Page 50: Discovery-driven design in social games: techniques, processes, and problems](https://reader034.vdocuments.us/reader034/viewer/2022051518/56815d0e550346895dcb0ed9/html5/thumbnails/50.jpg)
50
@HA
Star
k
There’s lots to do!...and it’s very engaging