3 COMMON MISTAKES WHEN LOOKING AT FREEMIUM

METRICS

Sunday, 3 February 13

INTRODUCTION

• Product Lead at NaturalMotion

• Previously worked on:

stan.beremski@naturalmotion.com

Sunday, 3 February 13

OVERVIEW OF FREEMIUM GAMES

Sunday, 3 February 13

THE FREEMIUM MODEL

App Store

Engage

Retain

Monetize + Invite

Sunday, 3 February 13

Good metrics give you insight into conversion at each stage of the model

App Store

Engage

Retain

Monetise + Invite

Good metrics have explanatory powers

Sunday, 3 February 13

MISTAKE #1

NOT FOCUSING ON EXPLANATORY METRICS

Sunday, 3 February 13

• Total Revenue / Total Users• Session per player / day• Invites sent / player• Invites accepted / Invite• ARPDAU• DAU/MAU

Vanity Explanatory

• Users• Page Views• Daily Revenue• Total Mins of Play• Total Sessions

• Cohort segmentation• Retention• LTV• Sessions pp / day

Counts Ratios

• Behavioural segmentation• Whales vs Free• Single player vs Multi

• Funnels• Tutorial / Quest• Virality funnel

Segmented Ratios

Sunday, 3 February 13

MISTAKE #2

NOT CONTROLLING FOR VARIABLES

Sunday, 3 February 13

You have just released an update to your game and you take a look at the metrics to gauge success...

• Did this update improve monetisation?

• How long did it take to us get a conclusive answer?

(Data is fictional)

Sunday, 3 February 13

There is a problem if you only look at New Users, Revenue and ARPDAU to gauge the success of an update...

Solution: Look at metrics which isolate what you are interested in and control for other variables (in this case the player’s lifecycle within the app)

Problem: Metrics can be affected by uncontrolled variables• In our fictional game players spend 90% of their LTV within 7 days of first playing the app

• An large influx of new players will cause a revenue spike even if the app remains unchanged

(Data is fictional)

Sunday, 3 February 13

The solution is to look at player LTV at specific points in the player’s lifecycle

LTV = Total Revenue from cohort

Total # Player of a cohortDay N LTV = Player LTV N days into

the lifetime of a cohort

(Data is fictional)

Sunday, 3 February 13

MISTAKE #3

NOT LOOKING AT THE DISTRIBUTION OF

UNDERLYING DATA

Sunday, 3 February 13

We often look at averaged data.

Mean = 3.89Median = 3Mode = 3

{1,2,3,3,3,4,4,5,10}

Sunday, 3 February 13

Power-law distribution are common in freemium games

Max: $624Min: $0Mean: $0.57Median: $0Mode: $0Std Dev: 7.12% Payers: 5%

Represents $65,000 revenue from110,000 players

(Data is fictional)

Sunday, 3 February 13

But statistics can be misleading...

Property Value

Mean  of  x  in  each  case 9  (exact)Variance  of  x  in  each  case 11  (exact)Mean  of  y  in  each  case 7.50  (to  2  decimal  places)Variance  of  y  in  each  case 4.122  or  4.127  (to  3  decimal  places)CorrelaAon  between  x  and  y  in  each  case 0.816  (to  3  decimal  places)Linear  regression  line  in  each  case y  =  3.00  +  0.500x  (to  2  and  3  decimal  places,  respecAvely)

Sunday, 3 February 13