Prepared for :Coca-Cola Mobile Innovation WorkshopSeptember 24-26, 2013Shanghai, China A/B Testing for Apps

A/B Testing for AppsHow I Learned to Stop Worrying and Love

Data-Driven Decisions

User Base

...and see which one is driving desirable user behavior.

Publish two different versions of your app...

50% sees version B50% sees version A

Which Version Won?

Version A Version B

Source: WhichTestWon

Version B: 114% Improvement

Version A Version B ✔ ✗

Marketer’s Surprise: ‘FREE’ Loses

Multivariate Testing

• Every screen has X components (ex: Marilyn’s hair)

• For each, we can test Y variations (ex.: Green)

• In total, we have [Y1 x Y2 x Y3] combinations

How Obama Raised $60 Million

Source: Optimizely

Four Button Variations

Six Media Variations

24 Combinations!

And the Winner is...

+40%increase in conversion

rate

2.9 millionadditional donators

$60 millionvalue of additional

donations

Obamalytics

• Original Conversion Rate: 8.3%

• New Conversion Rate: 11.6%

• 10 million signups from New Version would have been 7.12 million signups with the Original Version

• +2.88 million additional signups

• $21 average donation per signup

• Approximately $60 million in additional donations

Testing to Increase Virality

Headline

Call to Action

Workflow

Data-Driven App Marketing

• App marketing and analytics is in early stages

• App stores are opaque, approval processes slow down iteration cycles

• Few options for making decisions• HiPPO: Highest Paid Person’s Opinion• IHOP: In-House Optimization

• Scalable, reliable app measurement tools are emerging

Thank You!

For more information:

Zac Aghion, CEO & Co-Founderzac@splitforce.com

China: (+86)1592-1631-924USA: (+1)617-750-6684

www.splitforce.com