jordan christensen at analytics that excite

Effective Data Driven Organizations

Jordan Christensen Head of Data Wattpad

Jordan Christensen Head of Data Wattpad

Canada, Eh?

Joined Wattpad in Sept 2014 Before that, founding team member at Kobo

Only real world-wide competitor to Amazon Kindle

Built Product and Data

Also consulted with other orgs on how to build their Data Orgs

⃠

Effective Data Driven Organizations

Warning!

?

X

Also:

I made most of this up. From my experience.

Working with small to medium sized startups.

Lots of info out on the web. HBR, and a few books

Finally, this isn’t a prescription. Tools and Techniques

We let people discover and share stories about the things they love

A new storytelling experience that is mobile and social

Last month 40 million people spent 13 billion minutes on Wattpad

More than 150 million stories have been shared on Wattpad

Wattpad stories are available in over 50 languages

After 3 39 Parts, Ongoing

2.6M READS

27.5K VOTES

4089 COMMENTS

MOBILE Optimized for mobile

consumption

PERSONALIZED Via social & machine

PARTICIPATORY Create story worlds of fan

content on/off Wattpad

SERIAL Maintain an ongoing connection

with readers

ACTIVE Elevate conversations and interactions

in the story

How Wattpad works for today’s readers

Desktop

Android

Wattpad continues to experience exponential growth

Monthly Time Spent

Wattpad pushed the boundaries of

social and mobile entertainment

with over 1 billion chapter reads and 6 million comments

Effective Data Driven Organizations

But Jordan,

Aren’t all modern Organizations data driven?

Sort of….

Some Qualities of Data Driven Organizations

But we can do better

Effective Data Driven Organizations

Okay Jordan,

How do we build an Effective Data Driven Organization?

Simple! Focus on these THREE things

1. Culture 2. Models 3. Science

Data Culture

Every Effective Data Driven Org has a strong Data Driven Culture

Everyone shares in culture

Start …

Start with Coffee

Start with Coffee

Daily Coffee Email

At Wattpad, everyone gets an email in their Inbox (or on Slack) with the Daily Numbers

Arrives in time for Coffee (before 8am)

Not an overload Top Line Up or Down

Aside: Find your North Star

One Metric that Matters (OMTM)

MAU

Monthly Active Users

Quarterly Subscribers

Starts discussions

Why is iOS up so much today? Why is France slower than last week?

Most Importantly

Most Importantly

Public

Transparent

Next simple thing after Coffee Emails…

Dashboards

* Not Real

Why do Dashboards matter?

Public Transparent AND….

Self-Serve

Your Data team, if you have one, is small

You likely have more asks than you can possibly ever answer

Self-Serve is the only effective answer here

Push queries to the edges of the org

Aside: Where is your data?

Aside: Where is your data? Many organizations start not owning their dataThen they start collecting it

Now they have two problems

As your organization grows, you will outgrow hosted tools

They will get expensive, and their capabilities will be limiting

Plan ahead - you’ll need to build your own ‘Data Lake’.

But Jordan, How do people know how to self serve?

Office Hours

To make Self-Serve work, people need to know how to do it

Borrowed from Kickstarter

One Hour, Once a Week

Drop In, ask questions, get trained

Anyone

Get’s full? Take a number, come back next week

But it’s always there

Data Models

Analytics sometimes feels like being a Weather person

People ask you about something (The Weather), and you tell them (It’s Cold*)

Ultimately, that isn’t that helpful.

Weather people need to tell you what the Weather will be like tomorrow.

Or Why is it going to be Cold tomorrow?

Or what can they do to change The Weather?*

* May be limited to Super Villains

We’ve been predicting the Weather* for centuries

How do we do it?

Models

But Jordan, Why do I need these complicated models?

First, I didn’t say they had to be complicated

Second, so we know where we are going

Targets

Effective Data Driven Organizations use Models to set their Targets

1. Quarterly Goals 2. Yearly Goals 3. Multi-Year Goals

Targets

Without Models, Targets have another name

Guesses!

Aside on Targets: You’ll get them wrong and you’ll miss them (or exceed them)*

Learning how to model your business and set targets takes time

But get started, track variance and be Open and Transparent about them

Maybe put them in the Coffee Email, everyday, so people know how they are doing

Targets are in some way aspirational. They are where you WANT to be.

If we want to know where we think we’ll be, we need…

Forecasts

Just like the Weather, shorter term these are, the more accurate

Daily - Did it rain yesterday? It’ll probably rain tomorrow!

Weekly - Did it rain yesterday? Who knows what next week is like

Short Term barometer

Is this week above or below the Forecast?

Best indication of short term performance.

Proxies

Data Science lives here Finding Short Term correlations for Longer Term metrics

Example: MAU is SLOW

30 Day Moving average, would take about twice that long to see any real change

How do we know if MAU is going to get better or worse?

We need a Proxy.

At Wattpad, we looked at what our users were doing and found something magical…

People who Read More, Stay Longer

Reading Time vs Retention

If we increase Reading Time, we increase Retention, and that leads to more MAU

These Proxies do two things for us: 1. They let us see results quickly 2. They point us towards things we can effect

Finding Proxies is great, but how do you change things, and how do you know you’ve made things better?

“Do” Science

You’ve probably seen this book.

Random Walk experiments

Experts can’t tell the difference between randomness and causation

Looking at Charts (in, say, GA) is the same thing

Human’s have this natural need to find order in noise

It has served us well for thousands of years, how do we change?

A/B Test

Used to be hard, how we have great tools.

Sometimes you need to go deeper. You might need to build this, or there are other toolkits to leverage

We built this at Wattpad, most larger orgs do as it needs to be very close to your data.

Wattpad is about 120 staff, but we’re running > 20 experiments at any time

Causation

Learn what you can change, and why it works, or why it doesn’t.

At Wattpad, we were changing our on boarding experience.

Nothing Happened.

Tried again.

Nothing Happened.

Turned it off all together

Nothing Happened?! WTF?

Turns out our On Boarding didn’t do anything since it didn’t change the experience

Connected to the experience, we increased our retention by 10%

Council

Experiments are hard. You need a council.

Borrowed from a Google White Paper on how they run their process

Provide Governance, Best Practices, and share results

Learnings

Best part of Science

You Learn!

You learn what your users want, what they don’t, what makes them happy, what makes them sad

And you can tell the rest of your company. Science is a process by which you learn and improve your business

Ultimately, that’s why we are here - to use Data to improve our businesses

And you’re done!

In Summary

1. Culture 2. Models 3. Science

1. Email 2. Dashboard 3. Self Serve 4. Office Hours

Culture

1. Targets 2. Forecasts 3. Proxies

Models

1. A/B Test 2. Causation 3. Council 4. Learnings

Science

* Still Continuing on our Journey at Wattpad Working every day to make Data more of our day-to-day decision process

Thank Youjc@wattpad.com

@thebigjc