jordan christensen at analytics that excite
TRANSCRIPT
Effective Data Driven Organizations
Jordan Christensen Head of Data Wattpad
Jordan Christensen Head of Data Wattpad
Canada, Eh?
Joined Wattpad in Sept 2014 Before that, founding team member at Kobo
Only real world-wide competitor to Amazon Kindle
Built Product and Data
Also consulted with other orgs on how to build their Data Orgs
⃠
Effective Data Driven Organizations
Warning!
?
X
Also:
I made most of this up. From my experience.
Working with small to medium sized startups.
Lots of info out on the web. HBR, and a few books
Finally, this isn’t a prescription. Tools and Techniques
We let people discover and share stories about the things they love
A new storytelling experience that is mobile and social
Last month 40 million people spent 13 billion minutes on Wattpad
More than 150 million stories have been shared on Wattpad
Wattpad stories are available in over 50 languages
After 3 39 Parts, Ongoing
2.6M READS
27.5K VOTES
4089 COMMENTS
MOBILE Optimized for mobile
consumption
PERSONALIZED Via social & machine
PARTICIPATORY Create story worlds of fan
content on/off Wattpad
SERIAL Maintain an ongoing connection
with readers
ACTIVE Elevate conversations and interactions
in the story
How Wattpad works for today’s readers
Desktop
Android
Wattpad continues to experience exponential growth
Monthly Time Spent
Wattpad pushed the boundaries of
social and mobile entertainment
with over 1 billion chapter reads and 6 million comments
Effective Data Driven Organizations
But Jordan,
Aren’t all modern Organizations data driven?
Sort of….
Some Qualities of Data Driven Organizations
But we can do better
Effective Data Driven Organizations
Okay Jordan,
How do we build an Effective Data Driven Organization?
Simple! Focus on these THREE things
1. Culture 2. Models 3. Science
Data Culture
Every Effective Data Driven Org has a strong Data Driven Culture
Everyone shares in culture
Start …
Start with Coffee
Start with Coffee
Daily Coffee Email
At Wattpad, everyone gets an email in their Inbox (or on Slack) with the Daily Numbers
Arrives in time for Coffee (before 8am)
Not an overload Top Line Up or Down
Aside: Find your North Star
One Metric that Matters (OMTM)
MAU
Monthly Active Users
Quarterly Subscribers
Starts discussions
Why is iOS up so much today? Why is France slower than last week?
Most Importantly
Most Importantly
Public
Transparent
Next simple thing after Coffee Emails…
Dashboards
* Not Real
Why do Dashboards matter?
Public Transparent AND….
Self-Serve
Your Data team, if you have one, is small
You likely have more asks than you can possibly ever answer
Self-Serve is the only effective answer here
Push queries to the edges of the org
Aside: Where is your data?
Aside: Where is your data? Many organizations start not owning their dataThen they start collecting it
Now they have two problems
As your organization grows, you will outgrow hosted tools
They will get expensive, and their capabilities will be limiting
Plan ahead - you’ll need to build your own ‘Data Lake’.
But Jordan, How do people know how to self serve?
Office Hours
To make Self-Serve work, people need to know how to do it
Borrowed from Kickstarter
One Hour, Once a Week
Drop In, ask questions, get trained
Anyone
Get’s full? Take a number, come back next week
But it’s always there
Data Models
Analytics sometimes feels like being a Weather person
People ask you about something (The Weather), and you tell them (It’s Cold*)
Ultimately, that isn’t that helpful.
Weather people need to tell you what the Weather will be like tomorrow.
Or Why is it going to be Cold tomorrow?
Or what can they do to change The Weather?*
* May be limited to Super Villains
We’ve been predicting the Weather* for centuries
How do we do it?
Models
But Jordan, Why do I need these complicated models?
First, I didn’t say they had to be complicated
Second, so we know where we are going
Targets
Effective Data Driven Organizations use Models to set their Targets
1. Quarterly Goals 2. Yearly Goals 3. Multi-Year Goals
Targets
Without Models, Targets have another name
Guesses!
Aside on Targets: You’ll get them wrong and you’ll miss them (or exceed them)*
Learning how to model your business and set targets takes time
But get started, track variance and be Open and Transparent about them
Maybe put them in the Coffee Email, everyday, so people know how they are doing
Targets are in some way aspirational. They are where you WANT to be.
If we want to know where we think we’ll be, we need…
Forecasts
Just like the Weather, shorter term these are, the more accurate
Daily - Did it rain yesterday? It’ll probably rain tomorrow!
Weekly - Did it rain yesterday? Who knows what next week is like
Short Term barometer
Is this week above or below the Forecast?
Best indication of short term performance.
Proxies
Data Science lives here Finding Short Term correlations for Longer Term metrics
Example: MAU is SLOW
30 Day Moving average, would take about twice that long to see any real change
How do we know if MAU is going to get better or worse?
We need a Proxy.
At Wattpad, we looked at what our users were doing and found something magical…
People who Read More, Stay Longer
Reading Time vs Retention
If we increase Reading Time, we increase Retention, and that leads to more MAU
These Proxies do two things for us: 1. They let us see results quickly 2. They point us towards things we can effect
Finding Proxies is great, but how do you change things, and how do you know you’ve made things better?
“Do” Science
You’ve probably seen this book.
Random Walk experiments
Experts can’t tell the difference between randomness and causation
Looking at Charts (in, say, GA) is the same thing
Human’s have this natural need to find order in noise
It has served us well for thousands of years, how do we change?
A/B Test
Used to be hard, how we have great tools.
Sometimes you need to go deeper. You might need to build this, or there are other toolkits to leverage
We built this at Wattpad, most larger orgs do as it needs to be very close to your data.
Wattpad is about 120 staff, but we’re running > 20 experiments at any time
Causation
Learn what you can change, and why it works, or why it doesn’t.
At Wattpad, we were changing our on boarding experience.
Nothing Happened.
Tried again.
Nothing Happened.
Turned it off all together
Nothing Happened?! WTF?
Turns out our On Boarding didn’t do anything since it didn’t change the experience
Connected to the experience, we increased our retention by 10%
Council
Experiments are hard. You need a council.
Borrowed from a Google White Paper on how they run their process
Provide Governance, Best Practices, and share results
Learnings
Best part of Science
You Learn!
You learn what your users want, what they don’t, what makes them happy, what makes them sad
And you can tell the rest of your company. Science is a process by which you learn and improve your business
Ultimately, that’s why we are here - to use Data to improve our businesses
And you’re done!
In Summary
1. Culture 2. Models 3. Science
1. Email 2. Dashboard 3. Self Serve 4. Office Hours
Culture
1. Targets 2. Forecasts 3. Proxies
Models
1. A/B Test 2. Causation 3. Council 4. Learnings
Science
* Still Continuing on our Journey at Wattpad Working every day to make Data more of our day-to-day decision process
