using data to guide product development
TRANSCRIPT
Using data to guide product development
Mat Clayton Co-Founder
What is Mixcloud?
The Plan
The Plan
What does failure look like?
turntable.fm - MAU
Text
turntable.fm - DAU
Competitor analysis
• Sitemaps are an amazing resource
• Google - “site:competitor.com“
• OpenSiteExplorer, can be used to find links and any growth tactics being used
• graph.facebook.com/<app_id> can give you monthly Facebook actives for FB connected apps
• API’s
Key Performance Indicators (KPI’s)
Pick a single KPI
• Easily understood
• Provides focus and align the team
• Represent the core of the product, and usually isn’t uniques
Mixcloud - Listener minutes / 28 daysAirBnb - Nights stayed / 28 daysEventbrite - Tickets Booked / 28 daysSendGrid - Mail Sent / 28 days
Dashboards
• Distribute information amongst the team
• Make trends observable
• Give the team a solid understanding in top level metrics
• We’re big fans of graphite and grafana
• Alert on changes
User Accounting
User accountingSite A Site B
Active Users 1000 1000
Born +800 +100
Die -100 0
Sleep -300 -50
Awaken +100 +450
Next Period’s Active Users 1500 1500
Retention Curves
Identifying key product features for retention
The Brainstorm
What correlates with retention?
First week Listens
Key User Metrics - Twitter
Twitter’s data
Correlation does not imply Causation
Cause or effect?
How do you choose what to build?
What facts do we know about our users?• What information did they give us? name, email, Facebook?
• Do they use mobile web/desktop/iOS/Android?
• Do they have email notifications enabled?
• Which core features have they used and to what degree?
• Which features did they miss?
• What was their last interaction with the service?
• Where did they come from? (Referrer)
• Do they use competitor products?
Evaluating potential product features
• Do we have the resources to build it?
• How many users will the feature be relevant to?
• How frequently will they interact with it?
• What is the conversion rate against our primary goal?
AB Testing (Always Be Testing)
AB Testing Examples
Big changes == Big results
How big is big?
AB testing• Continually run AATests
• Requirements
- Fast to implement, a single line of code
- Negligible cost to product performance, including SEO
- Results need to be available to the entire team
• Tools
- Django Experiments (we wrote it)
- A/Bingo (Ruby on Rails)
- Optimizely
- Google Website Optimizer
Failure is acceptable
“A good plan, violently executed now, is better than a perfect plan next week.” - George S. Patton
