using apis for data science, justin johnson keen io
DESCRIPTION
"Using APIs for Data Science" will touch on the importance of analyzing more than page views, and how providing your employees and customers with deep analytics can benefit your business. Analytics doesn't have to be a big, hairy, and complicated ordeal. Using APIs can simplify the process of collecting, storing, and analyzing data - allowing all the complicated backend work to be done for you while providing the answers and insights needed to make better product and business decisions. Justin Johnson will give an overview to how APIs can be used for data science and how some of Keen IO's customers have used Keen's data collection and analysis APIs.TRANSCRIPT
Data Science via API
Presented by Justin @ All Things API on 9/23/15
Analytics is the discovery and communication of meaningful patterns in data.
•Some context on big data & analytics
•What is the goal of your app?
•Event data
•Common analytics methods
•Analyze some data
AGENDA
SOME CONTEXT ON BIG DATA AND ANALYTICS
Every company is becoming a software company. Every software company is becoming a data company.
Big Data and Analytics are kind of a thing right now.
COOL DATA STORIES
Johannes KeplerTycho Brahe
John Snow figured out Cholera spreads through water. !No one believed him :(
APPLYING ANALYTICS TO YOUR BUSINESS
Use analytics to measure progress toward a goal.
Use analytics to test new hypotheses.
Use analytics to explore.
WHAT IS THE GOAL OF YOUR APP? !Examples: !Vine: reach 1M user-generated videos. !Spotify: increase conversions to paying subscriptions
EXERCISE 1! 1 MINUTE
• Account creations
• Deploys
• Purchases
• App Launches
• Views
• Posts
• Shares/Tweets/Likes
A COMMON GOAL: ENGAGEMENT
INTRODUCING EVENT DATA
UID twitter handle age Account ID
773345 @hipsterhacker 29 443556
773346 @TNG_S8 27 432354
773347 @modernseinfeld 28 336658
773348 @elof 30 2115789
{ "event": "death", "timestamp": "2013-05-23T1:50:00-0600", "cause": "creeper explosion", "enemy": { "type": "creeper", "power": .887, "distance_from_player": 3.43, "age": .6677, }, "player": { "UID": "99234890823", "experience": 8873729, "age": 338, "inventory": [“diamond sword”, “torches”] } }
entity data event datastrict schema flexible schema
normalized denormalized
shorter wider
describes nouns describes verbs
describes now describes trends over time
updates appends
big data big big big data
ACTIONS: Signup, Login, Upgrade, Submit, Scroll, Send, Share, Search, Check-In, Vote, Update, Purchase, Level Up, Fail, Favorite, Vote, Crash, Rate, Start, Modify, Check, View, Capture
STATE INFO: User, Company, Organization, Team, Platform, Device, App, Level, Garden, Favorites, Interests, Inventory, Cart, Video, Location, Item, Record, Product, Account, Form, Picture, Story
MORE EXAMPLES
ELEVATOR PITCH TECHNIQUE
• Describe your app to a stranger and listen to the words you use.
• Verbs are the actions you should record.
• Nouns are the important contextual information you should include in your data model.
• Most apps can be very robustly described by 5-10 key events and 5-10 key nouns.
ANALYTICS DBCARS, TVs, ETC.
WEBSITES WEBSITES
CUSTOMERS
DASHBOARDSMOBILE APPS
queries
queries
queries
events
events
events
Recall your goal from the previous exercise. !Think at least one event you can track to measure your progress toward that goal.
EXERCISE 2! 2 MINUTES
!Get into groups of 4 and come up with a simple app idea or use an existing app as an example. !What are five key event’s that are important to track? !What properties that you would be useful to track for those events?
EXERCISE 3! 2 MINUTES
COMMON ANALYTICS TECHNIQUES
COUNTING!99% of analytics work involves what mathematical operation?
MORE BASICS
• Count Unique • Sum • Average • Min • Max
ADVANCED
• Statistical Analysis • Correlation Analysis • Predictive Analysis
Fancy Terms for Counting Stuff
DAU/MAU
EXAMPLEWhat was the average revenue per active user last month?
1. Count the number of unique users who performed some action in June (2300)
2. Sum all of the purchases from June ($5564)
3. Divide 2 by 1 ($2.40)
•Sorting data into buckets. Commonly used to sort users into groups.
•Examples: Gender, Age, Location, Department, Referrer, Version, Device
SEGMENTATION
•Used to reduce the data set to which a query applies
•Use any of your event properties to do filtering.
•Example: Count the number of purchases events where item.category = “add-ons” and item.price > 100.
FILTERING
21% more people clicked on the red button than on the green button!
A/B TESTING AKA SPLIT TESTING
Which version of the form was more effective?
EXAMPLE OF SPLIT TESTING DATA
opportunity!
LAUNCHED APP (720)
STARTED RECORDING (548)
FINISHED RECORDING (269)
UPLOADED (350)
FUNNELS
A cohort is a group of people who share a common characteristic over a certain period of time.
AVERAGE INCOME FOR THE GRADUATION CLASSES OF 2010 VS 2011
20K
25K
30K
35K
40K
45K
50K
YEAR 1 YEAR 2 YEAR 3 YEAR 4 YEAR 5
20102011
COHORT ANALYSIS
How many customers remain customers?
How many users came back a second time?
Do my customers value my product?
RETENTION
Measure retention by counting how many users did an action X days after their first usage.
RETENTION ANALYSIS BY COHORT
CHURN
How many users are we losing?
!
!
Churn is the total number of users you lost in a given timeframe, divided by the total number of users you had at the beginning of the timeframe.
CHURN
Churn impacts growth & profits significantly.
CUSTOMER LIFETIME VALUE(CLV, CLTV, LCV, LTV)
How much is a customer worth?
Monthly Revenue x Margin x Number of
MonthsCLV =
$100/mo x 25% x 10 monthsCLV = = $250
How much did it cost to get that user? !
CAC = $ spent / number of users acquired !Include amount invested in marketing, advertising, and sales. !
Customer Acquisition Cost
