Download - Lean analytics for startups - Leweb2010
Lean analytics for startups
Wednesday, December 8, 2010
@acroll(if you have questions)
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Lean Not lean
Wednesday, December 8, 2010
Ries, Mclure, and Blankare often misquoted.
Wednesday, December 8, 2010
They never said “fail faster.”
Wednesday, December 8, 2010
Instead:“Learn and adapt.”
Wednesday, December 8, 2010
Waterfall, agile, and leanThree approaches for three situations.
Wednesday, December 8, 2010
Waterfall approachYou know the problem and the solution.
Wednesday, December 8, 2010
Known set of requirements
Known ways to satisfy them
Spec Build Test Launch
Wednesday, December 8, 2010
Agile methodologiesKnow the problem, find the solution
Wednesday, December 8, 2010
Known set of requirements
Unclear how to satisfy them
Build Test LaunchViable?Problem
statement
Adjust
Sprints
Unknown set of requirements
Wednesday, December 8, 2010
Most startupsDon’t even know what problem they solve.
Wednesday, December 8, 2010
Possible problem space
Possible viable
offeringTrial startup
Possible viable
offeringTrial startup
Possible viable offering
Trial startup
Possible viable offering
Trial startup
You are here
PIVOT
Wednesday, December 8, 2010
All startups are lean at first.Which has some interesting consequences.
Wednesday, December 8, 2010
Getting features right matters less.
Knowing what’s not working, quickly, matters more.
Wednesday, December 8, 2010
“We should” matters less.
“What if” matters more.
Wednesday, December 8, 2010
Your roadmap matters less.
Parallel experiments matter more.
Wednesday, December 8, 2010
User groups and surveys matter less.
Real-world adoption & usage analysis matters more.
Wednesday, December 8, 2010
Business plans matter less.
Business models matter more.
Wednesday, December 8, 2010
http://www.flickr.com/photos/tinfoilraccoon/197640807/Wednesday, December 8, 2010
Cost
Revenue Lemonade( ) x Units sold + tips
Lemon + cups + sugar( ) x Units made
Markers + cardboard( ) x Signs made
Staffing( ) x Hours worked
Profit!Wednesday, December 8, 2010
Wednesday, December 8, 2010
http://www.flickr.com/photos/ethanhein/3207831470/#/
(maybe not that fun a game)
Wednesday, December 8, 2010
Clouds and analyticsIt’s never been a better time to start a company.
Wednesday, December 8, 2010
Clouds make experimenting easy, avoid upfront dilution.
Analytics make experimenting smart & fast.
Wednesday, December 8, 2010
http://www.flickr.com/photos/ecastro/3053916892/
I love to talk about clouds(but not today)
Wednesday, December 8, 2010
I’m going to focus on the other partWhat to watch on your website, if you’re a startup.
Wednesday, December 8, 2010
So what could we watch?
Wednesday, December 8, 2010
Analytics(what did they
do on the site?)
Competition(what are they
up to?)
Performability(could they do
what they wanted to?)
VoC(what were
their motivations?)
Usability(how did they interact with
it?)
Community(what were
they saying?)
“Hard” data
“Soft” data
Complete Web Monitoring
Wednesday, December 8, 2010
http://www.flickr.com/photos/itsgreg/446061432/
Analytics is the measurement of movement towards your business goals.
Wednesday, December 8, 2010
NEWVISITORSGROWTH
BOUNCERATE
LOSS
CONVERSIONRATE
GOALVALUE
xTIME
ONSITE
PAGESPER
VISIT
NUMBEROF VISITS
SEARCHES
TWEETS
MENTIONS
ADS SEEN
ATTENTION ENGAGEMENT CONVERSION
Wednesday, December 8, 2010
Conversions
Visits
Shopping cart
Payment options
Wednesday, December 8, 2010
Conversions
Visits
Shopping cart
Payment options
KPIsC
onversionrateBounce
rate
Wednesday, December 8, 2010
http
://w
ww
.flic
kr.c
om/p
hoto
s/m
rmoo
rey/
1606
5423
6
Wednesday, December 8, 2010
http://www.flickr.com/photos/duncan/1252272164/Wednesday, December 8, 2010
http://www.flickr.com/photos/intherough/3573333256/Wednesday, December 8, 2010
http://www.flickr.com/photos/jetheriot/648950773/Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Impact of page load time on average daily searches per user
-0.60%
-0.45%
-0.30%
-0.15%
0%
50ms pre-header 100ms pre-header 200ms post-header 200ms post-ad 400ms post-header
Wednesday, December 8, 2010
Wednesday, December 8, 2010
Impact of additional delay on business metrics
-5.00%
-3.75%
-2.50%
-1.25%
0%
50 200 500 1000 2000Queries/visitor Query refinement Revenue/visitorAny clicks Satisfaction
Wednesday, December 8, 2010
Shopzilla’s makeover
Wednesday, December 8, 2010
5-12%Wednesday, December 8, 2010
Wednesday, December 8, 2010
Traffic levels
0
2,250
4,500
6,750
9,000
Optimized Unoptimized
4,740
8,505
Tota
l num
ber o
f visi
ts
Visitor experience
Wednesday, December 8, 2010
Bounce rate
0
5
10
15
20
Optimized Unoptimized
14.35%13.38%
Visit
s tha
t bou
nced
Visitor experience
Wednesday, December 8, 2010
Average time on site
0
8
16
23
31
Optimized Unoptimized
23.8330.17
Time o
n site
(min
utes
)
Visitor experience
Wednesday, December 8, 2010
Pages per visit
0
4
8
12
16
Optimized Unoptimized
11.04
15.64
Aver
age p
ages
seen
Visitor experience
Wednesday, December 8, 2010
Conversion rateand order value
0
5
10
15
20
Conversion rate Order value
5.51
16.07
Diffe
renc
e due
to op
timiza
tion
Wednesday, December 8, 2010
Community management
Hugging
Bunnies
Retweets
Friends!
Wednesday, December 8, 2010
Conversions
Visits
Shopping cart
Payment options
Unpaid search Communitymentions
Emailcampaign Banner ad
• Google PageRank• Sessions-to-clicks
ratio
• Cost of ads(CPM)
• Clickthroughrate
• Open rate• Opt-out rate?
Wednesday, December 8, 2010
But there’s a danger:vanity metrics.
Wednesday, December 8, 2010
Wednesday, December 8, 2010
So what should you watch?The lean analytics shortlist.
Wednesday, December 8, 2010
In a startup, the purpose of analytics is to iterate to a product/market fit before the money runs out.
If you remember only one thing...
Wednesday, December 8, 2010
Message reach
Infrastructure health
Market sentiment
User engagement
Business model
Sustainability
Viral coefficient
Amplification
Performability
Customer research
Community sentiment
Time since last visit
Core goals
Extended funnel abandonment
Movement towards goals
Cost per engaged visitor
Peak to average ratio
Minimum sustainable burn
Wednesday, December 8, 2010
Viral coefficientOn average, for each message they hear, how many people does someone tell?
1Wednesday, December 8, 2010
------------------------------------------------------Get your free private email at http://www.hotmail.com------------------------------------------------------
Wednesday, December 8, 2010
!Wednesday, December 8, 2010
v ≠ 1, pt = δp0 (1 – vt+1) / (1 – v) + p0
http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/Wednesday, December 8, 2010
How well are messages being amplified?
2Wednesday, December 8, 2010
50% 75% 100% 50% = 4
2 3 1 = 6
14.5
Downstream reach (aggregate followers)
Amplification chance(how relevant the content
is + what the recipient thinks of the sender)
25%
.5
50%
2
50%
1 = 3.5
Wednesday, December 8, 2010
Repurposing(spread to othercommunities)
Amplification(virality andmessage spread)
Conversions
Visits
Shopping cart
Payment options
Seed (startingcommunity)
Reach (impressions)
Wednesday, December 8, 2010
Viral message spread
Conversions
Visits
Shopping cart
Payment options
Reach (impressions) Emphasis on
getting viral ratio above 1(Retweeting, Fan, Email
forward, Reddit upvote, other
loops)
Wednesday, December 8, 2010
Megablogger proponents
Conversions
Visits
Shopping cart
Payment options
Seed (startingcommunity)
Reach (impressions)
Emphasis on convincing highly-followed, highly
acted-upon seed group to spread
the word.
Wednesday, December 8, 2010
A call to action
Conversions
Visits
Shopping cart
Payment options
Reach (impressions)
Emphasis on maximizing
impression-to-click ratio within the community
Wednesday, December 8, 2010
Goal attainment: funnel
700,000s
1,642
150RT
327
1015
x $100x $20
x $7
20Seed ratio:
35,000:1
Followers
Visitors:0.23%
Conversions:1.95%
Revenues:Average: $39.54Median: $20
Total: $1,005
Amplification: 2.9%
2 Repurposing
2,000
Wednesday, December 8, 2010
How performable are we?Health of the infrastructure (uptime, latency)
3Wednesday, December 8, 2010
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
0
5,000
Uniq
ue p
age
view
s
>4s or error
2-4s
<2s
Conversions
$0
$10,000
Revenue (total sales)
Wednesday, December 8, 2010
What do visitors think of us?Both on the site (VoC) and in the world (community monitoring)
4&5Wednesday, December 8, 2010
Wednesday, December 8, 2010
Wednesday, December 8, 2010
How engaged are users?Time since last visit, as a histogram, not an average, compared to baselines.
6Wednesday, December 8, 2010
Days since last visit
1200
1000
800
600
400
200
01 2 3 4 5 6 7 8 9
Days since last engagement
January February
Disengaged(>10 days)
25000
20000
15000
10000
5000
0
Num
ber o
f use
rs
Wednesday, December 8, 2010
Core goals driving the businessPick the 3-4 tasks you’ve set for users. Always know what these are and why they correlate to business growth.
7Wednesday, December 8, 2010
Cost
Revenue Lemonade( ) x Units sold + tips
Lemon + cups + sugar( ) x Units made
Markers + cardboard( ) x Signs made
Staffing( ) x Hours worked
Profit!
(Remember the model?)
Wednesday, December 8, 2010
Extended funnel abandonmentNot just on the site: Email open rate
8Wednesday, December 8, 2010
Wednesday, December 8, 2010
In the last build, what moved us towards & away from goals?Requires knowing the goals and the key features added
9Wednesday, December 8, 2010
Cost per visitorOperations costs per site visitor
10Wednesday, December 8, 2010
(this had better be greater than the revenues and conversion rate.)
Wednesday, December 8, 2010
Peak-to-average ratioHow much busier are you at busy times?
What premium do you pay for that surplus?
Can you decommission well?
11Wednesday, December 8, 2010
http://community.microsoftadvertising.comWednesday, December 8, 2010
Wednesday, December 8, 2010
Minimum sustainable burn
12Wednesday, December 8, 2010
http://www.flickr.com/photos/dougww/833127501/Wednesday, December 8, 2010
Message reach
Infrastructure health
Market sentiment
User engagement
Business model
Sustainability
Viral coefficient
Amplification
Performability
Customer research
Community sentiment
Time since last visit
Core goals
Extended funnel abandonment
Movement towards goals
Cost per engaged visitor
Peak to average ratio
Minimum sustainable burn
12thingsto
watch
Wednesday, December 8, 2010
@acrollwww.yearonelabs.com
www.rpmmtl.com
www.watchingwebsites.com
www.bitcurrent.com
www.human20.com
Wednesday, December 8, 2010