(or, how to avoid drinking your own kool-aid) don’t fear the research thekramer.jpg

(Or, How to Avoid Drinking Your Own Kool-Aid)

Don’t Fear the Don’t Fear the ResearchResearch

thekramer.jpgthekramer.jpg

London,Ink LLC www.londonink.com2

Why don’t we use research to validate assumptions?

• Too expensive.

• Takes too much time.

• Tried it before and results weren’t actionable.

• Tried it before and we didn’t like the results.

Research: Pain, panacea or Research: Pain, panacea or practical tool?practical tool?


Today’s AgendaToday’s Agenda

• What are bad assumptions and where do they come from?

• Crafting your assumptions (before or after the fact).

• Rich, fast and cheap sources of raw data.

• Sticking up for traditional research.


Bob London’s BackgroundBob London’s Background

$150M

$6M

$2M

$75.00

London,Ink LLC

London,Ink LLC

Garden Variety Internet Start-UpGarden Variety

Internet Start-Up


First Rule: Its about First Rule: Its about business, not researchbusiness, not research

• Data points are a means to an end: Growing the business.

• Research should have an ROI (revenue increase and/or operational savings)

• Actionable, relevant.

• Two people should each spend at least 5 hours a week on intelligence, data gathering and analysis.

London,Ink Jargon WatchLondon,Ink Jargon Watch


What are bad What are bad assumptions and where assumptions and where

do they come from?do they come from?


Flawed Assumptions: Flawed Assumptions: They’re everywhere.They’re everywhere.


Welcome to the Bad Business Welcome to the Bad Business Assumption Hall of FameAssumption Hall of Fame

• We have no competition!• Our technology is disruptive…and that’s an advantage for us!• The collect calling market is ripe for competition. (c. 1992)• This $45 billion merger will yield cost-savings, revenue growth

and other synergies. It’s is a win-win-win for us, our customers and investors!

• Brick and mortar retailers are dead.• The world needs a better search engine. (c. 1999) • Consumers will pay $23.95 for monthly dial-up internet service

forever.• We can sell 1M Internet screen phones by 2001. (c. 2000)• We can tinker with our soda formula, then market it as “new”

and sell even more!• The key to our success is to go for market share vs. profitability.

(c. 1999)• People will pay for a monthly broadband content service on top

of their monthly broadband access fee.


Where do bad assumptions Where do bad assumptions come from?come from?

• The “Steinbrenner” (a/k/a cram-down)

• The “Emperor’s new clothes”

• The “Pull” (as in where did you pull that one out of?)

• The “Gartner/Forrester/Yankee said it so it must be true”

• The “I Googled it last year but can’t find it now”

• The “Our investors have already blessed it” (or “Our investors suggested it”)

• The “I found it in a research report…from 1997”

• The “Back of the envelope, placeholder, guesstimate, for planning purposes only assumption that just sort of got accepted”


Impact of bad assumptions: Impact of bad assumptions: Longer MeetingsLonger Meetings

• Missed Objectives

• Credibility

• Morale

• Operational Performance

• Churn

Side-Benefit: Solid data points can shorten endless meetings

Side-Benefit: Solid data points can shorten endless meetings

Crafting Your Crafting Your AssumptionsAssumptions


Triangulation vs. Triangulation vs. strangulationstrangulation

Primary and Secondary Research

First Hand Experience (Lessons Learned)

Competitors, Substitutes & Alternatives

Investors

Advisors

Friends & Family

More Investors

Investors’ Friends & FamilyCab Drivers

Separate fact from opinion. Arm yourself with data points before opinion takes root.

Reasonable assumptions


Market Sizing: Use both top Market Sizing: Use both top down down & bottom up& bottom up methodsmethods

Other13%

ASP13%

Software17%

I n-House57%

Competitor 4

10%

Competitor 3

14%

Competitor 1

14%

Competitor 2

19%

I n-House43%

Add up the known players (market share)

Use reputable segment estimates


The yes personThe questionerThe skepticThe group-thinkerThe advisorThe consultant

The yes personThe questionerThe skepticThe group-thinkerThe advisorThe consultant

How to avoid self-think and How to avoid self-think and group-thinkgroup-think

Put candid outsiders on your board of advisorsAnd tell them you want their input and feedback.

Be a questioner and hire questionersYour diverse team should keep you honest

Always remember: we filter everything based on our own background and experiences

Stop guessing

Be conservative: you have everything to gain


Best PracticesBest Practices

• Don’t start the justification process with the answer.

• Maintain Continuous Curiosity.

• Don’t discount any feedback immediately.

• Stay in your customers’ shoes. And promote others who follow suit.

– Customer name tag on your desk or wall

– Name conference rooms after customers

Rich Sources of DataRich Sources of DataPublic • Internal • Just a Phone Call Away


•Blogosphere

•Clusty.com

•Yahoo Video

•Analysts and reporters

•Patent Databases

•Blogosphere

•Clusty.com

•Yahoo Video

•Analysts and reporters

•Patent Databases

Information ViscosityInformation ViscosityPublic information is cheap, accessible and fast

• What are your competitors’ customers saying about them?

• What are your customers saying about you?

• What are your competitors’ customers saying about them?

• What are your customers saying about you?

“Server-Diving” • yourcompetitor.com/careers• Customer list• Speeches, interviews• What shows are they sponsoring?• What Google Adwords are they

using?

“Server-Diving” • yourcompetitor.com/careers• Customer list• Speeches, interviews• What shows are they sponsoring?• What Google Adwords are they

using?

Research the pain points, i.e. customer churn rate, not just

Company XYZ.

Research the pain points, i.e. customer churn rate, not just

Company XYZ.


In-House Data: In-House Data: Lessons LearnedLessons Learned

• Sales pipeline reports

• Churn/disconnect reports

• Win/Loss

• Bandwidth hogs

• One-offs

• Clicks per FAQ

Failure is data too.


When’s the last time you…

• Asked a customer for a critique of your product or service?

• Took a customer to lunch without trying to sell them something?

Rolodex Research: Friendly, meetings with prospects

Data that’s just a phone call awayData that’s just a phone call away

• “I’m looking for some feedback”

vs.

• “I want to sell you something”

•Everyone loves to be asked their opinion, especially in their area of expertise.

“Day in the life” of your customer.“Day in the life” of your customer.

Traditional Research for Traditional Research for Emerging BusinessesEmerging Businesses


Various types of Various types of researchresearch

• Market Sizing/Growth

• Target Definition/Segmentation

• Product/Pricing

• Customer Experience

• Positioning/Messaging

• Due Dilligence

Best done following secondary research.


Research ROI Case Study Research ROI Case Study #1: : Quantitative Market StudyQuantitative Market Study

• Service revenue growth falling well short of plan over 10 quarters.

• Invested $80,000 and 8 weeks in definitive market study

• Goal to evaluate (not validate) market and offerings.

• Findings:

– No real customer pain point.

– While intuitively valuable to the company, service had no commercial appeal.

• Results:

– EOL’d business costing $3M + $800K Capex annually.

– 12-Month Research ROI: 47:1


Research Case Study Research Case Study #2: : Qualitative TestingQualitative Testing

• MCI Friends & Family Outbound Telemarketing

• 150 live test calls to real consumers

• Management used test scripts

• Results/benefits:

– Gauged and documented various reactions to product

– Refined pitch until it worked

– Provided input to product team

– Built internal confidence


Research Case Study Research Case Study #3: : Simple Business Case AnalysisSimple Business Case Analysis

• e-Commerce Start-Up Serving B-to-C market.

• Revenue Model: Affiliate with e-tail sites and capture a small % of purchase price.

• Gut Assumption: Breakthrough in convenient customer experience plus volume will generate rapid revenue growth.

• Research: Revisited business plan to challenge assumptions.

• Flaw: To become just a $30M business, every U.S. household (90M+) would have to purchase $3,500 worth of goods from this company each year for 5 years.

• Result: Find new job.


Data Sources & ToolsData Sources & Tools

• Purchased lists

• Existing customers

• Your Marketing database

• Quantitative phone or Web

• Focus Groups: $10K per evening


Actionable dataActionable data

• Clear

• Interpretable

• Margin for error

• Risk of implementing

• Top down push


Got it. Now where do I start?Got it. Now where do I start?

1. Plan/Goal

• What are your biggest information gaps or operational challenges?

2. Ownership

• Take it on or assign a clear owner.

3. Scope of Activity

• Is this a one-time push, an ongoing program or both?

4. Must-Do’s

• Sanity check on business plan

• Customer interviews

• Competitive/substitute/alternative value analysis

• Secondary Web research

5. Resources

• Do we need specialized resources or do we have expertise in-house?

Don’t Fear the Don’t Fear the ResearchResearch

• Don’t start the justification process with the answer.

• Don’t discount any feedback immediately.

• Stay in your customers’ shoes. And promote others who follow suit.

• DIY and empower a team member

• Both should spend at least 5 solid hours per week on intelligence and data gathering

Thank Thank You.You.

(or, how to avoid drinking your own kool-aid) don’t fear the research thekramer.jpg

Documents

ink jargon watch slide

accepted slide

impact of bad assumptions

secondary research

flawed assumptions

research reportfrom

traditional research

internet service