Building a Company-Wide Culture of Experimentation

May 18, 2016Founder & CEO GrowthHackers

Sept 2008: Dropbox Blank Slate

• Excellent product coming out of Beta

• Team of 7 engineers

• Skeptical of marketing

• Goal: Create culture of growth experimentation

#ConvCon

Seeds of Testing Culture

• Built tracking and testing infrastructure

• Shared insights (qual and quant) with team

• Shared testing wins with team

• Encouraged broader participation

#ConvCon

Today Dropbox has ½ Billion Users

• No marketer from April 2009 – January 2010

• Product/engineers continued to experiment

#ConvCon

Growth is Company Wide Effort

Product

External Channels

Promise

1st Experience

Ongoing Experience

Acquisition

Activation

Revenue

Retention

Referral

Marketing PR

SupportSuccess

biz dev

CRO or Growth Team

#ConvCon

AKA Growth Hacking

Rapid experimentation across the full funnel to learn the most effective ways

to scale sustainable customer adoption.

#ConvCon

Testing Drives Growth

• No certainty about what works

• Rapid testing across all levers

• More testing = More learning

AcquisitionReferralActivationRevenueRetentionResurrection

#ConvCon

Twitter Example

#ConvCon

Change Harder Established Company

• Day onsite at Microsoft (May 2016)

• Goal: Transform to culture of growth experimentation

• Founded 1975, 118K employees

• Results: TBD

#ConvCon

Steps to Experimentation Culture

#ConvCon

Process First

#ConvCon

Effective Process

#ConvCon

Core Team to Manage Process

• Start with product manager

• Add dedicated people to core as needed to hit testing tempo goals

• Core often includes designers, growth engineers, analysts…

#ConvCon

Effective Process

#ConvCon

Goal: Build Large Backlog of Ideas

• Build large backlog of ideas

#ConvCon

Prioritize Backlog

#ConvCon

Select Tests in Weekly Meeting

• Use system for prioritization (we use ICE)• Choose tests to launch (no brainstorming)

IMPACT CONFIDENCE EASE (1 – 10) (1 – 10)

(1 – 10)

#ConvCon

Launch Tests

#ConvCon

Launch Tests

• Assign a project manager to each test• Use tools and allocate skills needed to implement

#ConvCon

Analyze Results

#ConvCon

Analyze Results

• Ensure tracking up front for hypothesis

• Run sufficient time

• Triage analysis by hypothesis first

#ConvCon

Example Weekly Meeting Agenda

15 min: KPI review & update focus area

10 min: Review last week’s testing sprint

15 min: Key lessons learned from analyzed tests

15 min: Select tests for this week’s sprint

5 min: Check growth of idea backlog

#ConvCon

Requires Transparent Learning

#ConvCon

Transparent Learning

• Every test leads to learning• Proactively share with idea originator• Share wins with opt in email list• Organize learning for everyone’s easy access

#ConvCon

Drive Broad Participation

#ConvCon

Company Kickoff

• CEO: “everyone seek opportunities for improvement and growth”

• Share recent wins and insights

• Idea input training

• Pizza and beer brainstorm

#ConvCon

Collect Ideas Widely

External Experts

Internal Experts

Ideas Tests Results

Core Growth Team Drives Process

#ConvCon

Standard Experiment Doc for Ideas

• Include research, hypothesis, target lever

#ConvCon

Provide Feedback on Ideas

• Acknowledge all ideas

• Praise favorite ideas

• Notify if selected

• Report back results

#ConvCon

Keep it Fun

#ConvCon

Idea from Engineer - oEmbed

• Data: Embeds = longer sessions

• Hypothesis: Same action = more embeds

• Result: 41% increase in embedded media

Slides

Videos

#ConvCon

Idea from Analyst: To-do-list

• Data: low % of newly registered users commenting in 1st week

• Hypothesis: Increase % of first week commenters

• Test: Use a To-do-list to guide new users (start with easy activities)

#ConvCon

Result: To-do-list

• Result: over 700% lift in 1st week commenting

#ConvCon

Key Takeaways

• Share learning & celebrate wins

• Empower full team to generate ideas

• Ensure core team has process to manage inputs

#ConvCon

Join GH Projects Private Betaat Projects.GrowthHackers.com

Where Teams Drive Results with High Tempo Testing:

1) Team Ideation 2) Prioritize Testing 3) Capture Learning

#ConvCon