increasing the roi of your beta tests
TRANSCRIPT
‹#›
AGENDA 1. BETA TESTING BASICS
2. EXAMPLE BETA ROI
3. THE TYPICAL ROI OF BETA
4. INCREASING THE ROI OF BETA
5. BEST PRACTICES FOR A HIGHER ROI
6. NEXT STEPS + Q&A
Founder & CEO. We offer software, managed services, pre-qualified testers, and educational resources to help companies improve beta testing ROI.
Automotive
Web
Robotics
Hardware
Accelerator Smart Home Wearables
Enterprise Software
Startup
Gaming B2B
Telecom
Mobile
OU
R C
LIE
NT
S
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
‹#›
A test of an unreleased product by real customers in real environments.
LEVEL SET
WHAT EXACTLY IS BETA TESTING?
‹#›
MANY NAMES Customer Validation (CV) Customer Acceptance Testing (CAT) User Acceptance Testing (UAT) Friendly User Testing (FUT) Field Trials Pre-Release Early Release Enhanced Testing User Testing Customer Testing Target Market Testing In the Wild Testing Beta Testing (we like this one)
WHY IT’S CRITICAL
A beta test validates the total performance of the product in the hands of real customers.
HOW IT WORKS 1. Recruit a pool of candidates from your target market
2. Select qualified candidates as participants
3. Prep and NDA participants
4. Distribute product
5. Solicit feedback
6. Evaluate, distribute, and implement the results
HOW IT’S CHALLENGING 1. Dedicating qualified resources
2. Recruiting appropriate beta testers
3. Maintaining user participation
4. Collecting relevant feedback
5. Evaluating and distributing feedback
WHY IT’S SO PAINFUL
1. It’s probably not your full-time job
2. It typically lacks a well-defined process
3. It frequently has no clear owner and/or contributors
4. You likely lack appropriate tools to help
5. The timing is horrible
6. It involves real customers using a buggy product
7. You’re likely worried about controlling information
8. If you succeed, you’ve got a ton of data to deal with
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
Sales and Costs
Estimated Sales 200k units
Unit Sale Price $100
Unit Cost $50
Beta Unit Cost $200
Incentive Costs $50/user
Product and Market
Release Type New
Product Type Hardware
Target Market Consumer
Competition High Complexity High
Quality Confidence High
Beta Management / Resources Tools Dedicated
Experience Part-Time Wage $40/h
Desired Testers 30
Participation Rate 30% Duration 4 Weeks
EXAMPLE ROI ELEMENTS & ASSUMPTIONS
Key Facts No Beta Basic Beta
Beta Management Hours 0 73
Total Beta Testers 0 100
Participation Rate (%) 0% 30%
Defects Found 0 41
Defects Fixed 0 25
Total Units Sold 190,000 200,000
Customers Requiring Support (%) 38% 36%
Product Return Rate 19% 19%
EXAMPLE BETA RESULTS
Financial Impact No Beta ($) Basic Beta ($)
Total Beta Management Cost - (5,491)
Total Logistics & Production Cost - (25,000)
Total Defect Cost (585,000) (562,250)
Total Support Cost (806,614) (787,510)
Total Return Cost (1,087,242) (1,092,025)
Total Warranty Cost (665,000) (633,333)
Gross Sales Profit 9,500,000 10,000,000
Net Gain - 538,246
TOTAL ROI 946%
FINANCIAL IMPACT
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
1 FOCUS ON GOALS
IDENTIFY YOUR CONSTRAINTS
DETERMINE YOUR OBJECTIVES
MAP YOUR GOALS TO A SCHEDULE
DEFINE YOUR TARGET MARKET
t
5 EMBRACE THE RESULTS
FREQUENCY * SEVERITY = IMPACT
SHARE, BUT AVOID A FIREHOSE
INTEGRATE IF POSSIBLE
CONSIDER FEEDBACK IN CONTEXT
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A