Download - Product Testing and Refinement
![Page 2: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/2.jpg)
What is the name of this cookie?
![Page 3: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/3.jpg)
Samoas or Caramel deLites?
![Page 4: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/4.jpg)
What should we name our product?
![Page 5: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/5.jpg)
The Scientific Method
![Page 6: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/6.jpg)
The Scientific Method
1. Ask a question2. Do background research3. Construct a hypothesis4. Test your hypothesis5. Analyze your data and draw a conclusion6. Report the results
![Page 7: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/7.jpg)
“Guess then test”
![Page 8: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/8.jpg)
Validate your hypothesis
![Page 9: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/9.jpg)
Talk to prospective customers
![Page 10: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/10.jpg)
Currency
![Page 11: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/11.jpg)
Currency
![Page 12: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/12.jpg)
Currency
![Page 13: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/13.jpg)
Currency
![Page 14: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/14.jpg)
Currency
![Page 15: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/15.jpg)
What should we name our product?
![Page 16: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/16.jpg)
Hypotheses
1. “Samoas”2. “Caramel deLites”3. “Caramel DeLites”4. …5. …6. ...7. …8. …9. …10. …
![Page 17: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/17.jpg)
How to test our hypotheses?
![Page 18: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/18.jpg)
What should we name our product?
![Page 19: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/19.jpg)
Book Funnel Analysis
• English speakers on earth – 100%• Read a business book – 45%• Discover the product ad – 30%• Click the product ad – 15%• Click the “Add to cart” button – 10%• Start the checkout process – 8%• Enter their credit card information – 7%• Enter their shipping information – 6%• Buy the book – 5%
![Page 20: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/20.jpg)
Optimize
![Page 21: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/21.jpg)
Hypotheses
1. “The 4-Hour work week”2. “How to work less”3. “How to get rich quick”4. …5. …6. ...7. …8. …9. …10. …
![Page 22: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/22.jpg)
Example Advertisement
![Page 23: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/23.jpg)
Budget Challenge: $???
• Given– We have 10 names to test– We will test each one on 10 different people– It costs $1 for each test (i.e. to buy an ad)
• How much money do we need?
![Page 24: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/24.jpg)
Budget Challenge: $100
![Page 25: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/25.jpg)
The 4-Hour Workweek (of course)
![Page 26: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/26.jpg)
How else could this have been tested?
![Page 27: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/27.jpg)
A/B Testing Examples
![Page 28: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/28.jpg)
A/B Testing Examples
![Page 29: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/29.jpg)
How will users want to sign in?
![Page 30: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/30.jpg)
Sign In Funnel Analysis
• English speakers on earth – 100%• Discover the product – 50%• Download the product – 25%• Launch the product – 20%• Create a username – 10%• Sign into their account – 9%• Launch the product monthly – 5%• Launch the product weekly – 3%• Launch the product daily – 2%• Launch the product hourly – 1%
![Page 31: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/31.jpg)
Sign In Page: Example
![Page 32: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/32.jpg)
Sign In Page: Borrowing
![Page 33: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/33.jpg)
Sign In Page: Borrowing
![Page 34: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/34.jpg)
Sign In Page: Borrowing 3x
![Page 35: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/35.jpg)
Sample Test Results
• Twitter converts 2% of people• Facebook converts 10% of people
![Page 36: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/36.jpg)
A/B Testing Execution
• Given 3,000 people• A/B test for the first 300 (10% of them)– 6 different campaigns with 50 tests each
• Then test final 2,700 people with best campaign.
• Results– Worst campaign (Twitter) converted 2%– Best campaign (Facebook) converted 10%– Difference between 54 and 270 people.– For a $50 product, that is $10,800!
![Page 37: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/37.jpg)
Budget Challenge: Implementation
• Implement Twitter login: $10• Implement Facebook login: $15• Implement LinkedIn login: $20
• But we only have $10!?! What do we do?
![Page 38: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/38.jpg)
Group Work
![Page 39: Product Testing and Refinement](https://reader036.vdocuments.us/reader036/viewer/2022081508/5589eef5d8b42afa0d8b4774/html5/thumbnails/39.jpg)
Group Work
• Assignment [50 minutes]– List 1-2 of your funnels, then “guess and test”– Identify a step to optimize– List your hypothesis and test(s)– Test them on other group members, collect currency
• Group Presentations [30 minutes]– Steps in the funnel(s)– Optimization step(s)– List your hypothesis and test(s)– Summarize the results