ntc 2016 step right up testing presentation 16x9
TRANSCRIPT
![Page 1: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/1.jpg)
Step Right Up! Everyone’s a Winner!
Steve Kerhli, PETA & Steve Daigneault, M+R
![Page 2: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/2.jpg)
Ask String on Main Donation Form
Test Control
Vote: Control Test Tie
Winner: Test
![Page 3: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/3.jpg)
Renewal Language to Prospects
Test Control
Vote: Control Test Tie
Winner: Test
![Page 4: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/4.jpg)
Higher Ask String for Prospects
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 5: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/5.jpg)
SMS Deadline Reminder
Test Control
Vote: Control Test Tie
Winner: Test
![Page 6: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/6.jpg)
Nudge to Prospects
Test Control
Vote: Control Test Tie
Winner: Test
![Page 7: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/7.jpg)
Font Type and Size
Test Control
Vote: Control Test Tie
Winner: Test
![Page 8: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/8.jpg)
$5 Ask to Prospects
Test Control
Vote: Control Test Tie
Winner: Test
Make your tax-deductible gift
of $50 or whatever you can
afford now. Help us meet our
urgent Year-End Challenge goal
of $125,000.
Make your tax-deductible gift
now. Pitch in $5 or more to help
us meet our urgent Year-End
Challenge goal of $125,000.
![Page 9: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/9.jpg)
Adding PayPal as Payment Option
Test Control
Vote: Control Test Tie
Winner: Test
![Page 10: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/10.jpg)
“Renew” Language to Prospects
Test Control
Vote: Control Test Tie
Winner: Test
![Page 11: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/11.jpg)
Another Nudge
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 12: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/12.jpg)
Cents Per Day
Test Control
Vote: Control Test Tie
Winner: Control
![Page 13: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/13.jpg)
Yes, Another Nudge Test
Test Control
Vote: Control Test Tie
Winner: Test
![Page 14: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/14.jpg)
Multi-Button vs. Single Button
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 15: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/15.jpg)
Quick Donate Based on HPC
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 16: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/16.jpg)
EOY Lightbox
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 17: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/17.jpg)
$10 Ask to Lapsed and Deeply Lapsed Donors
Test Control
Vote: Control Test Tie
Winner: Test
![Page 18: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/18.jpg)
“Read More” Button
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 19: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/19.jpg)
Nudge to Prospects + HPC<$125 Donors
Test
Control
Vote: Control Test Tie
Winner: Test
![Page 20: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/20.jpg)
Video on Donation Form
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 21: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/21.jpg)
Reverse Ask String
Test Control
Vote: Control Test Tie
Winner: Test
![Page 22: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/22.jpg)
Button Colour
Test Control
Vote: Control Test Tie
Winner: Test
![Page 23: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/23.jpg)
Email Recruitment Lightbox
Version 2 Version 1
Vote: Version 1 Version 2 Version 3
Winner: Version 2
Version 3
![Page 24: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/24.jpg)
Donate vs. Membership Lightbox at EOY
Test Control
Vote: Control Test Tie
Winner: Test
![Page 25: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/25.jpg)
Donate vs. Membership Lightbox at EOY
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 26: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/26.jpg)
Streamlining Donation Form
Test Control
Vote: Control Test Tie
Winner: Control
![Page 27: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/27.jpg)
OMG, Another Nudge Test
Test Control
Vote: Control Test Tie
Winner: Test
![Page 28: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/28.jpg)
Recommendations for Nudge Tests
• Tested variations of nudge tests to 10 organizations.
• 50% saw improvement in at least one key metric. No organization saw nudges depress response or revenue.
• Recommended language: • “Most people are donating at least $X right
now.”
![Page 29: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/29.jpg)
Upper Case vs. Sentence Case Lightbox
Test Control
Vote: Control Test Tie
Winner: Test
![Page 30: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/30.jpg)
Multistep vs. Single Column Form
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 31: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/31.jpg)
Another PayPal Payment Option
Test Control
Vote: Control Test Tie
Winner: Test
![Page 32: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/32.jpg)
Ask String vs. Open Field
Test
Control
Vote: Control Test Tie
Winner: Test
![Page 33: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/33.jpg)
Right Call-Out vs. Centered
Test Control
Vote: Control Test Tie
Winner: Test
![Page 34: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/34.jpg)
Review vs. Donate Button
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 35: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/35.jpg)
Paragraph Spacing
Test Control
Vote: Control Test Tie
Winner: Control
![Page 36: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/36.jpg)
One Column vs. Two Column
Test Control
Vote: Control Test Tie
Winner: Tie
![Page 37: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/37.jpg)
PETA Tests
![Page 38: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/38.jpg)
$5 Ask to Non-Donors
Test
Control
Vote: Control Test Tie
Winner: Test
![Page 39: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/39.jpg)
$5 Ask vs $10 Ask to Non-Donors
Test Control
Vote: Control Test Tie
Winner: Control
![Page 40: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/40.jpg)
Copy Length on Donation Form
Test Control
Vote: Control Test Tie
Winner: Test
![Page 41: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/41.jpg)
Single Image vs. Multi-Image
Test Control
Vote: Control Test Tie
Winner: Test
![Page 42: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/42.jpg)
Larger Link Size
Test Control
Vote: Control Test Tie
Winner: Test
![Page 43: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/43.jpg)
Animal Specific Donation Forms
Test
Control
Vote: Control Test Tie
Winner: Test
![Page 44: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/44.jpg)
Renewal Language to Current + Lapsed Donors
Test Control
Vote: Control Test Tie
Winner: Test
![Page 45: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/45.jpg)
Membership Buttons on Homepage
Test Control
Vote: Control Test Tie
Winner: Test
![Page 46: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/46.jpg)
Tools for Running a Strong Testing Program
![Page 47: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/47.jpg)
✳ The purpose of hypotheses is to make sure we're thinking critically about what we want to test and why and to make sure we learn from the test.
✳ To help create a hypothesis, fill in the blanks:
Changing (control) to (variation) will improve (test objective) because (reason you think the change will help) .
✳ If you craft a good hypothesis, you will learn something from every test, regardless of the outcome.
Crafting the hypothesis
![Page 48: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/48.jpg)
✳ Before running a test, figure out the sample size needed for the test.
✳ To do that, you’ll need to understand how much of a change you’ll be able to detect.
✳ If you’re evaluating specific audiences or pages (e.g. mobile traffic), do this for each page or segment that you will evaluate
Determine Sample Size needed
![Page 49: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/49.jpg)
Baseline Conversion Rate - the current average for the metric you are trying to improve (e.g. response rate, action rate)
Minimum Detectable Effect (MDE) - the amount of change in the conversion rate that you want to be able to detect. We recommend 25% or lower.
Elements of the sample size calculator
Minimum Audience Size - the total number of people you need to include in the experiment, per variation.
![Page 50: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/50.jpg)
Sample Testing Roadmap
![Page 51: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/51.jpg)
Sample Test Plan
![Page 52: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/52.jpg)
Sample Results Tracker
![Page 53: Ntc 2016 step right up testing presentation 16x9](https://reader031.vdocuments.us/reader031/viewer/2022030209/58ae691c1a28abd11c8b54ab/html5/thumbnails/53.jpg)
All In One Place