the effect on differential mailing methodologies on ...postcards 13.6 1, 2, 3 official package 10.4...
TRANSCRIPT
![Page 1: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/1.jpg)
The Effect on Differential Mailing Methodologies on Response RatesTesting Advanced Notices, Package Design, Postage, and Other
Treatments
American Association for Public Opinion ResearchOrlando, Florida
May 18, 2012
Presented by: Yelena Pens, Robin Gentry
![Page 2: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/2.jpg)
© 2012 Arbitron Inc.
Arbitron Diary Methodology
» Arbitron Radio Ratings recruits all household members (aged 12+) for a one-week diary survey of radio listening.
» Hybrid RDD and ABS sample.
» Once the person who answered the recruitment call agrees to participate in the radio ratings, the household is mailed a box.
Diary
Box
Brochure
Multi Person Flyer
![Page 3: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/3.jpg)
© 2012 Arbitron Inc.
Online Diary Vision
»Objective• To increase sample sizes while reducing costs • Improve representation for young adults and hard to
reach demos
»New Methodology• Primarily a web-based survey• Address based sample frame• More relevant to web-savvy younger adults and hard to
reach demos
![Page 4: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/4.jpg)
© 2012 Arbitron Inc.
Online Diary Field Test Methodology
»Sample: • ~11,000 households• ABS matched and unmatched telephone numbers• Five markets
»Methodology:• Fielded in Summer 2011• Tested several different mailing approaches, design, and personalization.• New online diary application (registration and radio listening portions)• Household flooding approach• Mobile option (iPhone and Android)• Emails (registration confirmation, reminders, thank you/incentive)• Promised Incentive choices (Amazon or Visa Gift Cards)
![Page 5: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/5.jpg)
© 2012 Arbitron Inc.
Treatments and Test Strategy
Control Test #1 Test #2 Test #3
Address Mail Label (Personalization Test)*Name Arbitron Radio Household
*See session The Effect on Personalized Address Labels on Response Rates and Postal Deliverability Rates by Vrinda Nair and Yelena Pens, Friday, May18, 2012 at 10:00 AM-11:30 AM
Box Invitation
6x9 envelope
Note Card
#10envelope
Snap Pack #2
Note Card
Snap Pack #3
Pre-alertPost Card
Pre-alert Phone
Message
Pre-alert: Post Card
Pre-alert Post Card
Promotional Package Official Package
Pre-alertNone
+
Snap Pack #1
![Page 6: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/6.jpg)
© 2012 Arbitron Inc.
Mailing Materials ( Promotional Look)
Wave 1: Pre-Alert postcard (Oversized)
Wave 2: Initial Invitation Box Mailer
Wave 3: 6x9 “Still Time” Reminder
Wave 5: Note card “Extended Deadline” Reminder
Wave 4: #10 “Last Chance” Reminder
$2 incentive
![Page 7: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/7.jpg)
© 2012 Arbitron Inc.
Mailing Materials (Official Look)
Wave 1: Pre-Alert postcard
Wave 2: Initial Invitation
Wave 3: “Still Time” Reminder
Wave 4: “Last Chance” Reminder
Wave 5: Note card “Extended Deadline” Reminder
![Page 8: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/8.jpg)
© 2012 Arbitron Inc.
Research Questions
» Mailing Treatments• Do pre-alert mailers alone increase response rates?• Do pre-recorded telephone messages enhance the effect of the
pre-alert mailers?• Does the official looking package increase response rates?
![Page 9: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/9.jpg)
© 2012 Arbitron Inc.
Response Rate Calculation
» Adjusted Response Rate:
» Registered Households• Households who successfully registered for the online diary• Not including completion of the online diary• This is a household level not person level statistic
» Estimated Usable Households• Starting sample • Removing the following:
– Non-deliverable Unusable Households (Vacant, No Such Number, Insufficient Address, or No Such Street)
– Media affiliated households– Group Quarters households (10+ persons living in a household such as
dormitories, military barracks, etc.)
= Registered Households
Estimated Usable Households÷(Also called registration rate)
![Page 10: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/10.jpg)
© 2012 Arbitron Inc.
Advanced Notices and Package Redesign
» Official looking package had a substantially lower rate compared to promotional mailers, thus the package did not increase response rates
» Pre-alert mailers alone did not increase response rates. • Surprising results as research
shows advanced notices help to increase response.
• This could be the design of mailer.
Treatment Groups % Sig.
Control 16.3 1
Phone and postcard 16.8 2
Postcards 13.6 1, 2, 3
Official Package 10.4 3
Overall 14.3
Household Response Rates by Treatment
1 Statistically significant differences between postcards and no postcards
2 Statistically significant differences between pre-recorded phone with postcards and postcards alone.
3. Statistically significant differences between promotional look and official look
Note: Rates based on a matched sample.
The pre-recorded telephone messages had a positive impact on response and enhanced the effect of the pre-alert postcards.
![Page 11: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/11.jpg)
© 2012 Arbitron Inc.
Advanced Notices and Package Redesign (Continued)
» Results of Pre-alert Postcard • Design (oversized postcard)
– A large majority of respondents identified it as junk mail and would throw it away.– Many said it appeared to look like something they would receive from a healthcare,
insurance or pharmaceutical company.– Considered to have too much text and unappealing pictures.
• Headlines– “Represent your community” – Some respondents thought “community” was having
to go somewhere and work or donate their time with others.– “You listen to the radio, now the radio listens to you!” – Most recognized that it was
too vague – they weren’t clear on what it was asking of them.– “Be part of the Arbitron radio ratings” – clearly states what they are being asked do.
» Focus Groups (Winter 2012)• Hispanic demos in three markets• Pre-alert postcard, box mailer, and
website
![Page 12: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/12.jpg)
© 2012 Arbitron Inc.
Postage
» The promotional mailers used first-class postage, while the official mailers used standard mail.
» First-Class Mail • Delivered within 2-3
days• Return of mail• Higher Cost
» Standard Mail• Delivered within 7-10
days• No return of mail• Lower cost
45.3%
33.1%
25.5%
31.0%
19.1%16.9%
10.1%
19.0%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
Promotional Official
InvitationStill TimeLast ChanceNote Card
Distribution of Total Response by Wave and Mailer
Note: The note cards were designed to have a flashy appearance and were mailed using first-class live stamps.
Unlike the promotional mailers where the note cards had the smallest impact, the official mailers had an increase in response.
![Page 13: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/13.jpg)
© 2012 Arbitron Inc.
Budget (Package Design and Postage)
» Promotional mailers had a positive impact on response compared to the official mailers.
» Promotional mailers have higher mailing costs compared to the official mailers with postage driving the increase.
» Is the increase in RR worth the increase in cost?
Promotional Package
Official Package
Total Mailers $0.288 $0.68
Pre-alert postcard $0.05 $0.015
Initial Invitation $0.155 $0.31
“Still Time” Invitation $0.025 $0.21
“Last Chance” Invitation $0.023 $0.11
“Extended Deadline” $0.035 $0.035
Total Postage $3.75 $1.79
Total (per household) $4.04 $2.47
Note: Labor costs for vendors are not included in the pricing; excludes incentives.
![Page 14: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/14.jpg)
© 2012 Arbitron Inc.
Budget (Package Design and Postage)(Continued)
» Higher response rates, lower costs.• More sample will have to be
mailed of the official items to reach the same targeted sample with promotional mailers.
» Sample representation better with promotional items.• Older demos had higher rates
for the official materials.• Hard to reach demos had
higher rates for the promotional materials.
Promotional Package
Official Package
Targeted Completes 10,000 10,000
Response Rate 16.8% 10.4%
Mailed Sample 59,524 96,154
Incentives ($2) $119,048 $192,308
Materials/Postage $240,357 $237,500
Total Costs $359,405 $429,808
![Page 15: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/15.jpg)
© 2012 Arbitron Inc.
Multiple Mailings
» If multiple mailings are being mailed to the same address, non-deliverable mail may differ by packages/waves.
» Reasons for inconsistencies:• When a mail piece is returned to the sender for any reason, the piece
is endorsed by the carrier.• There many be different carriers on a route on any given day, so
endorsements may vary.
Promotional Mailerpre-alert box 6x9 #10
Usable 84.0 76.7 82.4 82.8
Unusable 16.0 23.3 17.6 17.2
Non-Deliverables by Mailer
The box which is a bulky mailer has the highest unusable rate.
Note: Usable: Attempted Not Known , Illegible, Unable to Forward/Forwarding Order Expired, Deceased, Temporarily Away/hold mail, Refused, No Mail Receptacle, In Dispute, or Unclaimed; Unusable: Vacant, No Such Number, Insufficient Address, or No Such Street.
![Page 16: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/16.jpg)
© 2012 Arbitron Inc.
Postal Day Mailing» The promotional packages as well as the official packages were
mailed each postal day (except for Saturday mailings for official packages because of logistic reasons).
17.5
10.0
14.5
9.6
15.6
10.3
15.7
9.7
14.8
8.1
14.3
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
18.0
20.0
Promotional Official
MondayTuesdayWednesdayThursdayFridaySaturday
Response Rate by Day of Week
Monday mailings performed better than any other mailing of the week.
![Page 17: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/17.jpg)
© 2012 Arbitron Inc.
Additional Treatments (Effect on Response Rates)
» Incentive Choice (Visa vs. Amazon)• Mailed Visa gift cards were the preferred choice over online,
instantly available, Amazon gift cards (77% vs. 23%).
» Choice of Survey Start Date• About 93% of registrants selected a diary start date on the same
day as registration for the diary.
» Mobile Option• About 11% of new visitors going to the site were mobile users.
Desktop users were more likely to come back to the website as compared to mobile users.
![Page 18: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/18.jpg)
© 2012 Arbitron Inc.
Post Test (Effect on Response Rates)
» Emails• About half (46%) of the pre-registered household members did not have
an email address. Also, 93% of those pre-registered household members without an email address did not register for the online diary.
» Post Test• Pre-registered household members were mailed a package with a
personalized invitation to register for the online diary as well as a small token of appreciation.
• The packages were addressed to the original registrant with instructions to distribute the packages to unregistered household members.
• Results…– The household member registration rate was 36%, and after the Post Test it
was 47%, so the mailers had increased overall household member registration by over 10 points.
• It appears direct communication such as emails and mailers are important to response rates.
– Possibly the original registrant is a gatekeeper for other household members.
![Page 19: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/19.jpg)
© 2012 Arbitron Inc.
Conclusions
» The official looking package had a substantially lower household registration rates compared to the promotional, direct mail marketing materials.
» The pre-alert postcards plus pre-recorded telephone messages performed the best.
» On the other hand, the postcards alone did not perform better than no postcards.
• The results from a focus group show that design and messaging has an effect on response.
» Monday First Class mailings performed the best.
Advanced Notices
Pre-recorded telephone messages enhance mailers
Package Design
Promotional, direct marketing, bulk boxes work
Postage First Class Mail
Mail Date Monday, Received on Wednesday
![Page 20: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/20.jpg)
© 2012 Arbitron Inc.
Next Steps
» Planning a third Pilot Test in Fall 2012• Focus on collecting email addresses from all household members• Solving usability issues from previous tests• Testing pre-recorded telephone messages alone as a pre-alert• Testing different pre-alert designs and messaging• Testing a pre-alert incentive
![Page 21: The Effect on Differential Mailing Methodologies on ...Postcards 13.6 1, 2, 3 Official Package 10.4 3 Overall 14.3 Household Response Rates by Treatment 1 Statistically significant](https://reader033.vdocuments.us/reader033/viewer/2022043022/5f3e331222639f6d136a3032/html5/thumbnails/21.jpg)
© 2012 Arbitron Inc.
Thank You
» For more information please contact:• Yelena Pens
– [email protected]• Robin Gentry