improving response rates lessons from physician surveys pmrs ottawa chapter february 26, 2004
TRANSCRIPT
Presentation Overview
Survey Response Rates: The state of the art Particularities of Physician Surveys Response rate boosting tactics: What works
and what does not. On the use of Monetary Incentives in Physician
and Consumer Surveys Q & A period
Current response rates
Academic Surveys published between 1961 and 1977: 71%
Academic surveys published in 1991: 54% Academic surveys published between 1986 and
1995, sample size over 1,000 respondents: 52% Commercial/marketing physician surveys
(2002): 20% RETICULUM surveys: 12% to 66%
Current response rates
Surveys of executives, published in 1991: 21% PMRS Members surveys: 15.7%(1997);11.3%
(2000); One-time telephone surveys: 16% (1997); 13%
(2002)
Physician Surveys Particularities
More homogenous populations
Highly-regulated professionals
Better sampling frames
Better record-keeping
Physician Surveys Particularities
Highly-solicited respondents Highly-educated respondents ‘Well-connected’ respondents
Physician Surveys Particularities
Surrounded by ‘tough’ gatekeepers
‘Addicted’ to monetary incentives
Tactics that boost response by 50% or more
Monetary Incentives
Multiple contacts & multiple contact modes
Monetary Incentives
(Gallagher, 2001)1st contact by mail, no incentive: 11%2nd contact by phone, no incentive: 22% (cumul)3rd contact by courier, $20 incentive: 57% (cumul)
(Malin, 2000)1st mailing, no incentive: 17%2nd mailing, no incentive: 13%3rd mailing, $50 incentive: 66%Cumulative response rate: 76%
Multiple Contacts & Contact Modes
Typical response rates after multiple mailings:
1st mailing: X%
2nd mailing: X/2%
3rd mailing: X/4% (CDC, 1997):
1st contact by First Class mail: 60%
2nd contact by Fedex: 72% (cumul)
3rd contact by phone: 96% (cumul)
Tactics that boost response by a few % points
Pre-notification by phone Personalization Advertising the survey Choice of sponsors Shortening the questionnaire Instituting a draw
Pre-notification by phone
(Osborn, 1996)
No Pre-notification: 64%
Pre-notification: 77%
(Ward, 1994)
No Pre-notification: 69%
Pre-notification: 84%
Personalization
First Class mail Commemorative stamps Stamped return envelope Name & address printed on the envelope Personalized salutation Full date on Cover Letter Handwritten signature Handwritten note
Personalization
(Maheux, 1989)
Handwritten ‘thank you’ note: 30%
No ‘thank you’ note: 22% (Streiff, 1999)
Stamped return envelope: 38%
Business-reply envelope: 32%
Choice of sponsor
(Asch, 1994)
Veteran Affairs return address pulled 20% more than a Hospital Department of Medicine
(RETICULUM, 2000)
A joint study with IMS Health, Royal College, College of Family Physicians: 22%
IMS Health alone: 12%
Tactics that don’t boost response
Pre-notification by mail Offering non-monetary incentives (pens,
mouse pads, candy, booklets, software..) Mailing surveys on a specific day of the week Promising anonymity Gimmicks
On the use of Monetary Incentives
Even symbolic sums will boost response (Everett,1997)
$0: 45%
$1: 63% (one-dollar bill included in mailing) (Donaldson, 1999)
$0: 46%
$5: 58% (five-dollar cheque included)
On the use of Monetary Incentives
Larger incentives, Higher response rates (Asch, 1998)
$2 incentive: 46%$5 incentive: 63%
(Gunn, 1981)$0 incentive: 58%$25 incentive: 69%$50 incentive: 77%
On the use of Monetary Incentives
Larger incentives, Higher response rates:UP TO A POINT (VanGeest, 2001)
$5 incentive: 60% ;$10 incentive: 68%$20 incentive: 67%
(RETICULUM/ IMS Health, 2000)$25 incentive: 22%; $50 incentive: 34%$75 incentive: 36%
On the use of Monetary Incentives
Pre-paid incentives outperform
Post-paid incentives (Berry, 1987)
$20 incentive, pre-paid: 78%
$20 incentive, post-paid: 66%
Monetary Incentives in Lay Surveys
Sparse data Controversial practice Banned in certain jurisdictions
Monetary Incentives in Lay Surveys
(James & Bolstein, 1992)
$0: 52%
$2: 64% (Dillman and al., 1999) in 5 different studies
$2 incentive: boosted response by 19 to 31%
Monetary Incentives in Lay Surveys
Pre-paid incentives will outperform post-paid incentives (Johnson & McLaughlin, 1990):
$5 pre-paid: 83%
$10 post-paid: 72% (James & Bolstein, 1992): survey of small contractors
$1 pre-paid: 64%
$5 pre-paid: 72%
$50 post-paid: 57%
On the use of Monetary Incentives
Respondent appreciated, not taken for granted Value-creating Attention grabbing: Secretary Attention grabbing: Physician Pre-paid incentives: create trust
On the use of Monetary Incentives
Pre-paid incentives (Gallagher, 2001)
46% replied
3% declined and returned the 20-dollar pre-paid incentive
51% declined, but pocketed the 20-dollar pre-paid incentive