May 10, 2012

Respondent Engagement: Why do they respond anyway? Or ‘It’s not all ‘bout the Benjamins”

Panelists can be painful

2

Panelists are expensive to recruit, difficult to retain and,

sometimes, a royal pain

But they are also are the soil and water of our

research

Without them, we can do nothing

Why do they do it?

3

It is important for us to understand why they choose to

respond (or not) and why they choose to stay in or drop

out of our panels

In this talk we will:

• Look at what the literature tells about why people

respond to surveys

• Review some fresh data, collected specifically

for this meeting, on why people do or do not like

to do surveys

Why do they do it?

What the Literature Tells Us

Why do they do it?

5

“Factors affecting response rates of the web survey: a

systematic review” summarizes a great deal of the

literature on this subject. The authors identify four

steps, each of which have an important impact on

survey completion

Web Survey Surveyee Surveyorr

2. Survey Delivery

3. Survey Completion

1. Survey Development

4. Survey Delivery

From: Fan W, Yan Z, (2010) Factors affecting response rates of the web survey:

a systematic review, Computers in Human Behavior, 26, 132-139.

Why do they do it?

6

From: Fan W, Yan Z, (2010) Factors affecting response rates of the web survey:

a systematic review, Computers in Human Behavior, 26, 132-139.

Survey Development

Questionnaire content

Questionnaire presentations

Survey Delivery

Sampling methods

Delivery modes

Invitation design

Pre-notifications and reminders

Incentives

Survey Completion

Society-related

Respondent-related

(conscientiousness, agreeable,

openness to experience)

Design-related

Survey Return

Survey software

Data safety

Why do they do it?

7

Fan W, Yan Z, (2010) Factors affecting response rates of the web survey: a

systematic review, Computers in Human Behavior, 26, 132-139.

Survey Development

Questionnaire content

Questionnaire presentations

Survey Delivery

Sampling methods

Delivery modes

Invitation design

Pre-notifications and reminders

Incentives

Survey Completion

Society-related

Respondent-related

(conscientiousness, agreeable,

openness to experience)

Design-related

Survey Return

Survey software

Data safety

The Benjamins have a small but real effect

8

In “Incentives in Web Studies: Methodological Issues

and a Review”, Göritz conducted a meta-analysis of the

effectiveness of incentives

She concluded “material incentives increase response

and decrease drop-out” but “the combined effect of

incentives on response and retention is still small”

She also reminded readers that “using material

incentives is only one option to influence data quality

and quantity. We should not forget about other

possibly response-enhancing techniques such as

personalization, prenotification, deadlines, reminders,

offering result summaries and altruistic appeal”

Goritz, AS, (2006) Incentives in Web Surveys: Methodological Issues and a

Review, International Journal of Internet Science, 1, 58-70.

This meta-analysis reveals that adding incentives does make a difference to response rate, but it has a pretty modest effect

9

Goritz, AS, (2006) Incentives in Web Surveys: Methodological Issues and a

Review, International Journal of Internet Science, 1, 58-70.

0

5

10

15

20

25

30

35

40

45

50

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

Increase over baseline response rate without incentives

% in

cre

ase

in

re

sp

on

se r

ate

Baseline response rate without incentives

We did some of our own testing on the effects of incentives on our panelists and, as a result, we reduced the incentives paid on our panels

10

31% 30%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Fifty cents Dollar

The response rate at $.50 vs. $1 across 153 omnis in Canada, the US and UK

The response rate at $.50 vs. $1

across 153 omnis in Canada, the US and UK

We tested dropping direct incentives and moving to a sweepstakes and found lower response among young people, especially males

11

0

10

20

30

40

50

18-34 35-54 55+

50 cents Draw

0

10

20

30

40

50

18-34 35-54 55+

50 cents Draw

Canada

0

10

20

30

40

50

18-34 35-54 55+

50 cents DrawUS UK

Response rate for

50 cents vs. draw

Why do they do it?

What Panelists Tells Us

Survey of Springboard America Panelists

13

We conducted a survey of 1006 members of

Springboard America, our US panel, on April 12, 2012

We looked at attitudes toward completing surveys

and incentives, feedback, community mindedness,

curiosity and being valued

We found that intrinsic motivations are more

important than extrinsic ones—highlighting the

value and importance of respondent engagement

Respondents do surveys because they are curious

14

“I enjoy learning about new things and products

when I do surveys”

95 % agree

Respondents want to be good citizens

15

“I feel like I am doing my part as a good

consumer and citizen when I provide feedback”

89 % agree

Respondents want to help and be respected

16

“I feel like I am being a trusted advisor when I

provide feedback to a company on their

products”

87 % agree

Respondents want feedback

17

“I love it when I see the results of a survey I

participated in”

86 % agree

Personalizing winners of sweepstakes, when possible, is important

18

“I often wonder if anyone ever really wins when

a survey offers a sweepstakes for participating”

86 % agree

This doubt underscores the need for and value of providing constant feedback to panelists

19

“I sometimes wonder if anyone ever really even

sees and uses the feedback I provide through a

survey”

62 % agree

Money does it for a few

20

“I am too busy to take surveys unless I am paid”

36 % agree

Money does it for a few—especially younger respondents

21

“I am too busy to take surveys unless I am paid”

51

32 36

0

10

20

30

40

50

60

70

80

90

100

18-34 35-54 55+

% agree

In the end, respondents just want to contribute

22

“I feel like my opinion makes a difference”

86 % agree

Conclusions

Why they respond

24

The literature and our survey confirm the

importance of engaging panelists.

They feel good about the feedback they

provide.

But they want you to tell them that, and share

back some of your learnings—which tangibly

demonstrates that you value their input

We’ve proven the value of these best practices

25

Test “new” ideas, offers, anything novel with

respondents—they value it

Provide feedback from surveys, they love it and

it reassures them that you are listening

Show a picture of sweepstakes winner “Bob S

of Shawinigan”—it is another piece of feedback

Tell them they are trusted advisors and show

them how they make a difference

Thank you