Surveying Cell Phone-Only Canadians: Looking at the Possibilities

Presentation to the

Marketing Research and Intelligence Association

Ottawa Chapter

February 19, 2009

Why the research was conducted

- presence of cell-only households

- higher incidence among younger people, men and single-person households

- effect on representativeness of RDD samples

- part of wider study on the use the new technologies in communications

Who conducted the research:

The sampling frame development: ASDE Survey Sampler

The sampling and interviewing: Ekos Research Associates

The analysis: Arcturus Research Solutions

The client: The Public Opinion Research Directorate of Public Works and Government Services Canada

The current situation

- 14% of households in the US and 5% in Canada are cell-only

- The trend in Canada and especially the US is up (see citation below)

- cell-only are disproportionately

- one-person unattached- found in large urban areas- 18 to 34 yrs- below-average income

Source: Statistics Canada’s Residential Telephone Survey; and Ambrose, Don; Gray, Derrick; and Halpenny, Gary; Follow-up on cell-only households – a growing concern for telephone surveys;. VUE, January 2008; pages 16-18

The current situation (continued)

RDD Surveys that exclude cell-only households could encounter the following:

- increasingly incomplete sampling frame

- non response bias (young adults, single-person households)

If present trends continue, the problem is likely to get worse…

Comparison of Coverage of the Population:Telephone and the InternetTelephone: RTSS Survey, Statistics Canada 2006; Internet: New Technologies and Government of Canada Communications, 2007

94%79%

5%1%21%

0%

20%

40%

60%

80%

100%

Telephone Internet

no coverage

cell-only

Coverage

How the Cell Phone Only (CPO) sample

was developed

Randa BellVice President

ASDE Survey SamplerFebruary 19, 2009

• Discussions with clients at 2007 MRIA Conference in Collingwood

• Following growth closely in US

• 5% of Canadian Households are CPO (Stats Can December 2006 Report)

Idea comes to life!

Some numbers

• Cell phones first appeared in Canada around 1983

• Today, upwards of 67% penetration

• 3,000+ exchanges reserved for cell phone only (Telcordia)

Some numbers (continued)

• Each exchange can contain up to 10,000 numbers … potential of 30 million

• Estimate each exchange to be populated at 50 % … which means 15 million cell phones to be reached in Canada

The process

• Huge capacity of ASDE’s IVR System

• Generated numbers at random in cell phone dedicated exchanges

• Proportionate to population of Canada by province

The process (continued)

• Programmed and recorded survey with one question only

• Nature of cell phone we were reaching: CPO household or in addition to landline

• Simplified wording

The questionnaire

• 20 seconds, National Study• Only one question, nothing to sell

• Phone line which is not cell phone = PRESS 1• Cell phone only household = PRESS 2• Reimbursement of call = PRESS 3• Repeat question = PRESS 9

The results

• Two very useful lists

1. Any cell phone that connected

2. CPO households (most valuable list)

• PWGSC chose sample from each list

How to incorporate cell only samples into RDD

• Dual frame sampling method

Portion from cell phones

Portion from RDD

• Augment traditional sample with cell phone numbers (no overlap)

How to incorporate cell only samples into RDD

• Include proportion of CPO households (7%) in sample

• Possibly assign quota on completed interviews vs. starting sample

Research Approach

A text message was sent on January 9, 2008 to 2,391 numbers provided by ASDE Survey sampler

English message was: “The Government has hired EKOS to conduct a study on internet-based technologies and their use in Canada. You are invited to participate.”

Average length of interview: 12.8 minutes

Interviews conducted: January 14 to 28, 2008

The Government of Canada New Technologies in Communications Study: the three modes

Item Cell-only RDD GoC Online

Sample 203 1,718 2,619

Period January

2008

Oct/Nov

2007

Oct/Nov

2007

Response/

completion17% 22% 66%

Early Termination of Interviews

Question: “To begin, we would like to confirm that you are not currently operating a motorized vehicle or machinery of any kind such as a car, truck, van, all terrain vehicle, snowmobile, etc…?” n=293

97%

3%0%

20%

40%

60%

80%

100%

Not currently driving,operating

Currently driving,operating

Terminations owing to Presence of Landline

70 interviews terminated because they have landline in the household

Recall of Text Message

Question: “Do you recall receiving a text message on your cell-phone or wireless device from our company in the past five to seven days notifying you that we would be phoning you to complete an interview?

Please Note: All survey respondents who wish will be sent a gift certificate for $10.00 to either Chapters-Indigo or Tim Hortons, their choice, as a token of our appreciation.” n=203

59%

7%

33%

1%0%

20%

40%

60%

80%

100%

Yes,definitely

Yes,vaguely

No DK/NR

Preferred Incentive

Question: “As we mentioned at the beginning of the survey, we would like to send you a $10 gift certificate to either Chapters-Indigo or Tim Hortons for taking part in this surveys, would you like to receive a

$10 gift certificate?” n=203

28%

63%

10%

0%

20%

40%

60%

80%

100%

Yes, Chapters-Indigo

Yes, TimHortons

No

The Call Disposition

Total numbers accepted

2,391

Total out of scope 549

Unresolved 920

In-scope non-responding 611

Language difficulty 10

Unavailable 0

Refusals 598

Break-offs 3

In-scope responding 311

Completes 203

Ineligible 77

Quota filled 31

Response rate 16.9%

The Cell-Phone Interview Situation

Response to question, “While completing this interview, were you alone or with other people?” (n=203)

68%

29%

3%0%

20%

40%

60%

80%

100%

Alone Family andfriends

Others

Cell Phone Ownership

Response to question, “Are you the sole owner of this cell phone or is its shared with at least one other person 16 years of age or older?” (n=203)

95%

5%

0%

20%

40%

60%

80%

100%

Sole owner Not sole owner

Location of the Cell-Phone Interview Response to question, “When we contacted you, were at …work or school…at home…outdoors?” (n=203)

10%

75%

10%

0%

20%

40%

60%

80%

Work or school Home Outdoors

The Sample: by Gender(n=203)

69%59% 59%

49%

31%41% 41%

51%

0%

20%

40%

60%

80%

100%

Sample,unweight

Sample,weight

RTSS Census

Women

Men

The Sample: by Age(n=203)

28%

39% 38%31%

26%

6%15% 16%

0%

20%

40%

60%

80%

100%

16-24 25-34 35-44 55+

Sample,unweight

Census

The Sample: by Education(n=203)

15% 16%33%

27% 28%

23%

58% 56%44%

0%

20%

40%

60%

80%

100%

Sample,unweight

RTSS Census

Postsecondary

High School

<High School

The Sample: by Household Income (in $000)(n=203)

54% 54%33%

33% 31%51%

7% 8% 16%6% 7%

0%

20%

40%

60%

80%

100%

Sample,unweight

RTSS Census

DK/NA

100K+

40K to 99K

0K to 39K

The Sample: by Number of Persons in Household (n=203)

46%59%

28%

29%

28%

34%

24%13%

38%

0%

20%

40%

60%

80%

100%

Sample,unweight

Cell-only,RTSS

Census(RTSS)

3 +

2

1

Internet Use: Cell-only versus Landline Respondents, by Age

Question: “Do you personally use the Internet? This includes using e-mail.”(cell-only, n=203; landline, n=1718)

88%93% 90% 90%

70%74%

0%

20%

40%

60%

80%

100%

16 to 24 25 to 34 35+

Cell-only

Landline

Mean Hours per Week on the Internet: Cell-only versus Landline Respondents

Question: “ln a typical week, how many hours do you spend on the Internet, not including the time you spend sending or receiving e-mails? Please include all locations, such as home, work, school, etc…”

(cell-only, n=172; landline, n=1,314)

14.114

20.9

11.3

18.1

8.9

0

20

40

16 to 24 25 to 34 35+

Cell-only

Landline

Reading or Contributing to Blogs: Cell-only versus Landline Respondents

Question: “How frequently, if ever, do you… read or contribute to a blog?” (cell-only, n=168; landline, n=1203)

49% 51%

36%42%

15% 7%

0%

20%

40%

60%

80%

100%

Cell-only Landline

daily or more

weekly

never

Participating in Social Network Sites: Cell-only versus Landline Respondents

Question: “How frequently, if ever, do you…participate in a social networking site like Facebook or Myspace?”(cell-only, n=168; landline=1,271)

30%49%

31%

28%

38%23%

0%

20%

40%

60%

80%

100%

Cell-only Landline

daily or more

weekly

never

Using Sites like YouTube: Cell-only versus Landline Respondents

Question: “How frequently, if ever, do you…view, post or download videos from sites like YouTube?”(cell-only, n=168; landline, n=1340)

28%43%

54%

46%

18% 11%

0%

20%

40%

60%

80%

100%

Cell-only Landline

daily or more

weekly

never

Mean Ratings for Web 2.0 Reliability compared to Traditional Media

“Thinking about the different Internet-based applications and technologies that we have been talking about, do you think that the information available on these sources is more or less reliable than information obtained through traditional media, such as television, radio or print? Please use a 5-point scale where 1 means much less reliable, 5 means much more reliable and

the mid-point 3 means the same level of reliability.”(cell-only, n=168; landline=1,292; online, n=1,216)

2.732.88

3.15

0

0.5

1

1.5

2

2.5

3

3.5

Online RDD Cell-only

Mean

Significant differences between the Cell-Phone and Landline Samples: Demographics

(significant at the a=.05 level)

Significant Not significant

Gender

Age

Official Language

Education

Income

Household Size

Employment Status

Province

Urban/Rural Residence

Significant differences between the Cell-Phone and Landline Samples: Attitudes

(significant at the a=.05 level)

Significant Not significant

“Internet more reliable than traditional media”

“The Government should invest in Web 2.0”

“Canadians should be allowed to post content on GoC web sites”

“The Government should post material on popular web sites”

“If GoC material appears on a web site, it means that the Government endorses the web site”

Significant differences between the Cell-Phone and Landline Samples: Behaviour

(significant at the a=.05 level)

Significant Not significant

Hours spent on Internet

Keep informed of Government news and information

Contribute to blogs

Messages and chat rooms

Contribute to social networking sites

Download from YouTube

Internet use

Use of pod/video webcasts

Preference for Survey Mode: Cell-Only Sample

Question: “If you were invited to take part in a Government of Canada public opinion survey, which method would you prefer: completing an online survey or taking part in cell-phone interview, or you have no preference?”

50%

8%

41%

1%

Online

Cell

No preference

DK/NA

Likelihood of Participating in Another Cell-Phone Survey

Response to question, “How likely are you to agree in the future to participate in a Government of Canada survey over the cell phone: very likely, somewhat likely, not very likely or not at all likely?”.

33%

51%

12%

4%

0%

20%

40%

60%

Very Somewhat Not very Not at all

Reasons Given for Not Participating in Future Cell-Phone Survey (number of respondents)

(Among persons who said “not very likely” or “not at all likely”) Response to question, “Why do you say that?” Number of responses (n=26)

11

8

23

2

0

2

4

6

8

10

12

Toobusy/time

Dislikesurveys

Cost PreferOnline

DK/NA

Costing: the Final AnalysisTotal cost: $20,803

Cost per completed interview: $102.48

$1,800 $1,830

$7,741

$9,432

$0

$2,000

$4,000

$6,000

$8,000

$10,000

SMS, letter,postage

Incentives Interviewing SampleDevelopment

Costing: Hypothetical Survey of 1,000 AdultsCost per completed interview 50 cell-only: $102.48Cost per completed interview 950 landline: $24.21

$5,124

$23,000

$28,124

$0

$5,000

$10,000

$15,000

$20,000

$25,000

$30,000

50 cell-only 950 landline 1000 total

Conclusions

-Surveys of cell-phone only households feasible in Canada

- Significant differences between cell-only and landline respondents:

- subject matter- demographics- attitudes - behavioural

Conclusions (continued)

- Canadians will respond

- ethical/respondent burden

- keep the survey short (12 minutes or less)

- high cost relative to other modes

- keep in mind target population of survey

The complete report is also available

For a copy of the complete report (English) or the executive summary (English or French), see the report A Survey of Cellular-Telephone-Only Households The New Technologies “Web 2.0” and Government of Canada Communications Project at

http://www.porr-rrop.gc.ca/index-e.html