analysis of factors related to nonresponse for landline and cell phone surveys in china
DESCRIPTION
ITSEW 2010. Analysis of Factors Related to Nonresponse for Landline and Cell Phone Surveys in China. Yan Jiang School of Statistics, Renmin University of China Visiting Professor, Iowa State University, Department of Statistics and Center for Survey Statistics and Methodology. Outline. - PowerPoint PPT PresentationTRANSCRIPT
Analysis of Factors Related to Nonresponse for Landline and
Cell Phone Surveys in China
Yan Jiang
School of Statistics, Renmin University of China
Visiting Professor, Iowa State University, Department of Statistics and Center for Survey Statistics and Methodology
ITSEWITSEW20102010
04/20/232
Outline
Introduction Survey Design Nonresponse Rates Call Attempt Analysis Interviewer Analysis Conclusions
04/20/233
Introduction
Surveys in China High nonresponse rates
Low contact rates (RDD Sampling)High refusal rates
Change of coverage Steady increase in cell phone frame Decrease in landline phone frame
since 2006
04/20/234
Amount of telephone users
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
cell phone users
landline telephone households
(Ten thousands)
landline telephone users
2009
Cellphone Users:
747.38 million
Landline Users:697.02 million
Overlap: unknown
Total :1335 million
Survey Objectives
To evaluate the utility of selected methods (e.g., increased call effort) in telephone surveys of adults in China
To identify factors related to nonresponse rates (e.g., interviewer voice quality, call effort)
To study how the effect of these factors on nonresponse vary by sample frame (i.e., landline, cell)
04/20/235
Survey Design
Population: Adults in China (over 18 years old) Two-phase Sampling Design
Phase 1: Dual frame sample of phone numbers to sample adults
Phase 2: Sample of nonrespondents
Data Collection Mode: CATI Phase 2: Modified methods
04/20/236
Survey Design in First Phase Sample Size: 17,162 phone numbers
Cell phone: 9678 Landline: 7484
Stratified RDD Sampling Strata: Provinces Allocation: proportional to Province HH pop.
Standard call methods Up to 3 call attempts – usually 2 Ring for about 20 secs
04/20/237
Survey Design in Second Phase Population: nonrespondents in first phase
Sample size: 1549 Strata: province, frame source, noncontact
and refusal numbers Effort to improve the response rate (both
cell phone and landline survey) Up to 15 call attempts were made to contact Wait until at least one minute in each dial
process
04/20/238
Nonresponse Rate in First Phase Survey
Landline Survey Cell phone Survey
Number Percent Number Percent
Total 190,680 100% 169,116 100%
Non-Eligible 149,224 78.3% 108,213 64%
Eligible 41,456 21.7% 60,903 36%
Response 7,484 18.0% 9,678 15.9%
Non-Response 33,972 81.9%81.9% 51,225 84.1%84.1%
Non-Contact 23,446 56.6% 36,716 60.3%
Refusal 10,526 25.4% 14,509 23.8%
Nonresponse Rate in Second Phase Survey
landline cellphone
sample size rate sample size Rate
response 763 43.6%43.6% 786 45.0%45.0%
nonresponse 985 56.4% 961 55.0%
non-contact 443 25.4% 467 26.7%
refuse 542 31.0% 495 28.3%
04/20/2311
Number of call attempts in refusal sample (stratum) of Phase 2
Cumulated Refusal Conversion Rate of landline survey is higher than cell phone survey when the number of attempts is less than 5.
Cumulated Refusal Conversion Rate increases rapidly up to about 6 call attempts.
LandlineCell phone
Number of call attempts
04/20/2312
logistic model for Refusal Conversion Dep. Variable=1, refusal unit convert to respondent
landline survey cell phone survey
Variable B Sig. Variable B Sig.
Gender -.014 .604 Gender -.196 .000
Young -.791 .000 Young -1.056 .000
Middle-aged -.767 .000 Middle-aged -1.068 .000
Education -.189 .000 Education .187 .000
Income -.066 .000 Income -.090 .000
City -.066 .024 City .156 .000
East 1.236 .000 East .941 .000
Middle -.367 .000 Middle .984 .000
Numberof call attempts
.146 .000Numberof call attempts
.097 .000
Constant -1.621 .000 Constant -2.241 .000
The relationship between above factors and refusal conversion rates differs for the landline survey and the cell phone survey.
04/20/2313
Number of call attempts in non-contact sample (stratum) of Phase 2
Cumulated Phase 2 Response Rate of landline survey is lower than cell phone survey in non-contact sample.
LandlineCell phone
Number of call attempts
04/20/2314
Distribution of the number seconds until the phone is answered
100.0080.0060.0040.0020.000.00
等待时间
200
150
100
50
0
成功手机频数
Mean = 32.5089Std. Dev. = 10.2653N = 786
120.00100.0080.0060.0040.0020.000.00
等待时间
250
200
150
100
50
0
成功固定电话频数
Mean = 19.5464Std. Dev. = 10.30387N = 765
On average,10 additional seconds per interview are required for the phone to be answered in the cell phone survey relative to the landline survey.
Cell PhoneCell Phone
Land lineLand line
04/20/2315
Phase 1 Interviewer Analysis Cooperation rate distribution of interviewers
0.700.600.500.400.300.20
Cooperation Rate
20
15
10
5
0
Fre
qu
en
cy
Mean = 0.375
Std. Dev. = 0.083
N = 108
Phase 1 Interviewer Analysis
Which interviewer characteristics affect cooperation rates?
Interviewer characteristicsSkill levelVocal quality
Control for respondent characteristicsGender, education, career, income,
location of residence, sample frame
04/20/2316
Phase 1 Interviewer Analysis
Hierarchical Generalized Linear Model Level 1 model (respondent level)
Level 2 model : (interviewer level)
q qijqjj
ij
ij X)p
plog( 01
qjs sjqsqqj W 0
~ (0, )qj qqN
qijX
sjW
i : respondent; j: interviewer; :refusal rate : variables of respondent : variables of interviewer
ijp
04/20/2318
Variable Definitions
outcome of interview
y =1, refusal; y =0, response
respondent
X
gender=1, male; gender=0, female
edu1: low education, edu2: midium education
career1: student, career2: manager, career3: clerk, career4: self-employee
income>=0
survey =1, landline ; survey =0, cell phone
grade: =1, large city; grade=0, other
InterviewerW
W1: interview skill of interviewer
W2: vocal characteristic of interviewer
( | )ij ij sp prob y W
Results
Both the interviewers’ speaking skill and natural
vocal characteristics have significant impacts
on refusal rates.
Both the demographic characteristics of respondents and interviewers have
significant impacts on refusal rates.
Null model
Level 1 model
model
Level 2 model
model Fixed effect Coefficients SE Coefficients SE Coefficients SE
respondent level
intercept,00 0.545** 0.060 0.702** 0.105 0.691** 0.093
career1,10 -1.974** 0.409 -2.056** 0.426
career2,20 -2.275** 0.440 -2.194** 0.426
career4,30 -2.767** 0.463 -2.928** 0.488
income,40 0.00016* 0.00007 0.00016* 0.00007
grade,50 1.266** 0.205 1.307** 0.214
interviewer level
W1,01 -0.28** 0.059
W2,02 -0.146* 0.059
career4W2, 32 1.136** 0.134
Survey type does not have significant effects on refusal rates in the
model
Respondent Interviewer
Conclusions1. Dual frame sampling is necessary for reducing coverage bias
for telephone surveys in China. The ineligible (e.g., non-working, business) rate in cell phone surveys appears to be lower than in landline surveys in China.
2. High non-response rates can be reduced by effective voice training of interviewers, more call attempts, and extending waiting time before giving up on the call.
3. The demographic characteristics and behaviors of cell phone users and landline telephone users are different, which may have implications for nonresponse bias.
4. Interviewers should wait longer for the respondent to answer the phone in the cell phone survey relative to the landline survey.
5. Refusals occur more quickly in the cell phone survey than in the landline survey.