a comparison of telephone and personal...
TRANSCRIPT
A Comparison of Telephone and Personal In te rv iewing
Tom W. Smith
Apr i l , 1984
National Opinion Research Center Univers i ty of Chicago
This r e p o r t was prepared f o r t he General Soc ia l Survey (GSS) Board of Overseers. The GSS is d i r e c t e d by James A. Davis and Tom W. Smith. The p r o j e c t is funded by the National Science Foundation, Grant No. SOC7.7-03279.
GSS : T-PIntervw
In our review of telephone-personal interviews w e examined
approximately 80 pieces of l i t e r a t u r e ( see bibl iography) and corresponded with
s i x colleagues about t h e i r experience using both modes. Approximately 32
s t u d i e s have compared telephone and personal interviewing. These include 21
complete o r p a r t i a l experimental comparisons of telephone and personal cross-
sec t ions , two experimental comparisons of reinterviews, two laboratory
experiments, three examinations of telephone and non-telephone households on
personal surveys, and four miscellaneous non-experimental s tud ies . Nine were
nat ional s tud ies , 14 communities o r s t a t e s , 7 s p e c i a l populat ions and sub-
communities, and two convenience samples. Health care ( 1 1 ) was the most
f requent ly inves t iga ted topic , followed by crime ( 3 ) , mental s t a t e s (31, c i t y
se rv ices ( 2 ) , p o l i t i c s ( 21 , and miscellaneous ( 1 1 - unions, c a f e t e r i a , r a c i a l
a t t i t u d e s , paper readership, e t c . )
I n the fol lowing sec t ion we w i l l compare the r e l a t i v e s t rengths and
weaknesses of personal and telephone interviewing, focusing on the themes of
q u a l i t y and comparability.
1 , A telephone survey would not cover the sample populat ion covered by
personal household surveys. About 7-9% of households a r e without
telephones. While t h i s number has been dec l in ing s t e a d i l y over the l a s t
seve ra l decades, the re is some specula t ion t h a t the trend may not
continue because of sa tu ra t ion . Also, some concern has been ra ised t h a t
the d i v e s t i t u r e of ATCT w i l l a c t u a l l y lower the telephone penetrat ion
r a t e , because of higher c o s t s and l imi ted service. Telephone coverage
is not uniform across subpopulations and is e s p e c i a l l y low among blacks,
the poor, r u r a l r e s iden t s , Southerners, and the t r a n s i e n t , Because of
the incomplete and d ispropor t ionate coverage by telephone, a switch t o
telephone surveys would mean t h a t one would e i t h e r have t o 1 ) redefine
GSS : T-PIntervw
t h e sample popula t ion and e i t h e r a ) r e s t r i c t a n a l y s i s of previous
personal surveys t o te lephone households only o r b ) a d j u s t the telephone
sample through some type of p o s t s t r a t i f i c a t i o n weight t o at tempt t o
r ep re sen t the genera l household popula t ion o r 2 ) adopt a dua l frame
des ign wi th a personal in te rv iewing component among non-telephone
households t o supplement t h e te lephone survey. (This would however
negate c o s t sav ings gained from telephone interviewing.
2. Response r a t e s a r e lower on the te lephone than i n person. In e i g h t
s t u d i e s t he personal response rate was c l e a r l y higher , i n s i x ca ses
pe r sona l response r a t e s were s l i g h t l y h igher , i n f i v e cases no
d i f f e r e n c e s were repor ted and i n two cases response r a t e s were s l i g h t l y
h ighe r on the telephone. The b e s t e s t ima te based p r imar i ly on t h e
exper ience of the SRC is t h a t a te lephone GSS would have a response r a t e
of f i v e t o t e n pe rcen t p o i n t s lower on the telephone. This is b a s i c a l l y
due t o h igher r e f u s a l and break-off r a t e s on telephone surveys.
Likewise i n a l l t en surveys t h a t compared r e f u s a l l e v e l s on an income
q u e s t i o n te lephone r e f u s a l s w e r e s u b s t a n t i a l l y higher . I t is a l s o
harder t o judge non-response b i a s i n telephone surveys s i n c e a t t r i b u t e s
of the household, neighborhood, e t c . can no t be known f o r
nonrespondents.
3. Telephone RDD surveys have sma l l e r design e f f e c t s than mul t i s tage ,
c l u s t e r e d personal surveys. The des ign e f f e c t s from more cases per
i n t e rv i ewer l eads t o a g r e a t e r i n t e rv i ewer design e f f e c t on telephone
in te rv iewing , bu t the e l imina t ion of a r e a c l u s t e r i n g l eads t o a lower
des ign e f f e c t o v e r a l l on the te lephone and t h e r e f o r e t o g r e a t e r sampling
e f f i c i e n c y .
4. Because it is much e a s i e r t o monitor in te rv iewers on a telephone survey,
the l e v e l of s t a n d a r d i z a t i o n is much higher . On the o t h e r hand, because
in te rv iews are in te rv iewing "b l indly" from usua l ly one l o c a l i t y , it is
d i f f i c u l t o r impossible t o match respondents r a c i a l l y o r reg iona l ly . In
add i t i on , because the telephone in te rv iewing s t a f f is more homogeneous
than a n a t i o n a l f i e l d s t a f f t he re may be more p o t e n t i a l f o r interviewer
b i a s a s opposed t o in te rv iewer variance.
5. Because of t h e more l i m i t e d flow of communication i n t he telephone mode,
respondent / interviewing feedback and c l a r i f i c a $ i o n s a r e more
d i f f i c u l t . Likewise, probing is found t o work more e a s i l y ' a n d flow more
n a t u r a l l y i n pe r sona l s e t t i n g s . This r e s t r i c t e d communication exchange
may c o n t r i b u t e t o c e r t a i n d i f f e r e n c e s i n response as covered below.
6, Ce r t a in t ypes of complex in te rv iewing can be handled b e t t e r on the
te lephone whi le o t h e r s a r e more d i f f i c u l t . Questions s t r i n g s involving
a long s e r i e s on complicated f i l t e r s and s k i p s can be handled v i r t u a l l y
e r r o r f r e e on CAT1 telephone interviews. It is more d i f f i c u l t , however,
f o r respondents t o understand and fo l low complex response t a sks on the
phone such a s rankings, random response ques t ions , magnitude measurement
s ca l ing , o r long ques t ions . There a r e a l s o c e r t a i n problems with
l i s t i n g even t s and behaviors and wi th open ended ques t ions t h a t w i l l be
covered below. Also, des igns t h a t c a l l f o r t h e respondent checking
c e r t a i n r eco rds (e.g. check s tubs , brand of appl iances , e t c . ) a r e
considered more d i f f i c u l t on the telephone. F i n a l l y , in te rv iewer
observa t ions of respondents , t h e i r household, o r t h e i r neighborhood a r e
n o t poss ib le .
7. Telephone in t e rv i ews tend t o be more cursory , less cogn i t i ve ly
s t imu la t ing , and more burdensome t o respondents than pe r sona l
G S S : T-PIntervw -5-
i n t e r v i e w s . A d m i n i s t r a t i o n of the same i n s t r u m e n t on the te lephone
t a k e s less t i m e t h a n i n person. Th is is t r u e even though temporary
i n t e r r u p t i o n s (e.g. by phone c a l l s , peop le i n household) a r e more
f r e q u e n t i n p e r s o n a l i n t e r v i e w s . The b r i s k e r pace o f t e lephone
i n t e r v i e w s seems t o r e s u l t from 1) less exchange between respondent and
i n t e r v i e w e r and 2 ) b r i e f e r o r less complete answers on t h e te lephone.
Open ended responses a r e l o n g e r on p e r s o n a l i n t e r v i e w s and more e v e n t s
o r b e h a v i o r s tended t o be mentioned. Respondents t end t o rate t h e
t e l e p h o n e i n t e r v i e w e x p e r i e n c e more n e g a t i v e l y (less e n j o y a b l e , less
l i k e l y t o d o a g a i n , more s u s p i c i o u s , e t c . ) t h a n p e r s o n a l i n t e r v i e w s .
T h i s d i f f e r e n c e is e s p e c i a l l y l a r g e when i t comes t o e v a l u a t i n g the
l e n g t h o f t h e i n t e r v i e w . While t h e t e lephone i n t e r v i e w a c t u a l l y r a n 20
minu tes s h 6 r t e r than the p e r s o n a l i n t e r v i e w i n t h e Groves t Kahn s t u d y ,
36 p e r c e n t r a t e d t h e t e l e p h o n e i n t e r v i e w as t o o l o n g w h i l e o n l y 10
p e r c e n t r a t e d the p e r s o n a l i n t e r v i e w as t o o long. T h i s g r e a t e r
r e s p o n d e n t burden i n a major f a c t o r c o n t r i b u t i n g t o much h i g h e r break-
o f f rates on t e l e p h o n e t h a n i n person.
8. The g r e a t e r anomin i ty o f t h e t e l e p h o n e i n t e r v i e w is o f t e n assumed t o
r e d u c e s o c i a l d e s i r a b i l i t y e f f e c t s . Exper imental r e s u l t s are e q u i v o c a l
a t best. S t u d i e s o f race o f i n t e r v i e w / r e s p o n d e n t i n t e r a c t i o n s i n d i c a t e
t h a t t h e e f f e c t is a s l a r g e i n t e l e p h o n e s u r v e y s as i n person. ( T h i s
w a s t r u e b o t h i n s t u d i e s t h a t t r i e d t o make r a c i a l i d e n t i f i c a t i o n c l e a r
t o r e s p o n d e n t s and i n n a t u r a l s i t u a t i o n s . ) S t u d i e s that compared l e v e l s
o f s o c i a l l y d e s i r a b l e r e s p o n s e s a c r o s s mode showed no c l e a r
d i f f e r e n c e s . When l e a n i n g s are c o n s i d e r e d , s i x s t u d i e s r e p o r t e d more
s o c i a l l y d e s i r a b l e r e s p o n s e s i n p e r s o n a l i n t e r v i e w s , two s t u d i e s w i t h no
d i f f e r e n c e s , and f o u r s t u d i e s w i t h more s o c i a l l y d e s i r a b l e r e s p o n s e s t o
t e l e p h o n e surveys . However, when w e look a t o n l y clear and s i g n i f i c a n t
GSS : T-PIntervw - 6-
d i f f e r e n c e s , i n only one survey d id personal in te rv iews ob ta in more
s o c i a l l y d e s i r a b l e responses, i n seven cases t he re were no d i f f e r ences ,
and i n four cases telephone surveys showed more s o c i a l l y d e s i r a b l e
responses. Of some s p e c i a l i n t e r e s t is the f ind ing of Bradburn, Sudman,
e t a l . t h a t a l l modes and methods of asking s e n s i t i v e ques t ions produced - l a r g e b iases .
9. Comparisons ac ros s modes have usua l ly been very l imi t ed , f r equen t ly
r e s t r i c t e d t o ana lyses of a few u n i v a r i a t e d i s t r i b u t i o n s . Comparisons
invo lv ing m u l t i v a r i a t e e f f e c t s , v a l i d i t y , and r e l i a b i l i t y have been very
scarce .
A. When comparisons a r e made between telephone households interviewed
by telephone and i n person, d i s t r i b u t i o n s a r e f r equen t ly
comparable. In eleven s t u d i e s no d i f f e r e n c e s were found, four
s t u d i e s found small and l i m i t e d d i f f e r e n c e s , and s i x s t u d i e s found
l a r g e o r widespread d i f f e rences . There appears t o be a tendency f o r
d i f f e r e n c e s t o be l e s s common on f a c t u a l i tems and demographics and
more f r equen t on a t t i t u d e s .and some types of behavior.
B. I n only s i x s t u d i e s were any more e l a b o r a t e o r complex comparisons
c a r r i e d out . Three s t u d i e s looked a t s c a l e r e l i a b i l i t y o r i n t e r -
i t e m c o r r e l a t i o n s . One s tudy found a c l e a r s u p e r i o r i t y f o r personal
i n t e rv i ews ( l e s s agreement b i a s , evasiveness , and extremeness) while
t he o t h e r found no d i f f e r e n c e i n t he r e l i a b i l i t y c o e f f i c i e n t o r
extremeness. In a t h i r d s tudy the in t e r - i t em c o r r e l a t i o n s on a
number of s a t i s f a c t i o n i tems were judged comparable ac ros s modes.
Two s t u d i e s compared t e s t / r e t e s t c o e f f i c i e n t s . I n one very small
s tudy , educat ion responses were more c o n s i s t e n t on te lephone
r e in t e rv i ews than on pe r sona l r e in t e rv i ews ( i n a l l c a s e s the i n i t i a l
i n t e rv i ew w a s i n person) . The d i f f e r e n c e s do no t appear
s i g n i f i c a n t , however. In the second s tudy r e l i a b i l i t y coe f f i c i en t s
were h i g h e s t i n t h e personal-personal mode, (.83, .88) intermediate
on the telephone-telephone mode (.82, .86) and lowest on switched
modes ( -71 , .75, .75, ,811. The d i f f e r e n c e s between the two
c o n s i s t e n t modes were not s i g n i f i c a n t . F i n a l l y , one s tudy found
t h a t te lephone in t e rv i ews underrepresented the less educated and
t h a t t h i s b i a s i nc reased s i g n i f i c a n t l y among the e l d e r l y . These s i x
s t u d i e s u s ing t h r e e s c a l e s and f i v e o t h e r v a r i a b l e s a r e a very
sparce b a s i s f o r g e n e r a l i z i n g about r e l i a b i l i t y and mul t iva r i a t e
d i f f e r e n c e s ac ros s modes. Four show no s i g n i f i c a n t d i f f e r ences , bu t
two f i n d b e t t e r performance from personal in te rv iews .
C. V a l i d i t y checks were c a r r i e d o u t i n nine in s t ances . Personal
surveys had more a c c u r a t e responses i n t h r e e cases , i n f i v e cases no b
d i f f e r e n c e s were de t ec t ed , and i n one case te lephone d id b e t t e r . On
four s t u d i e s t h a t assumed h i g h e r . v a l i d i t y based on t h e o r e t i c a l
a n t i c i p a t i o n of under o r over repor t ing , more complete r epo r t s were
found on te lephone i n two cases and i n two c a s e s f o r personal .
In very genera l terms the i n v e s t i g a t o r s ' summary conclus ions on da t a
q u a l i t y can be c l a s s i f i e d a s fol lows:
personal s u p e r i o r t o te lephone 8
Personal equal t o o r s u p e r i o r t o te lephone 3
No important d i f f e r e n c e s 12
Telephone equa l t o o r s u p e r i o r t o personal 2
Telephone s u p e r i o r t o personal 2
Telephone and pe r sona l should be combined (mixed mode)
Both. should be improved
The tilt of t he i n d i v i d u a l r e sea rch f ind ings towards b e t t e r q u a l i t y da ta
from personal i n t e rv i ews is shared by our summary of d i f f e r ences . on
the 25 s tandards l i s t e d i n Table 1 , we f i nd personal supe r io r i n
t h i r t e e n cases , te lephone i n s i x cases , i n s u f f i c i e n t d a t a i n four cases ,
and no d i f f e r e n c e i n two ins tances . Among the n ine teen a reas on which
d i f f e r e n c e s were de tec t ed , twelve are considered of importance t o the
GSS. On t he se twelve, personal in te rv iews had the advantage i n nine
c a s e s while te lephone in te rv iews lead i n t h ree a r eas .
10. Telephone surveys (and e s p e c i a l l y CAT1 surveys) can be done more quick ly
than personal in te rv iews . While personal i n t e rv i ews have been designed,
f i e l d e d , and completed wi th in a week, t h i s r e p r e s e n t s a ex t r ao rd ina ry
and usua l ly e s p e c i a l l y c o s t l y e f f o r t . Given a n t i c i p a t e d ' l e v e l s of
phys i ca l equipment and personnel , it would probably be poss ib l e t o
reduce the design t o d a t a process ing s t a g e from 6 months t o four - f ive
months. Advantages to the GSS of t h i s reduct ion , o t h e r than those t h a t
a r e p a r t of the c o s t sav ings l i s t e d below, a r e minimal however.
A l l s t u d i e s have shown telephone surveys t o be s i g n i f i c a n t l y l e s s
expensive than comparable personal interviews. Cost advantages
d iminish and/or d i sappea r , however, when dual frames a r e adopted
and i n des igns t h a t use some type of personal fo l low up (e.g.
matching h o s p i t a l r eco rds ) on s c a t t e r e d te lephone samples (e.g.
n a t i o n a l RDD). NORC e s t i m a t e s t h a t the t o t a l r educ t ion i n da t a
c o l l e c t i o n and process ing c o s t ( i . e . from ques t ionna i r e
cons t ruc t ion t o product ion of a c lean , raw d a t a t a p e ) would be 25
percent .
G S S : T-PIntervw -9-
What should be the course of a c t i o n f o r 19851 On one hand we would expect
t h a t wi th adjustments f o r non-telephone households most i tems would not
s i g n i f i c a n t l y m r y by mode. On the o t h e r hand, i n genera l the q u a l i t y of
personal in te rv iews seems h igher than te lephone in te rv iews and comparative
ana lyses of m u l t i v a r i a t e d i f f e r e n c e s and comparative r e l i a b i l i t y and v a l i d i t y
a r e too sparce f o r f i rm conclusions. Therefore, an experimental comparison
should focus on t h e s e underexamined a reas .
To adequate ly a s s e s s the comparative r e l i a b i l i t y of personal and
telephone in t e rv i ewing a t e s t / r e t e s t des ign should be adopted. Following our
exper ience wi th e a r l i e r t e s t / r e t e s t des igns , w e sugges t a s i n g l e r e in t e rv i ew
about four t o s i x weeks a f t e r the i n i t i a l interview. This r e t e s t should
involve both same mode and c r o s s mode admin i s t r a t i ons . The fol lowing t a b l e
i l l u s t r a t e s t h i s design:
Reinterview
None Telephone Personal
I n i t i a l
In te rv iew
Telephone
Personal
500 500 500 1500
This des ign would maximize our a b i l i t y t o s tudy the almost t o t a l l y
ignored a r e a of comparative r e l i a b i l i t i e s , b u t would be c o s t l y and n e c e s s i t a t e
a d d i t i o n a l resources. On the r e t e s t on ly a subsample of ques t ions would be
readministered. These would be chosen t o r ep re sen t s i x c l a s s e s of items:
1 ) unchanging demographics, c u r r e n t s t a t u s , 2 ) unchanging demographics, p a s t
s t a t u s , 3 ) changeable demographics, 4 ) s t eady a t t i t u d i n a l items, 5) one o r two
a t t i t u d i n a l s c a l e s , and 6) ep i sod ic and mood items. In a d d i t i o n , t o check the
q u a l i t y and r e l i a b i l i t y of the data, items should be added including 1 ) s e l f -
va l ida t ing f a c t u a l items (e. g., Who is the Secre tary of S ta te? ), 2 ) i terns with
a high degree of both p o s i t i v e and negative s o c i a l d e s i r a b i l i t y (e.g.,
r eg i s t e red t o vote/have l i b r a r y card vs. d r iv ing a f t e r dr inking/ la te paying
b i l l s ) . Serious cons idera t ion should be given t o including record checks t o
va l ida te r epor t s on voting, e t c . In addi t ion , w e should r e p l i c a t e the
f indings t h a t telephone interviews produce shor te r and less complete answers
t o open-ended ques t ions . Since both the personal reinterview of i n i t i a l telephone respondents
and any record checks necess i t a t e having an interviewer i n the v i c i n i t y and
would be e s p e c i a l l y expensive i f sca t t e red , it would probably be bes t t o
c l u s t e r telephone interviews i n the same PSUs a s the personal interviews.
Also, both &e r o t a t i o n of GSS items (which means t h a t roughly a t h i r d
of our a t t i t u d e i tems w i l l not appear on the 1985 survey) and our experience
with our f u l l probabi l i ty /b lock quota experiments i n 1975 and 1976 argue t h a t
the mode comparisons would have t o be rep l i ca ted i n 1986.
There is, however, a se r ious quest ion whether t h e GSS could be
administered on the telephone. Due t o respondent burden, success can not be
predic ted f o r a telephone survey the length of the GSS (approximately 75
1 minutes ) . ....................
'Examples of the adminis t ra t ion of general top ic surveys t o a na t ional o r heterogeneous populat ion l a s t i n g over an hour a r e extremely rare. Most telephone surveys a r e very shor t . Observe the following repor t s on length of telephone surveys :
Source Surveys Studied Length
1. Wiseman & McDonald, 1979 Market research Less than 10 minutes-40.3% g r e a t e r than 1 0 mins. 49.7%
2. Dillman, 1978 Mostly various popu- l a t i o n s , Washington
GSS: T-PIntervw
Our b e s t e s t ima te is t h a t a reasonable expec ta t ion of success could be
expected only i f t h e survey schedule was reduced by 50-60 pe rcen t (and a few
ques t ions dropped o r s u b s t a n t i a l l y reformatted t o make them workable on the
te lephone 1.
There a r e c e r t a i n r a d i c a l changes i n the GSS des ign t h a t could
accommodate such a d r a s t i c reduct ion i n the survey instrument . These would
inc lude 1 ) continuous c ros s - sec t ions ( a l a CNS), 2 ) multi-wave panels , 3) and
t h e sampling of i tems ac ros s t i m e . While these designs each have c e r t a i n
a t t r a c t i o n s , it is d i s q u i e t i n g t o s e e them considered p r i m a r i l y t o c a t e r t o
l i m i t a t i o n s of te lephone interviewing. In' add i t i on , it is doub t fu l t h a t any
of t hese des igns would b e t t e r monitor h i s t o r i c a l developments and he lp expla in
generat ion-to-generat ion s o c i a l change than the annual c ros s - sec t iona l des ign
of t he c u r r e n t GSS.
Thus, i n a t tempt ing t o implement an experimental comparison between
t h e te lephone and personal i n t e rv i ewing on the 1985 GSS w e are l e f t with t h ree
choices 1 ) a t t empt a GSS in t e rv i ew on the telephone d e s p i t e concerns about its
p r a c t i c a l i t y , 2) r a d i c a l l y r edes ign t h e GSS t o make it f u n c t i o n a l wi th in
te lephone l i m i t a t i o n s , and 3) contaminate t h e te lephone/personal comparison on
the experiment by admin i s t e r ing a s h o r t form on the te lephone ( a s i n the 1982
S t a t e 0-5 mins -1 4 5-1 0 -1 2 11-1 5 - 1 20-30 - 4
3. Survey Research, 1973 Academic research Longest eve r done: 5-60, average of longest : 26 mins.
S i m i l a r l y , i n the twelve exper imenta l s t u d i e s , which mentioned l eng th of interview, covered he re i n only one case d i d t h e t i m e exceed 50 minutes on t h e telephone. I n a d d i t i o n , most p r a c t i o n e r s a r e wary about long telephone in te rv iews . Survey Research, i n its p o l l of 14 academic survey organiza t ion , found only one prac t ioner who thought t h a t a survey over 45 minutes was poss ib le . ( H e a c t u a l l y s a i d the re was n o t l i m i t i f t he survey was i n t e r e s t i n g ) . Many au tho r s i n t h e experimental s t u d i e s a l s o expressed concern about long telephone interviews.
GSS : T-PIntervw -1 2-
ANES experiment). (This l a s t choice presumes t h a t a f u l l GSS is not p r a c t i c a l
on the telephone and i n d i r e c t l y argues f o r u l t imate adoption of the second
option. 1
why ca r ry out a telephone/personal experiment? Cer ta in ly there a re
many methodological quest ions about personal/telephone interviewing t h a t the
design w e proposed would g r e a t l y c l a r i f y . Y e t it is doubtful t h a t such an
expensive and extens ive methodological experiment should be c a r r i e d out
without the se r ious in ten t ion of moving the GSS t o the telephone mode. Since
the re a r e no conceptual o r t h e o r e t i c a l reasons f o r t h i s s h i f t , it a l l comes
down t o money. Our best es t imate is t h a t a f t e r our two experimental years
(which would n e c e s s i t a t e an increase i n our budget of some unknown amount),
the telephone survey would save us 25 percent i n c o l l e c t i o n and processing
cos ts . his would mean t h a t the t o l d p r o j e c t savings ( inc luding codebook
prepara t ion , ana lys i s , user support, e t c . ) would be on the order of 18
percent. I f the GSS survey could be c a r r i e d out i n t a c t with a s high a degree.
of q u a l i t y i n the telephone mode a s i n the personal mode, we would be foo l i sh
t o ignore these savings. Unfortunately the f e a s i b i l i t y of doing the necessary
experimental comparison t o determine the r e l a t i v e r e l i a b i l i t y , v a l i d i t y , and
. r e p l i c a b i l i t y of the two modes is quest ionable. .
GSS: T-PIntervw -1 3-
TABLE 1
Comparison of Telephone and Personal In te rv iews
Magnitude of Important
Topic Advantage Advantage t o GSS? Comments
.I. SAMPLING
A. Complete Coverage P Strong Yes 7-9% of households without phones undercoverage much high i n some subpopulation
B. S e l e c t i n g Respondent N -- D i f f e r e n t techniques a r e used, b u t R can be s e l e c t e d
C. Nonresponse P S t rong Y e s
D. Design E f f e c t
11 INTERV IEWING
S t rong Y e s
A. In te rv iewer S t anda rd iza t ion T S t rong Yes
B. In te rv iewer x Respondent I n t e r - a c t i o n s U - - Race of in te rv iewer /
respondent i n t e r a c t i o n s happen on both T and P , b u t race matching not a t tempted on T r e g i o n a l / l i n g u i s t i c matches d i f f i c u l t on T
C. Respondent Feedback/ c l a r i f i c a t i o n s P
D. Probing P
E. Complex Ques t i o n s P
1. V i s u a l a i d s P
Yes
Y e s
Yes
Strong
S l i g h t
S t rong
St rong Yes I n some, bu t not a l l i n s t ances , d i f f e r e n t T techniques can handle wi thout v i sua l a id s
2. Complicated f i l t e r s Strong N o T advantage i f CAT1 only;
given c u r r e n t GSS con ten t manual f i l t e r i n g s works w e l l
3. Open ended P S t rong
4. L i s t s of events , exper iences P
5. Poss ib l e con- s u l t i n g of r eco rds , family P
F. Respondent Pr ivacy T
S t rong S l i g h t
No
S l i g h t
S t rong
S l i g h t
G S S : T - P I n t e r v w -1 4-
TABLE 1
(Continued) - -
Magnitude of Important
Topic Advantage Advantage t o GSS? Comments
111 . RESPONSES
A. General (non th rea t ing ) N - - Dis t r ibu t ions usua l ly not s i g n i f i c a n t l y d i f f e r e n t
B. Soc ia l D e s i r a b i l i t y U/N -- - - C. Va l ida t ion P S l i g h t S l i g h t
D. R e l i a b i l i t y U -- -- V i r t u a l l y unexamined
E. I t e m Nonrespbnse
1. Income Refusa ls
IV. OTHER DESIGN ISSUES
A. Quest ionnaire Length
B. Respondent ~ u r h e n
V. SPEED
V I . COST
U/N - - Unsure
P Strong Yes
P Strong Yes
P Strong Yes
T Strong No
T . Strong Yes
T g e t s s l i g h t l y higher DKs, d i f f i c u l t t o judge a s p o s i t i v e o r nega t ive development
T in te rv iew runs f a s t e r bu t lower acceptance of long in te rv iews
R p r e f e r s P t o T and r a t e s T a s too long and more of burden'
Dual frame designs o f ten negate T savings
- - - - -- -- - - -
P = Personal N = No d i f f e r e n c e
T = Telephone U = Unknown
TELEPHONE BIBLIOGRAPHY
Anderson, C.-Leroy, and Halford, Larry J., "A Four-State Comparisun of V a r i a b l e Sampling and Data C o l l e c t i o n Procedures," P a c i f i c Soc io logica l Review, 13 (Summer, 19701, 149-1 55.
Aneshensel, Carol S.; F re r i chs , Ralph R e , Clark, V i r g i n i a A * , and Yokopenic, P a t r i c i a A * , "Measuring Depression i n t he Community: A Comparison of Telephone and Personal In te rv iews , ' POQ, 46 (Spring, 19821, 110-121. -
Assael, Henry, "Comparison of Brand Share Data by.Three ~ e p o r t i n g Systems," J o u r n a l of Marketing Research, 4 (November, 19671, 400-401.
B i e m e r , Paul P:, "Optimal Dual Frame Design: Resu l t s of a Simulat ion Study," Paper presen ted t o t he American S t a t i s t i c a l Assoc ia t ion , Toronto, August, 1983.
Blankenship, A. B., P ro fe s s iona l Telephone Surveys. New York: McGraw-Hill, 1977.
Bradburn, Norman M.; Sudman, Seymour; and Assoc ia tes , Improving In te rv iew Method and Quest ionnaire Design: Response E f f ec t s to Threa ten ing Questions i n Survey Research. San Francisco: Jossey-Bass Pub l i she r s , 1979.
Bushery, John M.; Cowan, Char les D.; and Murphy, Linda R., "Experiments i n Telephone-Personal V i s i t Surveys," Paper presen ted t o the American S t a t i s t i c a l Assoc ia t ion , San Diego, August, 1978.
~ a h a l a n , Don, "Measuring Newspaper Readership by Telephone: Two Comparisons wi th Face-to-Face Interview's, " Journa l of Adver t i s ing Research, 1 - (19601, 1-6.
Cannel l , Char les F.; Groves, Robert M.; Magilavy, Lou J.; Mathiowetz, Nancy; and Miller, P e t e r V., "An Experimental Comparison of Telephone and Persona l Health Surveys,' Unpublished paper, Survey Research Center , Un ive r s i t y of Michigan, Apr i l , 1983.
Cannel l , Char les F., and Fowler, Floyd J., " In te rv iewers and In te rv iewing Techniques," i n Advances i n Heal th Survey Research Methods: Proceedings of a Nat iona l I n v i t a t i o n a l Conference, NCHSR Research Proceedings S e r i e s , ' 1975.
C a t l i n , Gary and Murray, Susan, "Canadian Vic t imiza t ion Surveys: A Report on P r e t e s t s i n Edmonton and Hamilton, " Survey Methodology, 5 (December, 1979).
Colombotos, John, "The E f f e c t s of Persona l vs. Telephone In te rv iews on S o c i a l l y Acceptable Responses," POQ, 19 ( F a l l , 19651, 457-458.
Colombotos, John, "Personal versus Telephone Interviews: E f f e c t on Responses," Pub l i c Health Reports , 84 (September, 19691, 773-782.
Coombs, Lolagene and Freedman, Ronald, "Use of Telephone Interviews i n a Lonqitudinal F e r t i l i t y Study," 28 (Sprinq, 1964), 112-117.
Co t t e r , Pa t r i ck R.; Cohen, Je f f rey ; and Coulter , P h i l i p B., "Race-of- Interviewer Effec ts i n Telephone Interviews , " POQ, 46 (Summer, 1982), - 278-2040
Czaja, Ronald; Bla i r , Johnny; and Sebest ik, Ju t t a P., "Respondent Select ion i n a Telephone Survey: A Comparison of Three Techniques," Journal of Marketinq Research, 19 (August, 19821, 381 -385.
Dillman, Don A., Mail and Telephone Surveys: The Tota l Desiqn Method. New York: John Wiley & Sons, 1978.
Frankel, Martin R., and Frankel, Les ter R., "Some Recent Developments i n Sample Survey Desiqn," Journal of Marketinq Research, 14 (Auqust, 19771, 280-293
Frey, James H., Survey Research by Telephone. Saqe Library of Socia l Research, Vol. 1 50. Beverly H i l l s : Sage ~ u b l i c a t i o n s , 1983.
Friedman, Linda, and Friedman, Hershey H., "Does the Preceived Race of a Telephone Interviewer Affect the Responses of W h i t e Subjec ts t o an At t i tudes toward Neqroes Scale?," Paper presented t o the American
. S t a t i s t i c a l Association, San Dieqo, August, 1978.
Groves, Robert M., "Actors and Questions i n Telephone and personal ~ n t e r v i e w Surveys," POQ, 43 (Summer, 19791, 190-205. -
Groves, Robert M., "Comparinq Telephone and Personal Interview Surveys," Economic Outlook USA, Summer, 1978, 49-51. -
Groves, Robert M., "An Empirical Comparison of Two Telephone Sample Designs," Journal of Marketing Research, 15 (November, 19781, 622-631.
Groves, Robert M., "An Experimental Comparison of National Telephone and Personal Interview Surveys," Paper presented t o the American S t a t i s t i c a l Association, 1977.
Groves, Robert M., "Implications of CATI: Costs, Errors , and Orqanization of Telephone Survey Research," Socioloqica l Methods and Research, 1 2 (November, 19831, 199-21 5.
Groves, Robert M., "A Researcher's V i e w of the SRC Computer-Based Interviewinq System: Measurement of Some Sources of Error i n Telephone Survey Data," ???
Groves, Robert M.; Berry, Marianne; and Mathiowetz, Nancy, "Some Impacts of Computer Assisted Telephone Interviewing on Survey Methods," Paper presented t o the American S t a t i s t i c a l Association, Auqust, 1980.
Groves, Robert M., and Kahn, Robert L., Surveys by Telephone: A National Com~arison with Personal Interviews. New York: Academic Press, 1979.
Groves, Robert Y., and Lepkowski, James M., "Dual Frame, Mixed Mode Survey Desiqns," Unpublished paper, Survey Research C e n t e r , University of Michigan.
Groves, Robert M., and Magilavy, Lou J., "Estimates of Interviewer Variance in Telephone Surveys," Paper presented t o the American S t a t i s t i c a l Association, 1980.
Groves, Robert M., and Magilavy, Lou J., "Increasing Response Rates to Telephone Surveys: A Door i n the Face f o r Foot-in-the-Door?, " POQ, 45 - ( F a l l , 19811, 346-358.
Groves, Robert M., and Magilavy, Lou J., "Interviewer Ef fec t s i n Telephone Survey Data," Unpublished paper, Survey Research Center, University of Michiqan.
Groves, Robert M., Yaqilavy, Lou J., and Bia lecki , Anthony, "HIS/SRC Telephone Experiment: Report on Samplinq Ac t iv i t i e s , " unpublished paper, Survey Research Center, Universi ty of Michigan, 1980.
Groves, Robert Y., Magilavy, Lou J., and Mathiowetz, Nancy, "Comparison of Results between Cornouter Assisted Telephone Interviewing (CAT11 and Paper and Penci l Quest ionnaires," Report t o the National Center f o r Health S t a t i s t i c s , September, 1981.
Groves, Robert Y., Maqilavy, Lou J., and Mathiowetz, Nancy A., "The Process of Interviewer Var iab i l i ty : Evidence from Telephone Surveys," Paper presented to the American S t a t i s t i c a l Association, 1981 .
Groves, Robert M., and Mathiowetz, Nancy A., "Computer ~ s s i s t e d Telephone Interviewinq: Ef fec t s on Interviewers and Respondents," Unpublished paper, Survey Research Center, University of Michigan.
Groves, Robert M.; Miller, Pe te r V.; and Cannell, Charles F., "A Methodolo- q i c a l Study of Telephone and Face-to Face Interviewinq," Paper presented to the American Associat ion f o r Public Opinion Research, Buck H i l l Fa l l s , Pa, 1981.
Groves, Robert M.; Miller, Pe te r V.; and Handlin, V e l m a J., "Telephone Survey Methodoloqy: A Review, " Unpublished paper, Survey Research C e n t e r , Universi ty of Michiqan, 1982.
Hauck, Mathew and Goldberq, J e f f r e y , "Telephone Interviewinq on the NLRB Elect ion Study," Survey Research, 5 (Jan., 19731, 11-16.
Henson, Ramon; Qoth, Aleda; and Cannell, Charles F., "Personal versus Telephone Interviews and the Ef fec t s of Telephone Reinterviews on Reporting of Psych ia t r i c Symptomatolog~," Survey Research Center, Universi ty of Michiqan, October, 1 974.
Herman, Jean Bre t t , "Mixed-Mode Data Collect ion: Telephone and Personal Interviewinq," Journal of Applied Psycholoqy, 6 2 (19771, 399-404.
n e r d ~ q , 3 . K e q u ~ d ; fi.oaonger3, n l ~ L a r 2 L* ; dnU kCLIICa, K l C f l a r n A., i f iZ?. ivle iJ lnq Older ~ d u l t s : A Comparison of Telephone and Face-to-Face Modalities," POQ, 47 ( F a l l , 19831, 405-418. -
Hochstim, Joseph R., "A Cri t ical Comparison of Three S t r a t e q i e s of Collect ing Data from ~ouseho lds , " Journal of the American S t a t i s t i c a l Association, LXII (Sept., 19671, 976-989.
Ibsen, Charles A., and Ballweg, John A., "Telephone ~ n t e r v i e w s i n Social R$search: Some Methodoloqical Considerations," Quality and Quantity, 8 (dune, 19741, 181-192.
"Interviewing i n Telephone Surveys," Survey Research, 5 (Jan., 1973), 9-13.
Janofsky, A. I rene, "Affective Self-Disclosure i n Telephone versus Face-to- Face Interviews, " Journal of Humanistic Psycholoqy, 11 (Spring, 1971 ) , 93-1 03.
Jordan, Lawrence A*; Yarcus, Alfred C.; and Reeder, Leo G., "Response Styles i n Telephone and Household Interviewing: A F ie ld Experiment," POQ, 44 - (Summer, 19801, 210-222.
Yegeles, S. Stephen; Fink, Clinton F.; and Kirscht , John P., "Interviewing a National Sample by Long-Distance Telephone, POQ, 33 ( F a l l , 19691, 41 2- - 41 9.
Klecka, William R., and n tchfarber , Alfred J., "Random D i q i t Dialing: A Comparison to Personal Surveys," POQ, 42 (Spring, 19781, 105-114. -
Larsen, Otto N., "The Comparative Va l id i ty of Telephone and Face-to-Face Interviews i n the Measurement of Messaqe Diffusion from Leaf le ts ," American Socioloqica l Review, 17 (Auqust, 1952), 471-476.
Locander, William B., and Burton, John P., "The Ef fec t of Question Form on Gatherinq Income Data by Telephone," Journal of Marketing Research, 13 (May, 19761, 189-1 92.
Locander, W i l l i a m ; Sudman, Seymourt and Bradburn, Norman, "An Inves t iga t ion of Interview Method, Threat and Response Dis to r t ion , " Journal of the American S t a t i s t i c a l Association, 71 (June, 19761, 269-275.
Lucas, William I., and Adams, W i l l i a m C., An Assessment of Telephone Survey Methods. Santa Monica. Cal.: Rand C o r ~ o r a t i o n , 1977.
Mathiowetz, Nanw A., and Cannell, Charles F., "Codinq Interviewer Behavior as a Method of Evaluating Performance," Paper presented t o the American S t a t i s t i c a l Association, 1980.
Monteiro, Lois A., Moni to r inq Health S ta tus and Medical Care. Cambridge, Mass.: Bal l inqer , 1976.
Mulry-Liggan, Mary Ha, "A Comparison of a Random D i g i t Dialing'Survey and the Current Population Survey," Paper presented t o t h e American S t a t i s t i c a l Association, Toronto, Auqust, 1984.
Nealon, Jack, "The Ef fec t s of Male vs . Female Telephone In te rv iewers , S t a t i s t i c a l Reportinq Serv ice of t he U.S. Department of Aqr icu l ture , S t a f f - Report No. AGE83061 7.
Oakes, Ralph H., "Differences i n Responsiveness i n Telephone versus Personal interview^,^^ Journa l of Marketinq, 19 (1954), 169.
O ' N e i l , Michael J., and Groves, Robert M., "Telephone ~ n t e r v i e w In t roduct ions and Refusal Rates: Experiments i n Increas inq Respondent C o o ~ e r a t i o n , ~ ~ Survey Research Center , Un ive r s i t y of Michigan, Auqust, 1979.
OIRourke, Diane and B l a i r , Johnny, "Improving Random Respondent Se l ec t ion i n Telephone Surveys," Jou rna l of Marketinq Research, 20 (November, 1983), 428-4320
Quinn, Robert P.; Gutek, Barbara A. ; and Walsh, J e f f r e y T., "Telephone Interviewing: A Reappra isa l and a F i e l d ~ x p e r i m e n t , " Basic and Applied S o c i a l Psycholoqy, 1 (1980), 127-153.
Rogers, Theresa F., "Interviews by Telephone and i n Person: Qual i ty of Responses and F i e l d Performance,' POQ, 40 (Sprinq, 19761, 51-65. -
Roshwalb, I rv ing ; Spector , Leonard; and Madansky, Alber t , "New Methods of Telephone In te rv iewing A & S/CATI, " ???
Salmon, Cha r l e s T., and Nichols, John Sp ice r , "The ~ e x t - B i r t h d a y Method of Respondent Se lec t ion , ' POQ, 47 (Summer, 1983), 270-276. -
San Auqustine, Andre J., and Friedman, Hershey Y., "The Use of t h e Telephone In te rv iew i n Obtaininq Information of a S e n s i t i v e Nature: A Comparative Study," Paper presented t o t h e American S t a t i s t i c a l Associat ion, San Diego, Auqust, 1978.
Schiedskamp, Jay W., "Reinterviews by Telephone," Jou rna l of Marketinq, 26 (Jan., 19621, 28-34.
Shanks, J. Merrill; Sanchez, Maria; and Morton, Betsey, "Al t e rna t ive Approaches t o Survey Data Co l l ec t ion f o r t h e Nat iona l E lec t ion S tud ie s , " Report t o NES Board of Overseers, March, 1983.
Shee ts , Thomas; Radl insk i , Allen; Kohne, James; and Brunner, Allen, "Deceived Respondents: Once B i t t en , m i c e Shy," POQ,, 38 (Summer, 1974), 261-263. -
Shih, Wen-Fu P., 'Nonresponse to Income Questions i n Telephone Surveys, "Paper presented t o the American S t a t i s t i c a l Associat ion, Toronto, August, 1984.
Siemiatycki , Jack, "A Comparison of Mail, Telephone, and Home In terv iew S t r a t e q i e s f o r Household Heal th Surveys, American Jou rna l of Pub l i c Heal th, 69 (March, 19791, 238-245.
Sykes, Wendy and Hoinvi l l e , Gerald, " ~ e t h o d o l o q i c a l Research on Telephone . In te rv iews ," Paper presented t o t h e Market Research Socie ty , March,
1984.
mornbe r ry , Owen T., "An valuation of Three S t r a t e q i e s f o r t h e Col lec t ion of Data from Households," Unpublished d i s s e r t a t i o n , Brown Universi ty , 1976.
Thornberry,. Owen T., and Massey, James T O , "Correct ing f o r Undercoveraqe Bias i n Random D i q i t Dialed Nat iona l Health Surveys," Paper presented t o the American S t a t i s t i c a l Associat ion, 1978.
mornbe r ry , Owen T., and Massey, James T., "Coverage and Response i n Random D i g i t Dialed National Surveys," Paper presented t o t he American S t a t i s t i c a l ~ s s o c i a t i o n , 1983.
~ u l l , Donald S., and Albaum, Gerald S., " B i a s i n Random ~ i q i t Dialed Surveys," POQ, 41 ( F a l l , 1977), 389-395. -
White, Andrew A., "Response Rate Ca lcu la t ion i n RDb ~ e l e p h o n e Health Surveys: Current P rac t i ce s , " Paper presented t o t he American S t a t i s t i c a l Associat ion, 1983.
W i l l i a m s , Fderyn, "Medium of Messaqe: Communications ~ e d i u m as a Determinant of I n t e r p e r s o n a l Evaluat ion," Sociometry, 38 (1 9751, 11 9-1 30.
Wiseman, Freder ick , "Methodoloqical B i a s i n Publ ic Opinion Surveys," POQ 36 (Spring, 1 972 1. 1 05-1 05.
Wiseman, Freder ick , and McDonald, P h i l i p , "Noncontact and Refusa l Rates i n Consumer Telephone Surveys, " ~ o u r n a l of Marketinq ~ e s e a r c h , 1 6 (November, 19791, 478-484.
Wolf le, Lee M., " C h a r a c t e r i s t i c s of Persons With and Without Home '~e l ephones , Journa l of Marketinq Research, 16 (August, 19791, 421-4250
Yaffe, Qichard; Shapiro, Sam; Fwhsberg, Robert R.; Rohde, Char les A.; and Corpeno, Helen C., "Medical Economics Survey-Methods Study: Cost- E f fec t iveness of A l t e rna t ive Survey s t r a t e g i e s , " Medical Care, 16 (Auqust, 19781, 641 -659.