using behavior coding to evaluate the effectiveness of dependent interviewing joanne pascale quest...
DESCRIPTION
Dependent Interviewing (DI) Used in longitudinal studies Carries reports from one wave into the next interview wave E.g.: ‘Last time you said you were receiving benefit X. Are you still receiving benefit X?’TRANSCRIPT
Using Behavior Coding to Evaluate the
Effectiveness of Dependent Interviewing
Joanne PascaleQUEST Conference
Ottawa, CanadaApril 26, 2007
Outline
• Dependent interviewing• Research questions addressed with
behavior coding• Methods• Findings• Summary and Recommendations
Dependent Interviewing (DI)
• Used in longitudinal studies• Carries reports from one wave into the
next interview wave• E.g.: ‘Last time you said you were
receiving benefit X. Are you still receiving benefit X?’
Purpose of DI
• Potential for smoother, smarter, more efficient interview
• Literature demonstrates:– Reduced burden– Reduced item non-response– Reduced seam bias– Respondents want and expect it
Research Questions
How is DI being implemented in the field:• Do interviews read questions as
worded?• Do respondents affirm or dispute the
fed-forward data from prior wave?• Does this vary by ‘style’ of DI?
Methods: The ELSA Study
• English Longitudinal Study on Aging (ELSA)
• Panel study of 50+ begun in 2002• Follow-ups every 2 years• CAPI face-to-face
Methods: The ELSA Pilot
• Vehicle for DI test: pilot of Wave 3• January, 2006• 4-week field period• Recorded onto laptops using CARI• 17 interviewers• 104 pilot interviews
Methods: DI Topics and Styles
• 3 different topic areas:– Demographics– Health conditions
• Eye (e.g. glaucoma)• Cardio-vascular (e.g. high blood pressure)• Chronic (e.g. arthritis)
– Vehicle ownership• 5 different styles of DI
Demographics Item Wording
1. Does John still live here?2. Can I just check, is John’s date of
birth January 1, 1960?
3. Our records show that when we last interviewed you, you had a child called Billy, whose date of birth is July 1, 2005. Are these details correct?
Health Item Wording
4a. Our records show that when we last interviewed you in [DATE], you said that you had [EYE CONDITION].
4b. Do you still have [EYE CONDITION]?
[same routine for CVD and chronic conditions]
Vehicle Item Wording
5. At the time of last interview, you or someone in your household owned [VEHICLE]. Is that vehicle still owned by you or someone in your household?
Behavior Coding• Listened to several pilot tapes to
develop code frame• First-level exchange sufficient• Developed Interviewer, Respondent
and Outcome codes
Interviewer Codes
Read as worded/slight changeMajor change FF statement became a question FF question became a statement Other major changeOmittedInaudible Interviewer/Other
Respondent Codes
Adequate Affirmed FF item
Disputed FF itemInadequate Answer/ElaborationClarification
Inaudible Respondent/Other
Demographics FindingsInterviewer Behavior Respondent
BehaviorExact/slight change
Q read as statement
Q omitted
Adequate Affirmed
AdequateDisputed
Does NAME still live here? 40 33 18 81 1
Can I check, is NAMES’s DOB? 57 37 1 91 0
Our records show child’s name, DOB. Details correct?
79 8 0 89 5
Health FindingsInterviewer Behavior R Behavior
Exact/slt change
S read as Q
Q read as S
Omitted Adequate Affirmed
Adequate Disputed
Last time you reported EYE cond 62 38 -- 0 62 10Do you still? 63 -- 16 16 -- --Last time you reported CVS cond 63 20 -- 0 87 5Do you still? 76 -- 3 13 -- --Last time you reported CHRONIC cond
41 34 -- 5 85 4
Do you still? 61 -- 14 18 -- --
Vehicle Findings
Interviewer Behavior Respondent Behavior
Exact/slight change
Q read as S
Omitted Adequate Affirmed
Adequate Disputed
Last time you owned …vehicle. Still own? 82 8 4 74 6
Summary: Interviewers
• Qs read as worded: 40-82%• Non-standard reading varied:
– Demographics & vehicle: questions turned into statements or omitted
– Health: statements of what was reported last time were turned into questions; question “is that still” omitted
Summary: Respondents
• Adequate answer: 72-94%• Disputed fed-forward data: 0-10%
– Confirmed prior wave report but said no longer the case
– Denied prior wave report– Disagreed with details of prior wave report
Recommendations
• Avoid simple statements of prior wave data
• For topics unlikely to change over time (e.g.: DOB) avoid re-asking; verify accuracy of first report
• Style of DI needs to be tailored to item• Prior wave data could be challenged;
need to allow for corrections