mixed mode in lfs: questionnaire design and mode-effects
DESCRIPTION
Mixed mode in LFS: questionnaire design and mode-effects. Liisa Larja, Statistics Finland 9th Workshop on Labour Force Survey Methodology Rome, 15 th of May 2014. StatFin/EssNet web-project:. Designing LFS-CAWI questionnaire + iterative testing with cognitive interviews (N = 22) - PowerPoint PPT PresentationTRANSCRIPT
Mixed mode in LFS: questionnaire design and mode-effects
Liisa Larja, Statistics Finland
9th Workshop on Labour Force Survey Methodology
Rome, 15th of May 2014
StatFin/EssNet web-project:
1. Designing LFS-CAWI questionnaire + iterative testing with cognitive interviews (N = 22)
2. Pilot survey (N = 8 000), experimental design (”split-plot”) with DK-option, working time and job search questions. Comparison to CAWI data from the same period. Paradata, respondent feedback.
Content of the presentation:
1. Questionnaire design: results & recommendations
2. Data collection process
3. Mode-effects: WSTATOR (employment status)
4. Mode-effects: HWACTUAL (actual working hours)
5. Qualitative feedback
1. Questionnaire design: layout
BlaiseIS default layout was not perceived as attractive Number of question / page Navigation bars Instructions Error messages Don’t know –option (DK)
Error message, DK
Instructions, info-button
Status bar
Number of questions, updating the remaining questions
Scrolling down for navigation bars
When the cursor points the info-symbol, the instructions appear in a box below.
Navigation bars:Takaisin = Back Jatka = ContinueJatka myöhemmin = Continue later/pause
Kyllä = YesEn = No(Don’t know not visible)
Kyllä = yesEi = noEn osaa sanoa = Don’t know
An error message: Please respond to this question or choose the “Don’t know” option.
DK-option, version 1
Kyllä = yesEi = noEn osaa sanoa = Don’t know
DK-option, version 2
2. Data collection process
Response rate 30 % potential for savings in data collection costs when the sample is large enough
CAWI-data more ”distorted” ? CAPI non-response more evenly distriputed with regard to educational background, gender and age than CAWI-data.
Speed: 40 % responses were received withing 3 days after that switch to CAPI?
3. Mode-effects: WSTATOR (employment status)
3. Mode-effects: WSTATOR (employment status)
No significant mode-effects, but….• media attention remains if unemployment rate 8,2 % instead of 7,4 %• sample for pilot study was small• CAWI respondents answer more often that they have been looking for job• = mode effect or selection bias?
4. Mode-effects: HWUSUAL (actual working hours)
”One question” –format for HWACTUAL:
“How many hours did you work in your main job during the week 7.10.-13.10.2013?”
___ hours ___ minutes
”Grid” –format for HWACTUAL:“How many hours did you work in your main job during the week 7.10.-13.10.2013?”
Date Hours Minutes
Monday 7.10.2013 ___ ___
Tuesday 8.10.2013 ___ ___
Wednesday 9.10.2013 ___ ___
Thursday 10.10.2013 ___ ___
Friday 11.10.2013 ___ ___
Saturday 12.10.2103 ___ ___
Sunday 13.10.29013 ___ ___
Total ___ ___
Average number of hours worked (CATI-data and two different CAWI-datas)
5. Qualitative feedback
Too intimate information on employer
Questions do not fit for people with multiple jobs without main job or with scholarship
All kind of entrepreneurial activity is not covered
Working time items do not fit for people in atypical employment
Pensioners do not understand point of reference week
Conclusions and lessons learnt
Response rate of 30 % and speed of data collection encourage to continue towards mixed-mode LFS.
Visual design is important: balanced layout, intuitive navigation bars, informative error messages, a few questions per page, hidden instructions, hidden DK.
CAWI data more ”distorted”
In working time questions different format for CATI and CAWI may be needed to reduce mode-effects (”functional equivalency”). Differences in WSTATOR need to be further analysed.
