matea paškvan introducing a push-to-web sarah hartleib … · 2019-11-28 · matea paškvan. sarah...
TRANSCRIPT
![Page 1: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/1.jpg)
www.statistik.at We provide information
Introducing a Push-to-Web Initiative for the Austrian Labour Force Survey: Who Can Be Reached and the Effects of Different Communication Strategies
Matea PaškvanSarah Hartleib
Statistics Austria
UNECE, Geneva14 – 16 October 2019
![Page 2: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/2.jpg)
www.statistik.at slide 2 | 7 October 2019
The Austrian LFS under construction…
Pilot Survey 2020
Pilot Survey 2019
CAWI / CATI / CAPI
Break in time seriesCommunication adapted
Web-friendly survey
Push – to – CAWICommunication strategy
![Page 3: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/3.jpg)
www.statistik.at slide 3 | 7 October 2019
Aim of the Pilot 2019
• Push as many households as possible to CAWI
• Test the „new“ communication-strategy
• Test the workflow introducing CAWI for the first time
Paradata
Surveydata
Interviews – Debriefing Calls
![Page 4: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/4.jpg)
www.statistik.at slide 4 | 7 October 2019
How it is done before vs. Pilot2019
1st Wave 2nd Wave 3rd - 5th Wave
2nd Wave1st Wave550 households
3rd - 5th Wave
Not (yet) tested
Pilot Survey 2019
Usual LFS
![Page 5: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/5.jpg)
I. HOW MANY HOUSEHOLD CAN BE REACHED VIA CAWI?
![Page 6: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/6.jpg)
www.statistik.at slide 6 | 7 October 2019
I. How many household can be reached via CAWI?
• CAPI first wave – 550 Households
• Second wave – 451 (82%) Households assigned to CAWI
• Introduction of CAWI reduced CATI rates from 70 % to 24%
82%
55%
11%
24%
70%
5%
11%20%
2%11% 10%
0%
20%
40%
60%
80%
100%
CAWI Pilot - AssignedMode
CAWI Pilot - FinishedMode
Usual LFS - FinishedMode
4 Not finished & INT
3 CAPI
2 CATI
1 CAWI
![Page 7: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/7.jpg)
II. WHICH CHARACTERISTICS ON THE HOUSEHOLD- AND PERSONAL LEVEL ARE RELATED TO CAWI COMPLETION?
![Page 8: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/8.jpg)
www.statistik.at slide 8 | 7 October 2019
II. Which characteristics are related to CAWI completion?
• The pilot shows that CAWI completion is mostly dependent on
if the household has access to the internet or if the contact person was willing to provide the email
address
• Internet literacy and use of a specific device (laptop / desktop / smartphone) are less important
• Socio-demographic characteristics of the contact persons are more important compared to characteristics of the household (aggregated on household-level)
![Page 9: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/9.jpg)
www.statistik.at slide 9 | 7 October 2019
II. Which characteristics are related to CAWI completion?
Do not measure a bunch of additional items to identify CAWI willing households!
Solely ask if an
internet access is available and if this is the case
push those household to CAWI
still using a sequential multi-mode-design
![Page 10: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/10.jpg)
III. WHICH SEQUENTIAL COMMUNICATION STRATEGY IS THE BEST TO FOSTER TIMELINESS OF RESPONSES AND TO INCREASE CAWI RESPONSE RATES?
![Page 11: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/11.jpg)
www.statistik.at slide 11 | 7 October 2019
III. Which sequential communication strategy is the best?
• Invitation letter:
Send the letter 4 days before (this may be specific for the Austrian Post office). Program a message informing users when the survey starts – in the case they
login too early
• Invitation letter via letter post and email:
Especially at day 0 the highest response rates per day could be reached. CAWI has the potential to achieve timely responses. Always combine the invitation via letter post with one send via email!
![Page 12: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/12.jpg)
www.statistik.at slide 12 | 7 October 2019
III. Which sequential communication strategy is the best?
• Email reminder:
Already one day after start of the survey the response rate decreased. Sent the reminder early enough - Two days after survey start
• SMS/postcard/call/no-reminder:
After the first reminder email the response rates again decreased. A (second) reminder is essential – send it before the first weekend Change of reminder mode (letter post, email).
• End of CAWI survey:
Due to time constraints 9 days should be enough.
![Page 13: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/13.jpg)
COMMUNICATION STRATEGY OF THE CAWI PILOT
![Page 14: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/14.jpg)
www.statistik.at slide 14 | 7 October 2019
IV. How does communication via e-mail effect response rates and timeliness of responses?
• Around 56% (N = 252) of the contact persons provided an email address.
Copy of the invitation & two days later a reminder via email All included the login credentials (user name and password) & direct, click-able link
to access the survey
• Email communication makes a huge difference!
![Page 15: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/15.jpg)
www.statistik.at slide 15 | 7 October 2019
V. Which reminder is the best?
Email of the contactperson available
First reminder
No email available
Second reminder
65 HH
–Group
1
73 HH
–Group
2
88 HH
–Group
3
26 HH
–Group
4
42 HH
–Group
5
25 HH
–Group
8
65 HH
–Group
7
67 HH
–Group
6
77%
47%
57%
56%
43%
63%
22%
21%
CAWI completion rate
451 households (HH) assigned to CAWI
Number of households per reminder group
![Page 16: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/16.jpg)
www.statistik.at slide 16 | 7 October 2019
V. Which reminder is the best?
• Sending a SMS is the cheapest and best reminder
• SMS is a very timely mode - it can be planned more precisely during field work
• Surprisingly the difference between sending a reminder postcard and not reminding (a second time) is very small in numbers. This could be explained by the fact that the postcards were sent out
three (T +3) days after the survey start, usually they would need between one (T +4) and two (T +5) days to arrive at the households address. T+5 was a Saturday!!
• The effects of reminders are similar for households without an email address although on a lower level of effectiveness due to the missing email augmentation and reminder.
![Page 17: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/17.jpg)
GENERAL RECOMMENDATIONS –INTRODUCING CAWI FOR THE LFS
![Page 18: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/18.jpg)
www.statistik.at slide 18 | 7 October 2019
General Recommendations –Introducing CAWI for the LFS
• Send the invitation letter via letter post. The invitation letter should arrive shortly before the survey starts but not afterwards
• Send a copy of the invitation letter as email at the day the survey starts (email augmentation)
• Send the first reminder shortly after the start
• A second reminder is essential – change the mode of reminder. Send the second reminder so that it is received shortly before the weekend
• Do not send each reminder on the same day / time. Consider also a mixture of non- and working days.
• Always remember to try hard to get an email address
• Do not forget to include the login details with invitations and reminders
![Page 19: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/19.jpg)
www.statistik.at slide 19 | 7 October 2019
Questions? Remarks?
![Page 20: Matea Paškvan Introducing a Push-to-Web Sarah Hartleib … · 2019-11-28 · Matea Paškvan. Sarah Hartleib. Statistics. Austria. UNECE, Geneva. 14 – 16 October 2019. slide 2 |](https://reader033.vdocuments.us/reader033/viewer/2022042217/5ec0bd23fe8f8c69dc5764e9/html5/thumbnails/20.jpg)
www.statistik.at slide 20 | 7 October 2019
First WaveCAPI
(550 households)
StartOctober 2018
451 households assigned toCAWI
T -11 days/
T -18 daysPostal invitation letter for CAWI
Start – T0 7/14.01.2019
CAWI Pilot - Second Wave
Copy of the postalinvitation via email
Email of the contactperson available
T +4 days
Registered letter –> routed to CAPI / CATI
First reminder
T +2 days First reminder via email
No email available
T +9 daysCAWI – end of survey
Second reminder
StartJanuary 2019