overview: online surveys vasja vehovar university of ljubljana, slovenia handbook of online...
TRANSCRIPT
![Page 1: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/1.jpg)
Overview: Online Surveys
Vasja VehovarUniversity of Ljubljana, Slovenia
http://WebSM.org
Handbook of Online Research Methods Colloquium 28-29 March 2007
![Page 2: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/2.jpg)
• Computer-assisted survey information collection
• Key methodological issues
• Related issues
• Applications
Structure
Introduction
![Page 3: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/3.jpg)
1930s: applications of probability sampling
1960s: expansion of telephone surveys
1970s: computer technology appears in surveys
1980s: computer-assisted surveying
1990s: Internet mediated surveys
Technology and surveys
Computer-assisted survey information collection
![Page 4: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/4.jpg)
• CATI – Computer-assisted telephone interviewing
• CAPI – Computer-assisted personal interviewing
• CASI – Computer-assisted self-interviewing
Early CASIC interviewer assisted modes
Computer-assisted survey information collection
![Page 5: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/5.jpg)
• Reduced time and costs for data input
• Elimination of errors during data transcription
• Implementation of advanced features:
automatic skips and branching
randomization of questions and response options
control of answers
inclusion of multimedia elements…
CASIC benefits
Computer-assisted survey information collection
![Page 6: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/6.jpg)
Different modes of CSAQ:
disk-by-mail
touch-tone data entry (TDE)
interactive voice response (IVR)
e-mail surveys
web surveys
Computerized self-administered questionnaires (CSAQ)
Computer-assisted survey information collection
![Page 7: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/7.jpg)
Computer-assisted survey information collection
Web surveys
Internet surveys
Online, Internet, Web, CASIC, CSAQ, CADAC
![Page 8: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/8.jpg)
Computer-assisted survey information collection
Web surveys
Internet surveys
CSAQ
Online, Internet, Web, CASIC, CSAQ, CADAC
![Page 9: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/9.jpg)
Computer-assisted survey information collection
Web surveys
Internet surveys
CSAQ
CADAC
Online, Internet, Web, CASIC, CSAQ, CADAC
![Page 10: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/10.jpg)
Interviewer involvement
Survey mode
Paper and pencil CASIC
Interviewer presence
Paper and pencil (face-to-face) interviewing (PAPI)
CAPI, CASI, Audio/Video CASI
Remote interviewer
Paper assisted telephone interviewing (PATI)
CATI, CAVI (computer assisted video interviewing)
No interviewerSelf-administered paper
questionnaires (mail questionnaires)
CSAQ telesurveys (web CSAQ, TDE, IVR, Virtual interviewer,...)
Interviewer-less and paper-less surveys
Computer-assisted survey information collection
![Page 11: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/11.jpg)
Interviewer involvement
Survey mode
Paper and pencil CASIC
Interviewer presence
Paper and pencil (face-to-face) interviewing (PAPI)
CAPI, CASI, Audio/Video CASI
Remote interviewer
Paper assisted telephone interviewing (PATI)
CATI, CAVI (computer assisted video interviewing)
No interviewerSelf-administered paper
questionnaires (mail questionnaires)
CSAQ telesurveys (web CSAQ, TDE, IVR, Virtual interviewer,...)
Interviewer-less and paper-less data collection
Computer-assisted survey information collection
![Page 12: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/12.jpg)
Asking question(INPUT)
Recording of responses (OUTPUT)
Manual recording Automatic voice recognition
Written questions
Standard CSAQ (PC, TV, PDA,
Mobile,...)Visual CSAQ with AVR
Audio questionsVideo CSAQ, Audio
CSAQ, TDE
IVR, Video IVR,TTS with AVR, Virtual
interview
CSAQ input-output technology
Computer-assisted survey information collection
![Page 13: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/13.jpg)
Asking question(INPUT)
Recording of responses (OUTPUT)
Manual recording Automatic voice recognition
Written questions
Standard CSAQ (PC, TV, PDA,
Mobile,...)Visual CSAQ with AVR
Audio questionsVideo CSAQ, Audio
CSAQ, TDE
IVR, Video IVR,TTS with AVR, Virtual
interview
CSAQ input-output technology
Computer-assisted survey information collection
![Page 14: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/14.jpg)
• Developments of the Internet and related technologies
• Importance of broadband Internet access
• Possibilities of distribution across various platforms and devices
• Fast and easy implementation using specialized software tools
Technological aspects
Computer-assisted survey information collection
![Page 15: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/15.jpg)
• Two major problems of probability samples in Internet surveys:
non-coverage
sampling frame problems
• Image of Internet surveys as inherently non-probability ones
1. Probability vs. non-probability sampling
Key methodological issues
![Page 16: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/16.jpg)
Issue of a statistical inference:probability vs. non-probability samples
Internet survey
Information-communication technologies
Mode of survey data collection
Management of the survey process
The spurious link
Key methodological issues
![Page 17: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/17.jpg)
Samplingtype
SolicitationType of invitation
Personal invitation General invitation
Probability samples
Samples from closed population with email,
Pre-recruited with mail, telephone, f2f
Web intercept surveys
Non-probability samples
Self-recruited Internet panels,
Opt-in lists of e-mail adresses
General banner-ad invitation
2. Sampling and invitation
Key methodological issues
![Page 18: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/18.jpg)
• Generally low and variable response rates to web surveys
• Complexity of the response process
• Measures to increase response rates:
incentives (e.g. using PayPal)
multiple follow-up contacts
other motivations
3. Non-response
Key methodological issues
![Page 19: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/19.jpg)
Target population
Operational population
Frame population
Sampled units
Absorbed units
Complete respondents
Partial respondents
Starting units
Clickers
Aware units
Solicitation
Informed units
Involvement process
Key methodological issues
![Page 20: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/20.jpg)
Respondent’s characteristics
Social environment
Researcher-respondent interaction
Survey design
Technological environment
No researcher’s control
Partially under researcher’s control
Participation in web surveys
Key methodological issues
![Page 21: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/21.jpg)
• Question types and visual elements
• Advanced features of computerized questionnaires
• Inclusion of multimedia
• Problems and considerations:
standardization
impact on respondent’s answers
technical problems
measurement error
4. Questionnaire design
Key methodological issues
![Page 22: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/22.jpg)
• Commonly utilized for correcting:
deviations from probability selections
non-coverage and non-response problems
• Importance for non-probability Internet surveys
• Promising approaches:
calibration methods
propensity score weighting
modelling and causal analysis
5. Post-survey adjustments
Key methodological issues
![Page 23: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/23.jpg)
• The managerial process of survey implementation:
costs optimization
data quality
relations between data quality and costs
• Cost effectiveness of web surveys:
costs for increasing response rates
evaluations of costs in relation to errors
1. Costs, errors and management
Related issues
![Page 24: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/24.jpg)
• Web surveys vs. other modes:
lower response rates (Lozar Manfreda et. al., 2007)
comparable or higher level of validity and reliability (e.g. Fricker et al., 2005; Chang & Krosnick, 2002)
2. Survey mode
Related issues
![Page 25: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/25.jpg)
Contact with the respondent
personal, telephone, e-mail, mail,…
Interview
PAPI, CAPI, CATI, CASI,…
Centralized data
management
Solicitation
Mixing survey modes
Related issues
![Page 26: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/26.jpg)
Response rates and behaviour
Modelling mixed modes
Data quality requirements
Willingness to participate
Structure of the data (MAR)
Cost models and impact of environment
Optimal combination
Selecting the right mixture
Related issues
![Page 27: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/27.jpg)
Programing of the questionaire
Related issues
User-friendly interface
Standardized questionnaire
description in QML
Paper questionnaire
CAPICATICASI
Internet survey
![Page 28: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/28.jpg)
• Quantitative and qualitative research methods
• Flexible combinations of approaches (Morgan, 1998):
qualitative preliminary
quantitative preliminary
qualitative follow-up
quantitative follow-up
• New possibilities with the Internet
3. Mixing research methods
Related issues
![Page 29: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/29.jpg)
Qualitative method
Quantitative method
Analysis
Beginning
Conclusion
Flexible on-line mixed methods (FOMM)
Related issues
![Page 30: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/30.jpg)
• New ethical dilemmas in survey research:
solicitation process, spam and privacy
data protection
children and minors
combining data…
• Development of new standards (e.g. ESOMAR, AoIR, MRA…)
4. Ethics, guides and standards
Related issues
![Page 31: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/31.jpg)
• Different applications according to:
type (internet access panels, on-line voting, intercept…)
organizations (official, academic, commercial)
topics (marketing research, polling opinions, health…)
complexity (simple one-shot surveys vs. complex integrated data collection)
Applications
Current and future applications
![Page 32: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/32.jpg)
psychological research
online experiments
paradata and online measurement
specific research fields (usability research, HCI...)
emerging areas (e-learning, e-government…)
Related areas
Current and future applications
![Page 33: Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia Handbook of Online Research Methods Colloquium 28-29 March 2007](https://reader038.vdocuments.us/reader038/viewer/2022110206/56649cf75503460f949c79ee/html5/thumbnails/33.jpg)
• Expansion of CASIC modes, options and mixtures
• Integration of devices (TV, mobile,..)
• Speech recognition and touch-screens
• Multimedia and interactivity
• Integration with other CADAC (GPS, paradata, observations,...)
• Specialisation and professionalisation
Trends
Current and future applications