1 sampling methods for web and e-mail surveys associate professor ron fricker naval postgraduate...
Post on 21-Dec-2015
214 views
TRANSCRIPT
1
Sampling Methods for Web and E-mail
SurveysAssociate Professor Ron Fricker
Naval Postgraduate School
Monterey, California
2
Today Anyone Can “Survey” for Little Cost
3
Internet-based Surveys and Sampling
• Focus is on survey data collection to support statistical inference
• Internet-based surveys often conducted via multiple media and mixed (survey) mode– “Contact mode” – respondent
contact medium– “Survey or response mode” –
survey medium– “Follow-up mode” –
non-response follow-up medium
4
Bigger Samples Are Not Always Better
• When conducting Internet-based surveys, temptation is to (attempt to) survey everyone. Why not? It’s cheap and easy…
• Example: Survey2000– Goal: Quantify how often people have moved,
what role they play in their communities, and how geography has shaped their tastes in food, music, and literature
• More than 80,000 surveys initiated and 50,000+ completed
5
Types of Internet-based Surveys
Sampling Method Web E-mail Probability-based Surveys using a list-based sampling frame Surveys using non-list-based random sampling Intercept (pop-up) surveys Mixed mode surveys with Internet-based option Pre-recruited panel surveys Non-probability Entertainment polls
Unrestricted self-selected surveys
Surveys using ‘Harvested’ E-mail Lists (and Data) Surveys using volunteer (opt-in) panels
Table 2. Types of Internet-based surveys and associated sampling methods.
6
Internet Surveys Pose Significant Challengesfor General Populations
• Internet users do not reflect the general population– Frame and coverage bias
• Very hard to impossible to generate a sampling frame– There is no master list of e-mail addresses
• Recruiting respondents via the web can introduce bias
• Unethical/illegal to solicit survey participation via e-mail (spam)
7
Unsolicited E-mail is Spam
3. Internet Research a. The unique characteristics of internet research require specific notice that the principle of respondent
privacy applies to this new technology and data collection methodology. The general principle of this section of the Code is that survey research organizations will not use unsolicited emails to recruit respondents for surveys.
(1) Research organizations are required to verify that individuals contacted for research by email have a reasonable expectation that they will receive email contact for research. Such agreement can be assumed when ALL of the following conditions exist.
a. A substantive pre-existing relationship exists between the individuals contacted and the research organization, the client or the list owners contracting the research (the latter being so identified);
b. Individuals have a reasonable expectation, based on the pre-existing relationship, that they may be contacted for research;
c. Individuals are offered the choice to be removed from future email contact in each invitation; and,
d. The invitation list excludes all individuals who have previously taken the appropriate and timely steps to request the list owner to remove them.
(2) Research organizations are prohibited from using any subterfuge in obtaining email addresses of potential respondents, such as collecting email addresses from public domains, using technologies or techniques to collect email addresses without individuals’ awareness, and collecting email addresses under the guise of some other activity.
Figure 2. Excerpt from Council of American Survey Research Organizations
Code of Standards and Ethics for Survey Research, accessed online at
www.casro.org/codeofstandards.cfm on September 30, 2006. Highlighting added.
8
Problems Mitigated for Some Populations
• Internet-based surveys within individual organizations often easier– List of e-mail addresses or common e-mail
address syntax often available– No problem with e-mail as spam
• But issues can remain– How complete is the e-mail address list?– Do all respondents have access to the Internet/
e-mail?– Non-response issues and bias
9
Summary
Sampling Strategy
Probability-Based Non-Probability-
Based
Li
st-b
ased
sam
plin
g fr
ames
Non
-list
-bas
ed r
ando
m s
ampl
ing
Sys
tem
atic
sam
plin
g
Mix
ed m
ode
surv
ey w
ith I
nter
net-
base
d op
tion
Pre
-rec
ruite
d su
rvey
pan
el
Ent
erta
inm
ent
polls
Unr
estr
icte
d se
lf-se
lect
ed s
urve
ys
Vol
unte
er (
opt-
in)
pane
ls
Web
Inte
rnet
-ba
sed
E-mail Telephone Postal Mail
Co
nta
ct
Me
tho
d
No
n-I
nte
rnet
-B
ased
Other: TV, print advertising, etc.
Table 3. Sampling strategies for Internet-based surveys by contact mode.