Chapter 17

Introduction to Survey Research

Surveys – why a survey?• Surveys are conducted to describe the

characteristics of a population. Examples of characteristics

DemographicPoliticalSocial issuesEducation issuesMarketing/Advertising

Associations between variables are often sought (e.g. correlational studies).

Major types of surveysCross-sectional

– Collected at one point in time– From members of the population across one (or

more) given characteristic (e.g. age, race, grade level)– Obviously, since one point in time, one sample is

needed

• Longitudinal– Collected at more than one point in time from the

same population– Not necessarily the same sample – might be, might

not be

Longitudinal studies • Collecting data (surveying) at more than one point in

time– Trend , Panel, Cohort

Differ along the dimensions of sampling from the population and the exact nature of that population. The Target Population remains the same.

Trend – Members of the population change – new sample from the population taken at each different point in time

Cohort – members of the population do NOT change - new sample taken at each different point in time

Panel – Population may or may not change – the same exact sample is used at each point in time

Steps to conducting a survey• Define or Identify the problem –

– Articulate the purpose. What is the research question? (What are 4 char of a good RQ?)

– Purpose or research questions can be further developed into objectives. Articulate each objective. (Book calls this “hierarchical set of RQ’s)

• Identify your target population– Is it clear? Can the members be easily identified, delineated?

Accessible?

• Select a mode of data collection– Mail? Phone? Mall clipboard? Interview? Direct administration to a

group?

• Prepare the questionnaire• Collect the data• Compile, then Analyze, then Interpret.

Mode of Data Collection• Direct Administration to a group

– Simplest, easiest, highest return rate, typically done in colleges

• Mail Surveys – Relatively cheap, time consuming, low-return rate– Use incentives to increase rates

• Telephone surveys– Relatively expensive, intensive-short time duration, effective &

efficient techniques, training, phone bank needed– Cell phones starting to make a negative impact

• Interviews– Time consuming, more in-depth responses, training, costly (labor)

Advantages and Disadvantages of Survey Data-Collection Methods

DirectAdmin Telephone Mail Interview

Comparative cost Lowest Same Same High

Facilities needed? Yes No No Yes

Require training of questioner? Yes Yes No Yes

Data collection time Shortest Short Longer Longest

Response rate Very high Good Poorest Very high

Group administration possible? Yes No No Yes

Allow for random sampling? Possibly Yes Yes Yes

Require literate sample? Yes No Yes No

Permit follow-up questions? No Yes No Yes

Encourage response to sensitive topics? Somewhat Somewhat Best Weak

Standardization of responses Easy Somewhat Easy Hardest

Example of an Ideal vs. an Actual Telephone Sample for a Specific Question

Advantages and Disadvantages of Closed-Ended vs. Open-Ended Questions

• Enhances consistency of response across respondents.

• Easier and faster to tabulate.• More popular with respondents.

• Allows more freedom of response.• Easier to construct.• Permits follow-up by interviewer.

• May limit breadth of responses.• Takes more time to construct.• Requires more questions to cover

the research topic.

• Responses tend to be inconsistent in length and content across respondents.

• Both questions and responses subject to misinterpretation.

• Harder to tabulate and synthesize.

Advantages

Disadvantages

Close - ended Open- ended

Close - endedOpen- ended

Non-respondents

• Question to answer: Are the respondents different than the non-respondents.

• You hope not.• How do you know?

– Know the important characteristics of the population– Collect data about these important characteristics– Compare the data from the respondents to the

population– If the characteristics of respondents = population:

Good!– If the characteristics of respondents differ from the

population : Bad

• Some basics about conducting a mail survey– Use an incentive– Use 2-3 mailings– For anonymity, use coding– Cover letter (contains informed consent)