l 1

Chapter 13

Survey DesignsEDUC 640- Dr. William M.

Bauer

l 2

Key Ideas

History of Survey Research Types of Survey Designs Key Characteristics of Designs Constructing Questionnaires Conducting Interviews Steps in the Survey Research Process Criteria for Evaluating Survey

Research

l 3

A Brief History of Survey Research

1817 International Survey of Education Systems

1890 Stanley Hall survey of children

1907 the Pittsburgh Survey examined social problems

l 4

A Brief History of Survey Research

WWI and WWI sampling techniques improved

Applications began to vary from social science to media

Universities established social science research centers

Private organizations such as Gallup, Rand Corporation, and Roper formed

l 5

Types of Survey Designs

Cross-sectional longitudinal Longitudinal

trend cohort panel

l 6

Types of Cross-sectional Longitudinal Survey Designs

Time of Data Collection

Study Over Time Study at One Point in Time

Longitudinal Cross-sectional

Changesin a

sub-populationgroup identifiedby a commoncharacteristic

over time

Changes in thesame

peopleover time

Trends inthe same

populationover time

Attitudesand

Practices

CommunityNeeds

ProgramEvaluation

Trend Cohort PanelGroup

ComparisonsNational

Assessment

l 7

Key Characteristics of Survey Designs Sampling from a population Collecting data through questionnaires

or interviews Designing instruments for data collection Obtaining a high response rate Designing and using a mailed

questionnaire Conducting an interview survey

l 8

Population, Target Population, Sampling Frame, Sample

The Populationis the group of individuals having one characteristic that distinguishes them from other groups.

The Target Population or Sampling Frameis the actual list of sampling units from which the sample is selected.

The SampleIs the group of participants in a study selectedfrom the target population from which theresearcher generalizes to the target population.

l 9

Collecting Data Through Questionnaires

Mailed questionnaires Electronic questionnaires One-on-one interviews Focus group interviews Telephone interviews

l 10

Forms of Data Collection in Survey Research

Who Completes or Records the Data?

Participant Researcher

MailedQuestionnaire

ElectronicQuestionnaire

One onOne

To aGroup

OverTelephone

IndividualInterview

Focus GroupInterview

TelephoneInterview

l 11

Designing Instruments For Data Collection: Types of Questions

personal attitudinal behavioral sensitive scale open-ended

l 12

Question Construction Problems

Question is unclear Multiple questions Question is wordy Question is negatively worded Overlapping responses

l 13

Question Construction Problems

Unbalanced response options Mismatch between questions and

answers Question includes overly technical

language Not all questions are applicable to

all the participants

l 14

Pilot Testing

Test on a small number of individuals in the sample

Ask for written feedback on the questions

Revise the survey based on the written comments

Exclude the pilot participants from the final sample for the study

l 15

Obtaining a High Response Rate

Mail out original questionnaire with cover letter

Follow it in two weeks with a second mailing of questionnaire to those who have not responded

Follow again in two weeks with a postcard to non-respondents reminding them to complete the questionnaire

l 16

Three Phase Survey Administration Procedure

Step 1:First Mailing

of Survey

Step 2:Second Mailing

of Survey

Step 3:PostcardMailing

2 Weeks 2 Weeks 2 WeeksStart 6 Weeks

Time

l 17

Planning and Designing a Mailed Questionnaire

Write a cover letter to invite the participants to complete the questionnaire

Form and construct the questionnaire

Identify what statistical procedures will be used to analyze data from the mailed questionnaire

l 18

Conducting an Interview Survey

The interviewer should maintain a neutral stance during the interview

Train the interviewers prior to the interview

Take good notes of responses or use a tape recorder

For telephone interviews, develop a telephone interview guide prior to the interview

l 19

Steps in Conducting Survey Research

Decide if a survey is the best deign to use

Identify the research questions or hypotheses

Identify the population, the sampling frame and the sample

Determine the survey design and data collection procedures

l 20

Steps in Conducting Survey Research

Develop or locate an instrument Administer the instrument Analyze the data to address the

research questions or hypotheses Write the report

l 21

Criteria For Evaluating Survey Research

Was the target population or sampling frame clearly specified?

Was the sampling procedure specified? If a random sample was not used, were modifications from it explained?

Was the sample clearly identified and the basis on which it was chosen specified?

l 22

Criteria For Evaluating Survey Research

Did the type of survey match the questions or hypotheses advanced by the author?

Was it clear whether a questionnaire or an interview survey comprised the form of data collection? Did the researcher identify the basis for selecting or developing the instrument?

l 23

Criteria For Evaluating Survey Research

Was information reported on reliability and validity of scores from past use of the questionnaire or interview?

Did the author mention the dates on which the questionnaire or interview was administered?

Were the administration procedures clearly identified?

l 24

Criteria For Evaluating Survey Research

If a questionnaire was used, were the procedures for obtaining responses identified? Were follow-up procedures used to ensure a high response rate?

l 25

Criteria For Evaluating Survey Research

If an interview was used, were the field procedures for collecting this information specified? Were demographic characteristics, previous experience, training, and monitoring discussed for the interviewer(s)?

Did the data analysis match the research questions or hypothesis to be answered in the study?

l 26

Criteria For Evaluating Survey Research

Did the researcher check on the reliability and validity of scores from the data collection in the study?

Was the study written scientifically and ethically?

l 27

Applying What you Have Learned: A Survey Study

Review the article and look for the following: The research problem and use of quantitative

research Use of the literature The purpose statement and research hypothesis Types and procedures of data collection Types and procedures of data analysis and

interpretation The overall report structure